I've been back to this letter so many times my head is spinning, so it's possible I'm not remembering correctly, but it seems that the first time I tried, I got the confused balloon right away, later I got another type of balloon and I thought I was actually going to complete it, but, no, the confused balloon again. Again, my mind may be playing tricks on me. However, I do know for certain that there are certain areas in the game where you can see the relationship between Busho and Lilith grow. There are certain dialogues where he seems to be singing her praises and she his, even fawning all over each other. While you're in the Fay area, try to enter some of the other areas. You can't do it, but you will get some interesting dialogue. In particular, try to enter Busho's tree home. Maybe we have to find enough of these romantic (don't get your hopes up, I said romantic NOT erotic) encounters to be able to finish the letter. So, I'll list 3 of these encounters here.
Renumbering
1-Conversation between Caius and Busho when sleeping at inn Elven outpost (Thank you, Lilac. See Lilac's message below)
2-Attempting to open the 'formidable' chest in the Ice Shrine
3-Attempting to lift the sword from the stone on the crescent island
4-Attempting to return to Busho's tree home outside of Fay
Maybe it's connected? Maybe it's not.... But it's worth a try because those buffs didn't seem to make a darn difference.