Hmmm....I did that, and nothing happened. I'll try again. Maybe it matters who is in the party.
edit: I just tried again, and all the merchant dwarf does is say, "I hope the Devil Witch won't be back again."
edit #2: I just discovered that
you have to leave the town, and then re-enter, and then talk to the merchant again.