Alright, that's what I feared, this instalment seems to have far more missable secret rooms as compared to Millennium 2, and I also blew up the Orc fortress before reading that I didn't need to do so.
Thanks, at least I still have the opportunity to loot the 5th room.