Sure you can artificially boost your Perception using items, or recruit a Henchman exclusively for the purposes of spotting the button, but you'd only know to do so if you were somehow forewarned that it would be necessary.
But you were. You would have to be blind to don't spot the passage in Witch's Cabin. And if you know passage is here, but you can't open it, and can't find anything helpful near, it's kinda obvious solution to use some perception potions/gear. At least it was obvious for me.
