Oh I fully support this.
Finding that grey hidden button on the backside of a pillar after a perception check drove me crazy a few times.
"I know it's in here... somewhere..."