I always homeruled the same thing at my table, it has never been an issue luckily.
I’m a DM and constantly forget that myself. It’s not until afterwards that I realize that the player couldn’t do what they had done but then it feels cruel to roll back too far so I let it slide. I wouldn’t be surprised if this rule isn’t commonly broken in a similar fashion.
