Longtime tabletop D&D player here. It depends on the personality and player-type that you are with but most DMs i’ve played under prioritize things like story, pacing and having fun over slavishly following any rule books. At least all of the great DMs in my opinion operate this way. It even says so in all the DM books.
There is something to this, yes. Slavishly following the rules can bog the game down and even cause issues, turning the game into something not fun.
That said, there is an extreme as well. Start veering too much from the rules and the game becomes too imbalanced and ruins it. Also, players tend to like it when they legit win; not winning because a DM fudged the rules and made it so they win no matter how they play.
The real challenge of a good DM is to find that perfect balance between playing by the rules and tweaking them.
Taking away feats would make ASI boring, IMO with absolutely no other options to make your characters more special. I think MORE feat options would be better, options that would make taking a feat a real temptation over ASI.
Think higher levels. If I create a barbarian with 17 Strength, and at level 4 give him ASI +2 Strength. Now he's 19. At level 8, ASI again, but now 20 is max. So, 1 goes to Strength. Then ASI what? Constitution or Dex maybe? Okay. That's fine. Level 12. Do it again. No real deviation or variation. All characters who are fighters/barbarians, you'll probably do the exact same thing. No difference between Lae'zel and your fighter. Both pretty close to same stats.
However, with more feats, Lae'zel might take Great Weapon Master while my custom fighter might choose dual wielding and wield to battle-axes effectively in combat with +1 AC. Or maybe my fighter will do Shield Master and shove enemies as a Bonus rather than Action (like it should be instead of everyone shoving as bonus). Whatever. The point is, you can with feats have different characters even of the same class with unique abilities instead of just always building up the same stats every 4 levels.
More feats that are more effective, is what I think is needed.
