There are already bugbears and a hobgoblin. Bugbears and Hobgoblins would not change the story at all. Another ogre fight wouldn't be strange at all.
It's not that difficult, D&D 5e is designed with the idea of being able to add in monsters on-the-fly.