I've found that BG3 doesn't always properly take into account the slowing effects of difficult terrain. Any time I had to walk through Hunger of Hadar, the game would show that my toons expect to reach point A, but wouldn't actually reach that point when I clicked "go".