i think its because areas get different maps when you return, so for example 'camp' is actually dozens of maps that look 'more or less' the same... this is also why the dead stand back up and some scripted animations [pets running around etc] bug out so much
Larian is aware of it and must have a to do list but that will not be fixed until all the different patches are finished imo
