Maybe this has already been mentioned, but the Expansion mod on nexus adds exactly such a toggle button, and it works perfectly. I hope Larian implements this as well some time, because it obviously works.