Hey Epimetus, I am not entirely sure, if opening the gate via the key triggers anything important.
If that is not the case you could use the cheat engine table by atom0s. Increase the players speed to the point where you can traverse walls to enter the garden.
Main post:
https://forums.larian.com/ubbthreads.php?ubb=showflat&Number=665942Where to find the download now:
https://atom0s.com/posts/2023/2023-08-04-downloads-moved/