Hey Elisii, I can understand your frustration. I also play on xbox serie X, but i have to admit that i don't have that much problem to play. I have bugs and crashes but not as many as you mention.
On my case, i noticed that maybe the Quick resume functionality of the xbox may cause some trouble with the game (I also have this issue with FFXV). Every time i don't completely shut down the game manually after a session, I have issues (bug or crash). So now I always close the game manually after every session. That's the only tips I can give you unfortunately. I hope this will help a little.
Also i noticed that the VRR can cause issues, did you try to disable it ? On one of the pinned topics of the technical & Gameplay problem section, you have a topic called "Radial Wheel Lag/Freeze - Workaround" maybe try what they say. VRR cause a lot of various issue in many games. So you can try to turn off Variable Refresh Rate (VRR), disable Performance Mode. To disable VRR on your Xbox (X or S), navigate to Settings > General > TV & display options > Video modes > Variable refresh rate and select Off.
Let me know if this help, it could be good news!