Pretty confident from that this is related to the shared turn in initiative pairs from watching it happen a few times.
Using stadia controller on TV I also some times miss turns when I press B to go back after selecting a spell and the interface reacts like I am holding B and skips my turn too. I am starting to get anxiety about touching the B button at all! 🤦🏻‍♂️