I think controller play is in early early access.
But I'm experiencing the same issues on stadia.