If Game Mode is just "nice-ing" background other tasks, that would make difference only if there were other processes competing for resources, no?

I look at the command buffer and see GPU commands that are taking forever—not because resources are sitting idle, but because some of them are stupid heavy.