Having saved games in the game folder makes it easy to uninstall since all you have to do is delete that folder.
And the registry keys. And the services. And the shared files. And the components in the assembly cache. And the policies. And whatever else the program has installed in this modern age of multi-user, multi-process, secure operating systems.
To put it simply, programs (.exe, .dll, etc) go in C:\Program Files. My documents (including savegames) go in <myprofile>\Documents. My settings go in <myprofile>\Appdata. It doesn't matter whether we are talking a game or a productivity app - they should all be consistent.
