It depends on the game -- like Sacred 2 will ask you if you want to use the PhysX mode or not, and tells you that if you use the Phys-X mode, it may slow down the 3D graphics -- and it does -- very noticeably. The phys-x graphics look really n ice, but they made that game very slow, so I'm glade it had a toggle.
So turning the phys-X on or off is something that was present IN the GAME -- not just on the Nvidia panel.
You say it's is just as much a part of the game as is the lighting -- but you can turn various parts of the lighting on or off in the game -- so you disprove your point -- i.e. just as you can turn off Screenspace anti-aliasing, or high-quality geometry or shadows, you would have an option to turn off physX usage if physX usage was a part of the game.
I don't think physX usage is used in this game -- because there is no toggle for it. All games that have used phys-X processing have had a toggle for it because it creates a big impact on 3D processing when it's on. So it depends on what card you have and what other graphics features you have, as to whether or not you want it turned on.