I have 6 GBs of RAM and a GTX 570 on Win 7 so my RAM is capped at around 4GBS
It occurred to me: doesn't the GTX-570 come with 1.25GB of VRAM (more in some editions)?
Being a 32-bit game, your available main menory and graphics memory can't be more than 4GB in total (as you're obviously already aware). Maybe your graphics card is eating up too much memory for storing high-textures in main memory?
Hopefully not - with a card like that you'd definitely want maximum detail!