I think load times are just part of the game (there were similar issues with Oblivion and Fallout 3 which were based on the same 'Gamebryo' engine).

I'm running the PC version on a 6-disk RAID-0 partition (6* faster for reads & writes than a standard 7200 rpm disk) and I sill wait up to 5 seconds on transitions.