I was having this problem almost as soon as I started Driftwood, and it kept getting worse and worse, but I slogged through it anyways. It got better after I cleared the act and moved onto the next zone, so perhaps it's some consolation that whatever the issue is isn't permanent? I'm now an act farther than that and my quicksaving and quickloading are working just fine.
Curiously, the bloating file size thing hasn't changed (my last one was 20.5 mb), but the problems with loading and such have gone away.
