Wasn,'t it in act 1, at the celebration party when you've gone away privately, Shadow asks you to tell something about yourself and your history. After this, you can ask her to say something about herself. She says she can't really say anything, and then you respond that she likes night orchids and can't swim. Something she said earlier. She's glad that you remembered that.

Maybe you need to have had that dialogue to trigger the swim scene later.
And probably also have to give her the night orchid.

In my swimscene, BTW, we weren't skinny dipping. Both kept their underwear on.