I don't think sarongs are the problem, I think the problem is that Str and Dex wearers don't have equivalents to fill in that gear slot.

It's honestly kind of odd, but itemization in this game is kind of odd anyway.