3-5 turns ruins its potential on rogue so if we are talking about limitations make it so you can only swap a maximum of 1-3 times per encounter. I don't mind it being a talent though it really limits rogues on talent choices early game since they already have to get backstab.
