sounds painful to balance but it should be a case where its one positive and a negative e.g. -2 strength but + 2 finesse on plate armor

Rogues are the best