If you give the fancy mediums a +1 or +2 additional dex bonus then medium armor mastery should be changed to add another additional +1 (it may do that already, not sure whether its additive, or just sets the bonus to +3). +2 normal, +2 from special armor, and +1 from feat gets you to full bonus from 20 dex.
I do still think heavy armor should be buffed a bit too, to compensate for the (likely) low initiative.
