Now that I think about it, it may well be a dev time cost that they didn't want to spend. If just the PC can be short, then you only have to program for male and female scenes with at most one short (or tall for Dragon born) partner in one configuration. If you include both extra sizes as NPC relationships you just tripled the number of scenes you have to account for. That's a lot of dev work.
In addition to just short-normal, normal-normal, and tall-normal; you have short-short, short-tall, normal-short, normal-tall, tall-short, and tall-tall.