With my characters at level 15 in the act 2 BF, the level 9 mutants and mages give 761 experience; on level 16 they give 299 experience, and on level 17 give zero.
BTW, the relative experience given at different character levels was the same for two different monsters I checked. With the characters about 3 levels below the opponent, the experience increases exponentially as the character level drops. At about 3 levels higher than the opponent up to 7 levels higher, the experience decreases linearly, until at 8 levels higher no experience was given.