I definitely do not want to see Berserkers throwing Fireballs and casting Blur and Mirror Image from a scroll.

I wouldn't mind exact 5e rules with scroll use. At the very least the class restriction needs to apply.