On a d4 the average roll goes from 2.5 to 3 when you reroll 1 and 2 (once).
Calculation: 0.5 X (3+4)/2 + 0.5 X (1+2+3+4)/4 = 3
On a d6 the average roll goes from 3.5 to 4.1666 (similar calculation).
This is of course better than nothing. But the opportunity cost is +1 AC whenever you wear armor, in a game where AC is hard to come by. GWM by raw is not worth it (IMO). The larian version (bugged or not) might be, but I don't think even then that its a no brainer. +1 AC is pretty strong.