Wyll is going to do mediocre damage with weapon's even with blade pact because his stats don't line up properly. This can be fixed if it is possible to move him to hexblade, but without that his stats are just not set up for swording.