You can certainly complete the game that way. Your character also has magic-like abilities, so he'll not be entirely inept at magic.