Yeah, while spamming Vicious Mockery is super fun in the demon battle on the ship I don't know how useful it is at 3rd level when you get pact of the tome even with hex to make it hard to resist. Shillelagh might be more handy (and makes up for not choosing pact of the blade a bit).
I think overall it is hard to assess pact of the tome without knowing what Book of Ancient Secrets does if it is in or how ritual magic works (we know Ritual Caster feat is in). It could end up being super powerful if ritual magic is.