The memory stat is specifically so that you have to choose between power and versatility. Well, with all the 3-turn cooldowns, it's more than just versatility. Generally any ability is going to be more powerful than a normal attack, especially for mages. If you based memory slots off of Intelligence, then any caster would just dump all points into Intelligence without question, and at that point, why even give a choice of where to put stat points?
That said, I do agree with your point about source skills. I haven't found them to be powerful enough to justify 3 memory slots even without the source point cost. Maybe that'll change in Act 2+, but in Act 1 I just don't find enough value in any source abilities that I didn't get from equipment.
