I can argue that Blind is more of a case for balancing spells AP costs, cooldowns, granting bosses higher saves or occasionally outright immunity.
Your claim about Teleport is exaggerated. I made a 5 Int and 10 Int char. the 5 INT char's Teleport had a 10 turn cooldown. The 10 INT char's Teleport had a 7 turn cooldown. So you are right that there is an undocumented cooldown reduction for points into INT.
If 5 Attribute points only gives a 3 turn cooldown reduction, then you'll need 15 INT to reduce the teleport cooldown to four turns, and 20 INT to cast it every other round. I don't believe it'll even be possible to even get as high as 20 INT. To get close to that high, you'll have to sacrifice all other attribute points, which means you'll be very slow and have only the base HP, which will leave you very vulnerable to being hit - especially if you take Glass Cannon to compensate for your low speed.
I'm currently playing a lvl 2 character with 14 Intelligence. It won't be that hard.