The screenshot from Pathfinder was a good example of a way to help alleviate the issue. I think some sort of codex with brief blurbs like the example would help a lot. Common knowledge should be there from the start, and more niche stuff could get unlocked as it comes up. It doesn't need to involve changes to spoken dialogue, but perhaps as certain things get mentioned, a little notification could pop up, something like: "[New/Updated] Codex entry: Waterdeep", for instance that players could freely ignore, or read if they really do want to know more. A lot of the lore already exists, which could save time creating it, although deciding what goes into the entry could be a lot of work, not to mention finding all points in conversations where things come up, so it would still be a significant amount of work.
Yes, like a very minimal notification (with option to turn of in gameplay menu's off course) appearing when a new 'thing' is mentioned and added as a codex entry would be nice. Especially if they could integrate it with the sticky/hover tooltip thing. For instance 'waterdeep' is mentioned, you get the notification and either click T on 'waterdeep' in the dialogue or click the notification to get the codex entry for waterdeep pop up. With tooltip implementation, you could continue browsing the codex for terms you want more info on. Also please make it a nice looking codex, take inspiration from previous BG manuals or phb's to ensure there is enough 'flavour' to it , slap in some nice artwork in there too alongside some 'hints' regarding legendary items or other quest related stuff relevant to each entry. Preferably it reminds the player of a physical old codex and not some flashy 21st century wiki.