This involved something relatively recent and something that I was very, very familiar with. What I realized after the fact was that I had no trouble at all remembering all the details of the game. Even though I couldn't place the Prince of Thieves, once I did I remembered every twist and turn in the sewers of Freehaven and where every rat and cleric were located. Same with Gharik's Forge - I could see every little bit of it in detail once I realized which dungeon it was. And there are lots of other examples - I can 'see' the details of every zone and every dungeon even though I may not recall their names immediately.
So it was just the names that escaped me (temporarily, usually), and in some ways those are not essential to playing the game. I guess if the visual details of Enroth start to escape me I'll be a little more worried.