So a simple idea I’m trying out for my most recent d&d campaign is as follows:
1. Every player has a public goal, which is one of the big reasons they’re traveling with the party. It’s either in the party’s best interest to pursue the goal or the goal is appropriately noble. Example: Create a center of learning that will draw people from throughout the land!
2. Every player also has a private goal, something that is probably secret and that they don’t really need the group’s help to accomplish. In fact, such a goal might even run contrary to the group’s goals. Example: Become a God!
3. The players are all united by a few principle goals that I set before them (assuming they care about them at all). Example: Discover the mystery as to why the the boundaries between the planes are breaking down.
4. Because I’ve been really annoyed in the past at having to separate in game knowledge from out of game knowledge I’m instituting a rule where if some piece of information comes out at the gaming table, anyone can invent a reason why they might have figured it out. This is a little silly, but is my best guess for how to actually preserve a sense of intrigue at the table. So if someone is about to have an important conversation, I ask them if they want to have it in another room away from earshot of other players. And people are encouraged to slip me notes and the like.
5. Players all have quests and minor quests associated with their goals (which give XP to the entire group, naturally). Thus, mechanical reasons to pursue your goals!
The main goal is to encourage rich interwoven storytelling where what might seem like a straightforward adventure like exploring a cool tower actually has several layers of subtext. Plus it’s fun in D&D to have a personal sense of accomplishment that goes beyond the mechanical benefits of leveling up and the mundane satisfaction of saving the world yet again!