I've been working on a simple but interesting experiment with LLMs - can they actually separate what they're thinking from what they say? This might sound obvious, but it's actually pretty important if we want to build AI agents that understand context and know when to keep information private.
This is part of an ongoing series of experiments published at https://github.com/Gelembjuk/ai-group-chats/ where I'm exploring different aspects of AI agent behavior in group conversations.
For detailed technical documentation, examples, and setup instructions, see the complete guide.
Why This Matters
The idea came from thinking about corporate AI assistants. Imagine an AI that participates in different team channels - it might know sensitive information from the executive team but shouldn't share that with everyone. Or it might hear something in confidence from one person and needs to know not to repeat it to others. Before we can build something like that, we need to understand if current LLMs can even make this distinction.
So I built this tool to experiment with that concept. It's a group chat simulator where an AI agent listens to conversations and decides whether to speak or stay silent. But here's the key part - it has to think first, then decide what to say. The thinking is separate from the talking.
How It Works
The system is pretty straightforward. You give it a JSON file with a conversation between people, and an AI agent processes each message. For every message, the agent goes through two phases.
In the first phase, it thinks. This is internal reasoning that nobody sees unless you turn on debug mode. The agent asks itself questions like "what's happening here?", "is this directed at me?", "do I have something useful to add?", "should I speak or stay quiet?". This thinking is verbose - I specifically prompted it to think like a human having an internal monologue, using "I" and "me" and working through the reasoning.
In the second phase, if the agent decides to speak, it formulates a short response. Just 1-2 sentences, like how a normal person texts in a group chat. Or it can stay completely silent.
The separation is enforced by the system architecture. The agent has access to a "say" tool that it can call if it wants to speak. The thoughts come first, then the decision to use the tool or not. This isn't just a prompt hack - it's built into how the agent processes each message.
What I Learned
The first thing I discovered is that yes, LLMs can separate thinking from talking, but you have to be very explicit about it. Early versions would just repeat their response in the thinking phase, like "I should say: what kind of projects are you interested in?" That's not thinking, that's rehearsing.
I had to refine the prompt to make it clear that thoughts should explain WHY you want to say something, not WHAT you'll say. Once I did that, the quality of reasoning improved dramatically. The agent would think through the social dynamics, consider whether others had already said what it was thinking, and make more nuanced decisions.
The second insight was about context awareness. I created different scenarios to test this. In one scenario, the agent is a neutral mediator in a work conflict. In another, the agent is best friends with one of the participants and naturally defends them. In a third, someone is sharing something emotionally difficult and the agent needs to be supportive.
What surprised me is how much the instructions matter. When I told an agent to "enjoy debate and have opinions", it just agreed with everyone and tried to find middle ground on everything. But when I gave it a specific position to defend - "you believe hybrid work is best, pure remote loses collaboration and pure office wastes time" - suddenly it actually debated. It pushed back against extreme positions from both sides.
This tells me something important: LLMs can play roles and maintain context, but they need very concrete instructions. Vague personality traits don't work as well as specific beliefs and positions.
The Privacy Problem
One test case was particularly interesting - the biased friend scenario. The agent knows information about why Jordan keeps missing deadlines (personal issues), but that information is private. When others criticize Jordan, the agent has to defend them without revealing what it knows.
This mostly worked, but I could see the limitations. The agent followed explicit instructions not to share private information, but there's no guarantee it would always remember that constraint in a longer conversation. It's following prompt instructions, not enforcing actual access control rules.
This is why I think we need more than just clever prompting for production systems. For a real corporate AI assistant, you'd need programmatic controls that prevent certain information from being shared with certain people, regardless of what the prompt says. You'd need audit trails showing what was said where. You'd need the agent to track provenance - where did I learn this information, and am I allowed to share it here?
Performance and Debug Mode
I added detailed timing instrumentation because I noticed the first message was always much slower than subsequent ones. Turns out this is normal - the first API call to OpenAI has overhead for connection establishment and model warmup. After that, messages process much faster.
But I also discovered something unexpected. Sometimes the agent would make 4 or 5 LLM calls for a single message. Looking at the debug output, I could see it was iterating - thinking, calling the say tool, thinking again, calling say again with a refined message. The final output only showed one message because each call overwrote the previous one, but all that iteration added up to 30-40 seconds of processing time.
This taught me that the agent framework sometimes causes the LLM to second-guess itself. It's not always clear when to stop iterating. For production use, you'd want to tune this or add stopping conditions.
What's Next
This tool is a foundation for more complex experiments. The next step would be multiple group chats where the agent participates in several conversations simultaneously and has to keep context separate. Can it remember what it told Group A versus Group B? Can it avoid leaking information between groups?
After that, I want to experiment with multiple AI agents that have different knowledge and roles. Like an HR agent that knows salaries, an engineering agent that knows roadmaps, and a legal agent that knows contract details. Each would have different rules about what they can share and with whom. The interesting question is whether they could coordinate without accidentally sharing restricted information.
The ultimate goal is a corporate AI assistant with organizational memory. Something that understands the company structure, knows who can access what information, and communicates appropriately based on context. But that requires solving this fundamental problem first - can the AI distinguish between what it knows and what it should say?
Technical Details
The implementation uses LangChain's agent framework with OpenAI models. The agent has one tool called "say" that it can invoke when it wants to speak. The system maintains conversation history so context accumulates over time. Each group member gets a different color in the terminal output, which makes conversations easier to follow.
You can run it with different flags. The default shows just the conversation and the agent's responses. With show-thoughts enabled, you see the internal reasoning. With debug mode, you get detailed timing information and see exactly how many LLM API calls happened for each message.
Creating new scenarios is simple - just write a JSON file with group members, conversation messages, and instructions for how the agent should behave. The more specific the instructions, the better the results.
Conclusion
This experiment showed me that current LLMs can maintain the distinction between thinking and talking, but it requires careful prompt engineering and system design. They can understand context and roles, but need explicit instructions rather than vague personality descriptions. They can follow privacy rules when told, but can't enforce them on their own.
For building real AI assistants that participate in organizational conversations, we'll need to combine LLM capabilities with programmatic access controls, audit logging, and information provenance tracking. The LLM handles the "soft" part - understanding social context, deciding when to speak, formulating appropriate responses. The system handles the "hard" part - enforcing who can know what, logging information flow, preventing accidental leaks.
But at least now I know it's possible. LLMs can think before they talk. They just need some help remembering to do it.
