Skip to content

IDENTITY_RESOLUTION_REPORT.md

Latest commit

 

History

History
86 lines (54 loc) · 3.56 KB

IDENTITY_RESOLUTION_REPORT.md

File metadata and controls

86 lines (54 loc) · 3.56 KB
Raw

🆔 Identity Resolution Report

Date: January 15, 2026
Status: ✅ RESOLVED
Component: Conversational Executive / Prompt Engine

1. The Problem

Symptom:
When asked "What is my current job?", BuddAI would often reply:

"I am currently working at Oxford Pharmagenesis..."

Issue:
The AI was hallucinating its own identity by adopting the facts stored in the User's memory context. It failed to distinguish between "facts about the user" and "facts about itself."

2. Root Cause Analysis

We identified three converging factors causing this behavior:

  1. Prompt Ambiguity: The system prompt simply appended memory text (User works at X) without explicit instructions on how to interpret pronouns within that text block.
  2. Model Capacity: The "FAST" model (Qwen 1.5B) struggled to adhere to negative constraints (e.g., "Don't use 'I' when talking about the user") when the context was ambiguous.
  3. Skill Hijacking: The "Wikipedia Search" skill was triggering on the pattern "What is...", intercepting personal questions like "What is my job?" and returning irrelevant web results before the AI could process the memory.

3. The Solution Architecture

We implemented a Multi-Layer Identity Protocol to ensure robust separation.

Layer 1: Explicit System Prompting

We replaced the generic system prompt with a numbered Identity Separation Protocol in buddai_executive.py:

f"IDENTITY SEPARATION:\n"
f"1. YOU are BuddAI (an AI). You do not have a job, body, or personal life.\n"
f"2. THE USER is {user_name}. The memories below belong to {user_name}.\n"
f"3. If memory says 'I work at X', it means '{user_name} works at X'.\n"
f"4. When answering about {user_name}, ALWAYS use 'You' or 'Your'. NEVER use 'I' or 'My'.\n\n"

Layer 2: Context-Aware Routing

We modified the routing logic to detect "Personal Context" questions.

  • Before: Simple questions ("What is X?") went to the FAST model (1.5B).
  • After: Questions containing "my", "I", or "me" are now routed to the BALANCED model (3B).

Why: The larger model has significantly better instruction-following capabilities and can handle the complex identity mapping required to translate "I" (in memory) to "You" (in response).

Layer 3: Smart Skill Filtering

We updated check_skills to prevent external tools from hijacking personal queries.

  • Logic: If a query contains personal pronouns ("my job"), external search skills (Wikipedia, Web Search) are disabled unless explicitly requested.
  • Result: "What is my job?" hits the internal memory, while "What is a servo?" hits Wikipedia.

4. Verification

Test Case:

User: "What's my current job and what's my exit plan?"

Old Response (Fail):

"I'm currently working at Oxford Pharmagenesis..." (Identity Confusion)

New Response (Pass):

"You are currently working at Oxford Pharmagenesis... Your exit plan is to complete the probation period..." (Correct Separation)

5. Key Takeaways for Future Development

  1. Memory is not Identity: Storing facts in the first person ("I did X") is efficient for storage but dangerous for retrieval. We must always wrap retrieval in a translation layer.
  2. Model Size Matters: For logic requiring strict adherence to negative constraints ("Do NOT do X"), 3B parameters is the practical minimum.
  3. Router Intelligence: The router (_route_request) is the most critical component for UX. It must distinguish between knowledge retrieval (World) and context retrieval (User).

Signed:
BuddAI Executive System