> ## Documentation Index
> Fetch the complete documentation index at: https://docs.sammylabs.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Context Management System

> Centralized context management for Sammy Agent conversations with automatic memory searches and pull-based retrieval

<Note>
  The Context Management System uses a **"pull" strategy** where the AI model naturally requests context through tool calls rather than having context pushed to it. This ensures seamless integration with the Gemini Live API.
</Note>

## Recent Updates

<Card title="Fixes" icon="wrench">
  Fixed getContext tool registration and added automatic system prompt augmentation for context tool usage. The AI now proactively calls the tool when needing user information, page context, or memories.
</Card>

## Architecture Overview

<Steps>
  <Step title="Context Sources">
    Multiple sources feed into the context system:

    * Page Metadata (URL, title, domain)
    * User Information (preferences, settings)
    * Memory Search (semantic search)
    * Custom Context (application-specific)
  </Step>

  <Step title="Context State Manager">
    Centralized state storage with:

    * Context formatting for LLM
    * Update tracking and management
    * Type-safe state access
  </Step>

  <Step title="Context Memory Manager">
    Automatic memory handling:

    * Transcription-based searches
    * Debouncing and thresholds
    * Memory backend integration
  </Step>

  <Step title="GetContext Tool">
    Pull-based retrieval:

    * Synchronous tool execution
    * INTERRUPT scheduling for immediate use
    * No API calls - just formatting
  </Step>
</Steps>

## Key Components

### SystemPromptAugmentor

<Info>
  The `SystemPromptAugmentor` is a singleton that manages all system prompt modifications, ensuring the AI knows about available tools and context.
</Info>

<CodeGroup>
  ```typescript Basic Usage theme={null}
  const augmentor = SystemPromptAugmentor.getInstance();

  // Set language for the session
  augmentor.setLanguage('es-ES');

  // Add custom augmentation
  augmentor.addAugmentation({
    id: 'custom-rules',
    content: '## Custom Rules\n\nAlways be polite and professional.',
    priority: 40,
    enabled: true,
  });

  // Apply augmentations to base prompt
  const finalPrompt = augmentor.augmentPrompt(basePrompt);
  ```

  ```typescript Advanced Configuration theme={null}
  // Disable specific augmentation
  augmentor.setAugmentationEnabled('tool-usage', false);

  // Get debug information
  const debugInfo = augmentor.getDebugInfo();
  console.log('Active augmentations:', debugInfo);
  ```
</CodeGroup>

### ContextStateManager

The central state management class for all contextual information:

<Tabs>
  <Tab title="Page Metadata">
    ```typescript theme={null}
    contextManager.updatePageMetadata({
      url: 'https://example.com/dashboard',
      title: 'User Dashboard',
      domain: 'example.com',
      path: '/dashboard'
    });
    ```
  </Tab>

  <Tab title="User Information">
    ```typescript theme={null}
    contextManager.updateUserPreferences({
      name: 'Joseph Marinio',
      preferences: { 
        theme: 'dark', 
        language: 'en' 
      }
    });
    ```
  </Tab>

  <Tab title="Custom Context">
    ```typescript theme={null}
    contextManager.setCustomContext(
      'currentTask', 
      'reviewing invoices'
    );
    ```
  </Tab>

  <Tab title="Format Output">
    ```typescript theme={null}
    const contextString = 
      contextManager.formatForToolResponse();
    ```
  </Tab>
</Tabs>

### ContextMemoryManager

Handles automatic memory searches based on transcriptions:

<CodeGroup>
  ```typescript Configuration theme={null}
  const memoryManager = new ContextMemoryManager(
    memoryService,
    contextStateManager,
    {
      userInputThreshold: 20,      // Min chars for user input
      agentOutputThreshold: 50,    // Min chars for agent output
      searchLimit: 3,              // Max memories to return
      similarityThreshold: 0.2     // Min relevance score
    }
  );
  ```

  ```typescript Automatic Processing theme={null}
  // Automatically called by transcription handlers
  await memoryManager.processUserTranscription(
    transcript, 
    session
  );

  await memoryManager.processAgentTranscription(
    transcript, 
    session
  );
  ```
</CodeGroup>

## Automatic Context Tracking

SAMMY Three provides hooks for automatic context updates without manual intervention:

### useContextUpdater Hook

<Info>
  The `useContextUpdater` hook automatically tracks page changes and updates context in real-time.
</Info>

```tsx theme={null}
import { useContextUpdater } from '@sammy-labs/sammy-three';

function MyComponent() {
  // Automatically track page changes
  useContextUpdater({
    trackPageChanges: true,    // Monitor URL and title changes
    updateInterval: 5000,       // Check for changes every 5 seconds
    includeMetadata: true,      // Include page metadata
    customContext: {            // Add custom context data
      userId: 'user-123',
      sessionId: 'session-456',
      theme: 'dark',
      permissions: ['read', 'write'],
    },
  });

  return <div>Content tracked automatically</div>;
}
```

### Manual Context Updates

For more granular control, update context manually:

<CodeGroup>
  ```tsx Page Information theme={null}
  import { ContextStateManager } from '@sammy-labs/sammy-three';

  const contextManager = new ContextStateManager();

  // Update page information
  contextManager.updatePageMetadata({
    url: window.location.href,
    title: document.title,
    timestamp: Date.now(),
    viewport: {
      width: window.innerWidth,
      height: window.innerHeight,
    },
  });
  ```

  ```tsx User Preferences theme={null}
  // Update user preferences
  contextManager.updateUserPreferences({
    name: 'John Doe',
    email: 'john@example.com',
    role: 'admin',
    settings: {
      notifications: true,
      theme: 'dark',
      language: 'en-US',
    },
  });
  ```

  ```tsx Custom Context theme={null}
  // Add application-specific context
  contextManager.updateCustomContext({
    activeFeature: 'dashboard',
    currentTask: 'data-analysis',
    openModals: ['settings', 'help'],
    formData: {
      step: 3,
      completed: ['personal', 'business'],
    },
  });
  ```
</CodeGroup>

### Context Lifecycle

<Steps>
  <Step title="Initialization" icon="power">
    Context managers are created when the agent starts
  </Step>

  <Step title="Automatic Updates" icon="refresh">
    Page changes and user actions trigger context updates
  </Step>

  <Step title="Memory Search" icon="search">
    Transcriptions automatically trigger memory searches
  </Step>

  <Step title="AI Retrieval" icon="download">
    AI calls getContext tool when needed
  </Step>

  <Step title="Cleanup" icon="broom">
    Context is cleared when agent stops
  </Step>
</Steps>

## Integration Guide

### Basic Setup

<Tip>
  Context management is automatically initialized when creating a new agent. No manual setup required!
</Tip>

```typescript theme={null}
const agent = new SammyAgentCore({
  // ... other options
});

// Context management is initialized automatically
// ✅ Transcriptions trigger memory searches
// ✅ GetContext tool is registered
// ✅ System prompt includes context instructions
```

### Manual Context Updates

<Tabs>
  <Tab title="Page Context">
    ```typescript theme={null}
    agent.updatePageContext({
      url: window.location.href,
      title: document.title
    });
    ```
  </Tab>

  <Tab title="User Context">
    ```typescript theme={null}
    agent.updateUserContext({
      name: 'Joseph Marinio',
      preferences: { role: 'admin' }
    });
    ```
  </Tab>

  <Tab title="Custom Context">
    ```typescript theme={null}
    const contextManager = 
      agent.getContextStateManager();

    contextManager.setCustomContext(
      'activeWorkflow', 
      workflowData
    );
    ```
  </Tab>
</Tabs>

### React Hook Usage

For React applications, use the `useContextUpdater` hook:

```tsx theme={null}
import { useContextUpdater } from '@sammy-labs/sammy-three';

function MyApp() {
  const { 
    startMonitoring, 
    updateUserContext, 
    setCustomContext 
  } = useContextUpdater({
    updateInterval: 1000,    // Check every second
    includeTitle: true,      // Track page title
    includeDomain: true,     // Track domain
    includePath: true        // Track URL path
  });

  // Start monitoring when agent is ready
  useEffect(() => {
    if (agentCoreRef.current) {
      startMonitoring(agentCoreRef.current);
    }
  }, [agentCoreRef.current]);

  // Manual updates
  const handleUserLogin = (user) => {
    updateUserContext({
      name: user.name,
      preferences: user.preferences
    });
  };

  // Custom context
  const handleTaskChange = (task) => {
    setCustomContext('currentTask', task);
  };
}
```

## Context Flow Example

<Steps>
  <Step title="Page Navigation">
    User navigates to a new page

    * URL change detected by context updater
    * Page metadata updated in ContextStateManager
  </Step>

  <Step title="User Speech">
    User speaks: "What is my name?"

    * Transcription processed by ContextMemoryManager
    * Memory search triggered after 20+ characters
    * Relevant memories stored in context state
  </Step>

  <Step title="AI Response">
    AI recognizes it needs context

    * System prompt instructs AI to use getContext tool
    * AI calls getContext with query "user information"
    * Tool retrieves formatted context from ContextStateManager
    * Context returned with INTERRUPT scheduling
    * AI uses context: "Your name is Joseph Marinio"
  </Step>
</Steps>

## System Prompt Augmentations

### System Prompt Augmentation Details

<Tabs>
  <Tab title="Context Tool Instructions (Priority: 10)">
    ```
    ## Context Retrieval Tool

    IMPORTANT: When the user asks questions about themselves, 
    their preferences, or their context, you MUST use the 
    getContext tool to retrieve this information. 

    The getContext tool provides:
    - User information (name, preferences, settings)
    - Current page context (URL, title, domain)
    - Relevant memories from past interactions
    - Custom contextual information

    Call getContext with queries like:
    - "user information" - for user details
    - "current page" - for page context
    - "everything" - for all available context
    - "memories about [topic]" - for specific memories

    Always use this tool when you need contextual 
    information to provide personalized assistance.
    ```
  </Tab>

  <Tab title="Language Instructions (Priority: 5)">
    When a non-English language is configured, appropriate instructions are added in the target language. For example, for Spanish:

    ```
    ## Language Settings

    El idioma elegido por el usuario es español (España). 
    DEBES hablar SOLO en español.
    ```
  </Tab>

  <Tab title="Memory Management (Priority: 20)">
    ```
    ## Memory Management

    When handling conversations:
    1. Pay attention to context provided through the getContext tool
    2. Use retrieved memories to maintain conversation continuity
    3. Reference past interactions when relevant
    4. Personalize responses based on user preferences and history
    ```
  </Tab>

  <Tab title="Tool Usage Guidelines (Priority: 30)">
    ```
    ## Tool Usage Guidelines

    When using tools:
    1. Always check if a tool can help answer the user's question
    2. Use tools proactively rather than asking users for information
    3. Combine multiple tools when needed for comprehensive responses
    4. Explain what you're doing when using tools that affect the user's system
    ```
  </Tab>
</Tabs>

## Context Formatting

### Tool Response Format

<Card title="Context Retrieved" icon="database">
  The system formats context with clear sections for easy AI parsing:

  ```
  [CONTEXT RETRIEVED]
  📍 Current Page:
     URL: https://example.com/dashboard
     Title: User Dashboard
     Timestamp: 2024-01-15T10:30:00Z

  👤 User Information:
     Name: Joseph Marinio
     Preferences: {"theme":"dark"}

  💭 Relevant Knowledge:
     1. User prefers dark theme (0.95)
     2. User is an admin (0.87)
     3. Last login was yesterday (0.72)

  🔧 Additional Context:
     currentTask: "reviewing invoices"

  Use this context to provide personalized 
  and accurate assistance.
  ```
</Card>

### Custom Formatting Options

```typescript theme={null}
const formatted = contextManager.formatContext({
  includeMemories: true,
  includePageMetadata: true,
  includeUserPreferences: true,
  includeCustomContext: true,
  memoryLimit: 5,
  format: 'detailed' // or 'compact'
});
```

## Configuration Reference

### Memory Search Configuration

<CodeGroup>
  ```typescript Interface theme={null}
  interface MemorySearchConfig {
    userInputThreshold?: number;     // Default: 20
    agentOutputThreshold?: number;   // Default: 50
    searchLimit?: number;            // Default: 3
    similarityThreshold?: number;    // Default: 0.2
    useFeatureIdFilter?: boolean;    // Default: true
    searchGlobalOnly?: boolean;      // Default: false
    userContextPrefix?: string;      // Default: '[LIVE CONTEXT]'
    agentContextPrefix?: string;     // Default: '[VALIDATION]'
  }
  ```

  ```typescript Example theme={null}
  const config: MemorySearchConfig = {
    userInputThreshold: 30,    // Require more text
    searchLimit: 5,            // Return more memories
    similarityThreshold: 0.3,  // Higher quality matches
    searchGlobalOnly: true     // Global memories only
  };
  ```
</CodeGroup>

### Context State Structure

```typescript theme={null}
interface ContextState {
  memories: MemoryEntry[];
  pageMetadata: PageMetadata | null;
  userPreferences: UserPreferences | null;
  customContext: Record<string, any>;
  lastUpdated: {
    memories?: Date;
    pageMetadata?: Date;
    userPreferences?: Date;
  };
}
```

## Best Practices

<Columns cols={2}>
  <Card title="Update Proactively" icon="refresh">
    * Use context updater hook for automatic page tracking
    * Update user context after authentication
    * Set custom context for important state changes
  </Card>

  <Card title="Optimize Memory Search" icon="search">
    * Adjust thresholds based on your use case
    * Use debouncing to prevent excessive searches
    * Consider feature-scoped vs global searches
  </Card>

  <Card title="Use Custom Context" icon="code">
    * Store workflow states
    * Track user actions
    * Add temporary contextual data
  </Card>

  <Card title="Performance Tips" icon="gauge">
    * Context is only formatted when requested
    * Memory searches are debounced
    * State updates are lightweight
  </Card>
</Columns>

## Troubleshooting

### GetContext Tool Not Being Called

<Warning>
  If the AI isn't calling the getContext tool, follow these steps:
</Warning>

<Steps>
  <Step title="Check Registration">
    Run `agent.debugContextSystem()` to verify the tool is registered
  </Step>

  <Step title="Verify System Prompt">
    Check console logs to ensure system prompt includes context instructions
  </Step>

  <Step title="Test Manually">
    Try asking explicit questions like "What is my name?" or "What page am I on?"
  </Step>

  <Step title="Check Logs">
    Look for these log messages:

    * `🔧 [SAMMY-AGENT-CORE] Registered getContext tool`
    * `✅ [SAMMY-AGENT-CORE] getContext tool is present in declarations`
  </Step>
</Steps>

### Memory Search Not Working

<Steps>
  <Step title="Check Thresholds">
    Ensure transcription meets minimum character thresholds
  </Step>

  <Step title="Verify API Connection">
    Check memory service is properly initialized
  </Step>

  <Step title="Monitor Logs">
    Look for `💭 [CONTEXT-MEMORY-MANAGER]` messages
  </Step>
</Steps>

## Debug Logging

Enable debug logging to see context updates:

```
🎯 [CONTEXT-STATE-MANAGER] initialized
📚 [CONTEXT-STATE-MANAGER] Updating memories: 3 entries
🌐 [CONTEXT-STATE-MANAGER] Updating page metadata
🔍 [GET-CONTEXT-TOOL] Called with query: user information
💭 [CONTEXT-MEMORY-MANAGER] Searching memories for user: What is my name...
🔧 [SAMMY-AGENT-CORE] Registered getContext tool
✅ [SAMMY-AGENT-CORE] getContext tool is present in declarations
```

## Migration from Legacy System

<Note>
  The old MemoryManager has been completely removed. All memory management now goes through the centralized context system.
</Note>

<CodeGroup>
  ```typescript Old (Removed) theme={null}
  // ❌ Don't use this anymore
  memoryManager.search(query)
  memoryManager.processTranscription(text)
  ```

  ```typescript New (Required) theme={null}
  // ✅ Use the new system
  contextMemoryManager.processUserTranscription(
    transcript, 
    session
  );
  // Context automatically updated and 
  // available via getContext tool
  ```
</CodeGroup>

## Future Enhancements

<Info>
  When Google fixes the `sendClientContent` bug, the system can easily switch from tool-based to text-based injection:
</Info>

```mermaid theme={null}
graph LR
    A[Current: Tool-based pull] --> B[AI calls getContext]
    B --> C[Context Manager]
    C --> D[Formatted Context]
    
    E[Future: Text-based push] --> F[Context Manager]
    F --> G[sendClientContent]
    G --> H[AI receives context]
```

The modular design ensures a smooth migration path without changing the core context management logic.