> ## Documentation Index
> Fetch the complete documentation index at: https://docs.sammylabs.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Tools System

> Comprehensive guide to the SAMMY Three tools system for extending AI agent capabilities with custom actions

<Note>
  The SAMMY Three package implements a sophisticated tool system that allows the Live API agent to perform actions beyond conversation. Tools are type-safe, event-driven, and seamlessly integrated with Google's Gemini Live API.
</Note>

## Architecture Overview

<Card title="System Components" icon="sitemap">
  The tools system consists of several integrated components:

  ```mermaid theme={null}
  graph LR
      A[Tool Manager] --> B[Tool Registry]
      B --> C[Function Calls]
      A --> D[Event System]
      B --> E[Type Safety]
      C --> F[Live API]
  ```
</Card>

### Core Components

<Columns cols={2}>
  <Card title="ToolManager" icon="database">
    Central registry for managing tool registration, execution, and events
  </Card>

  <Card title="Tool Definitions" icon="code">
    Type-safe tool declarations with handlers and metadata
  </Card>

  <Card title="Event System" icon="broadcast">
    Type-safe event emission and subscription for tool interactions
  </Card>

  <Card title="Live API Integration" icon="link">
    Seamless integration with Google's Gemini Live API
  </Card>
</Columns>

## Tool Definition Structure

<Info>
  Every tool in the system follows a consistent structure defined by the `ToolDefinition` interface.
</Info>

```typescript theme={null}
interface ToolDefinition<TEventMap> {
  handler: (fc: FunctionCall, context: ToolContext<TEventMap>) => Promise<FunctionResponse>;
  category: ToolCategory;
  declaration: TypedFunctionDeclaration;
}
```

### Components Breakdown

<Tabs>
  <Tab title="Declaration">
    The `declaration` defines the tool's schema for the Live API:

    ```typescript theme={null}
    {
      name: ToolName.EXAMPLE_TOOL,
      description: "Clear description of what the tool does",
      behavior: Behavior.NON_BLOCKING, // or Behavior.BLOCKING
      parameters: {
        type: Type.OBJECT,
        properties: {
          paramName: {
            type: Type.STRING,
            description: "Parameter description"
          }
        },
        required: ['paramName']
      }
    }
    ```
  </Tab>

  <Tab title="Handler Function">
    The handler processes the actual tool execution:

    ```typescript theme={null}
    handler: async (fc: FunctionCall, context: ToolContext) => {
      // Parse arguments
      const args = typeof fc.args === 'string' 
        ? JSON.parse(fc.args) 
        : fc.args;
      
      // Perform tool logic
      const result = await performToolAction(args);
      
      // Emit events if needed
      context.emit('customEvent', result);
      
      // Return response
      return {
        id: fc.id,
        name: fc.name,
        response: {
          output: result,
          scheduling: FunctionResponseScheduling.WHEN_IDLE
        }
      };
    }
    ```
  </Tab>

  <Tab title="Tool Context">
    The context provides access to services, state, and event emission:

    ```typescript theme={null}
    interface ToolContext<TEventMap> {
      emit: (event: keyof TEventMap, ...args: any[]) => void;
      getState: () => any; // Access to agent state
      services: SammyAgentServices; // Access to service layer
    }
    ```
  </Tab>
</Tabs>

## Tool Lifecycle

### Registration Phase

<Steps>
  <Step title="Tool Registration">
    Tools are registered during agent initialization:

    ```typescript theme={null}
    // In SammyAgentCore constructor
    this.toolManager = new ToolManager(
      this.services, 
      this.tools
    );
    ```
  </Step>

  <Step title="Automatic Setup">
    The ToolManager automatically:

    * Registers default tools from `DefaultToolDefinitions`
    * Registers custom tools passed via constructor
    * Creates type-safe event emitters
  </Step>
</Steps>

### Configuration Phase

During agent startup, tools are integrated with the Live API:

```typescript theme={null}
// In agent start() method
const agentConnectConfig: LiveConnectConfig = getAgentConfig(
  this.toolManager.getAllDeclarations(), // Tools passed here
  systemPrompt
);

// The getAgentConfig function structures tools for the Live API:
tools: [
  {
    functionDeclarations, // Array of tool declarations
  },
],
```

### Execution Phase

<Steps>
  <Step title="Message Reception">
    Live API sends tool call message
  </Step>

  <Step title="Event Emission">
    GenAI client emits 'toolcall' event
  </Step>

  <Step title="Handler Routing">
    ToolManager routes to appropriate handler
  </Step>

  <Step title="Execution">
    Tool handler executes with context
  </Step>

  <Step title="Response">
    Function response sent back to Live API
  </Step>
</Steps>

```typescript theme={null}
// In SammyAgentCore.handleToolCall()
private async handleToolCall(toolCall: LiveServerToolCall): Promise<void> {
  console.log('[Agent] Tool call:', toolCall);
  
  // Notify callback
  this.callbacks.onToolCall?.(toolCall);
  
  // Handle tool calls
  const functionResponses = await this.toolManager.handleToolCall(toolCall);
  
  // Send responses back
  if (functionResponses && functionResponses.length > 0) {
    this.client.sendToolResponse({ functionResponses });
  }
}
```

## Built-in Tools

SAMMY Three includes several pre-configured tools that handle common scenarios:

<Tabs>
  <Tab title="End Session Tool">
    <Card title="Session Management" icon="stop-circle">
      **Purpose**: Manages agent session termination gracefully

      **Category**: Action

      **Behavior**: Non-blocking

      ```typescript theme={null}
      // Usage: Agent calls when user wants to end session
      {
        name: "endSession",
        parameters: {
          reason: "user requested",
          confirmed: true
        }
      }
      ```

      **Automatic Triggers:**

      * User says goodbye or wants to end chat
      * Task has been completed successfully
      * User explicitly requests to stop

      **Handler Logic:**

      * Validates confirmation parameter
      * Emits session end event if confirmed
      * Saves conversation state
      * Triggers cleanup processes
    </Card>
  </Tab>

  <Tab title="Get Context Tool">
    <Card title="Context Retrieval" icon="info-circle">
      **Purpose**: Retrieves current page and session context

      **Category**: Context

      **Behavior**: Non-blocking

      ```typescript theme={null}
      // Usage: Agent calls to understand current environment
      {
        name: "getCurrentContext",
        parameters: {
          reason: "Getting context for better assistance"
        }
      }
      ```

      **Returns:**

      * Current page URL and title
      * User preferences and settings
      * Session metadata
      * Custom context data
      * Timestamp information

      **Use Cases:**

      * Understanding user's current location in app
      * Providing contextual help
      * Personalizing responses
    </Card>
  </Tab>

  <Tab title="Escalate Tool">
    <Card title="Human Escalation" icon="arrow-up">
      **Purpose**: Escalates to human support when needed

      **Category**: System

      **Behavior**: Blocking

      ```typescript theme={null}
      // Automatically triggered by AI when needed
      {
        name: "escalate",
        parameters: {
          reason: "Unable to resolve user issue",
          category: "technical_support",
          urgency: "medium"
        }
      }
      ```

      <Warning>
        The escalate tool is called automatically by the AI when it cannot help. You don't need to manually trigger it.
      </Warning>

      **Automatic Triggers:**

      * AI cannot understand the request
      * User explicitly asks for human help
      * Sensitive topics are detected
      * Complex issues beyond AI scope

      **Process:**

      1. Marks conversation for review
      2. Notifies support team
      3. Provides escalation reason
      4. Maintains full conversation context
      5. Optionally transfers to live chat
    </Card>
  </Tab>
</Tabs>

## Creating Custom Tools

### Step 1: Define Tool Types

Add your tool to the enum and event map:

```typescript theme={null}
// In types.ts
export enum ToolName {
  CUSTOM_TOOL = 'customTool',
}

export interface CustomToolEventMap extends BaseToolEventMap {
  [ToolName.CUSTOM_TOOL]: (data: any) => void;
}
```

### Step 2: Create Tool Definition

<CodeGroup>
  ```typescript custom-tool.ts theme={null}
  import { Behavior, Type } from '@google/genai';
  import { ToolDefinition, ToolCategories } from '../types';

  export const customTool: ToolDefinition<CustomToolEventMap> = {
    declaration: {
      name: ToolName.CUSTOM_TOOL,
      description: "Description of your custom tool",
      behavior: Behavior.NON_BLOCKING,
      parameters: {
        type: Type.OBJECT,
        properties: {
          input: {
            type: Type.STRING,
            description: "Input parameter"
          }
        },
        required: ['input']
      }
    },
    category: ToolCategories.ACTION,
    handler: async (fc, context) => {
      const args = typeof fc.args === 'string' 
        ? JSON.parse(fc.args) 
        : fc.args;
      
      // Your custom logic here
      const result = await processCustomLogic(args.input);
      
      // Emit custom event
      context.emit(ToolName.CUSTOM_TOOL, result);
      
      return {
        id: fc.id,
        name: fc.name,
        response: {
          output: result,
          scheduling: FunctionResponseScheduling.WHEN_IDLE
        }
      };
    }
  };
  ```

  ```typescript Registration theme={null}
  // Pass your tool when creating the agent
  const customTools = [customTool];

  const agentCore = new SammyAgentCore({
    services,
    config,
    tools: customTools,
    // ... other options
  });
  ```
</CodeGroup>

## Tool Execution Flow

### Parallel Processing

<Note>
  The ToolManager processes multiple function calls in parallel for optimal performance.
</Note>

```typescript theme={null}
async handleToolCall(toolCall: LiveServerToolCall): Promise<FunctionResponse[]> {
  const responses: FunctionResponse[] = [];
  
  if (!toolCall.functionCalls) return responses;
  
  // Process all function calls in parallel
  const functionResponses = await Promise.all(
    toolCall.functionCalls.map(async (fc) => {
      return await this.handleFunctionCall(fc);
    })
  );
  
  responses.push(...functionResponses);
  return responses;
}
```

### Error Handling

<Warning>
  Robust error handling is implemented at multiple levels:
</Warning>

<Steps>
  <Step title="Missing Tool Name">
    Returns structured error response
  </Step>

  <Step title="Missing Handler">
    Returns "tool not found" error
  </Step>

  <Step title="Handler Errors">
    Catches and formats exceptions
  </Step>
</Steps>

```typescript theme={null}
async handleFunctionCall(fc: FunctionCall, agentState?: any): Promise<FunctionResponse> {
  if (!fc.name) return missingToolNameError(fc);
  
  const tool = this.tools.get(fc.name);
  if (!tool) return missingToolHandlerError(fc);
  
  try {
    return await tool.handler(fc, context);
  } catch (error) {
    return toolError(fc, error);
  }
}
```

## Event System

### Type-Safe Events

<Card title="Compile-Time Safety" icon="shield-check">
  The event system provides compile-time type safety:

  ```typescript theme={null}
  // Tool handler can emit events
  context.emit(ToolName.END_SESSION); // ✅ Type-safe
  context.emit('invalidEvent'); // ❌ Compile error

  // External listeners
  toolManager.on(ToolName.END_SESSION, () => {
    console.log('Session ending...');
  });
  ```
</Card>

### Event Flow

```mermaid theme={null}
graph LR
    A[Tool Execution] --> B[Event Emission]
    B --> C[Context.emit]
    C --> D[External Listeners]
    D --> E[Component State]
```

## Error Response Structure

All errors return consistent `FunctionResponse` objects:

```typescript theme={null}
{
  id: fc.id,
  name: fc.name,
  response: {
    output: {
      error: "Error message",
      success: false
    },
    scheduling: FunctionResponseScheduling.SILENT
  }
}
```

### Error Types

<Columns cols={3}>
  <Card title="Missing Tool Name" icon="exclamation">
    Tool call without name
  </Card>

  <Card title="Missing Handler" icon="question">
    Tool not registered
  </Card>

  <Card title="Handler Exceptions" icon="bug">
    Runtime errors in tool logic
  </Card>
</Columns>

<Info>
  Errors use `SILENT` scheduling to avoid interrupting conversation flow.
</Info>

## Best Practices

### Tool Design

<Steps>
  <Step title="Single Responsibility">
    Each tool should have one clear purpose
  </Step>

  <Step title="Clear Descriptions">
    Provide comprehensive descriptions for the LLM
  </Step>

  <Step title="Parameter Validation">
    Always validate input parameters
  </Step>

  <Step title="Error Handling">
    Implement robust error handling
  </Step>
</Steps>

### Async Operations

* Use `Behavior.NON_BLOCKING` for async operations
* Handle timing with appropriate `FunctionResponseScheduling`
* Consider user experience when choosing scheduling

### Event Usage

* Emit events for significant state changes
* Use type-safe event definitions
* Document event contracts

### Performance

* Process multiple calls in parallel when possible
* Avoid blocking operations in tool handlers
* Use appropriate scheduling for response timing

## Advanced Features

### Asynchronous Function Calling

<Tabs>
  <Tab title="Scheduling Options">
    For non-blocking operations, tools can use different scheduling options:

    ```typescript theme={null}
    // Interrupt current conversation
    scheduling: FunctionResponseScheduling.INTERRUPT

    // Wait for natural pause
    scheduling: FunctionResponseScheduling.WHEN_IDLE

    // Silent execution
    scheduling: FunctionResponseScheduling.SILENT
    ```
  </Tab>

  <Tab title="Custom Event Maps">
    Extend the base event map for custom tool events:

    ```typescript theme={null}
    interface CustomEventMap extends BaseToolEventMap {
      'custom:action': (data: ActionData) => void;
      'custom:error': (error: Error) => void;
    }
    ```
  </Tab>
</Tabs>

### Service Integration

Tools have full access to the service layer:

```typescript theme={null}
handler: async (fc, context) => {
  // Access to all services
  const { sammyApi, coreServices, memoryServices } = context.services;
  
  // Perform service operations
  const result = await sammyApi.processRequest(data);
  
  return response;
}
```

### State Access

Tools can access current agent state:

```typescript theme={null}
handler: async (fc, context) => {
  const currentState = context.getState();
  
  // Use state in tool logic
  if (currentState.userVolume > 0.8) {
    // Handle high volume scenario
  }
  
  return response;
}
```

## Integration with Live API

<Note>
  The tools system integrates seamlessly with Live API capabilities:
</Note>

<Columns cols={2}>
  <Card title="Function Calling" icon="function">
    Direct integration with Gemini's function calling
  </Card>

  <Card title="Code Execution" icon="code">
    Can be combined with code execution tools
  </Card>

  <Card title="Google Search" icon="search">
    Compatible with search grounding
  </Card>

  <Card title="Multi-modal" icon="images">
    Supports audio, text, and visual contexts
  </Card>
</Columns>

## Summary

The comprehensive tool system enables powerful agent capabilities while maintaining:

* **Type Safety**: Full TypeScript support throughout
* **Performance**: Parallel processing and optimal scheduling
* **Developer Experience**: Clean API and clear patterns
* **Extensibility**: Easy to add custom tools without breaking existing ones