> ## Documentation Index
> Fetch the complete documentation index at: https://docs.sammylabs.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Performance Optimization

> Optimize SAMMY Three for maximum performance and smooth user experience

# Performance Optimization

Achieve optimal performance with SAMMY Three's built-in optimization strategies and worker-based architecture.

## Overview

SAMMY Three is designed for high performance with multiple optimization layers working together.

<Columns cols={3}>
  <Card title="Worker Architecture" icon="network-wired">
    Offload heavy processing to background workers
  </Card>

  <Card title="Audio-Aware Capture" icon="waveform-lines">
    Intelligent throttling during audio playback
  </Card>

  <Card title="Critical Renders" icon="crosshairs">
    Automatic capture at conversation boundaries
  </Card>
</Columns>

## Worker Architecture

SAMMY Three uses multiple workers to prevent main thread blocking.

### Worker Types

<Tabs>
  <Tab title="Canvas Worker" icon="image">
    Handles image encoding and compression.

    ```tsx theme={null}
    // Automatic - no configuration needed
    // Processes screen captures in background
    // Falls back to main thread if unavailable
    ```

    **Operations:**

    * JPEG/PNG encoding
    * Image compression
    * Resolution scaling
    * Format conversion
  </Tab>

  <Tab title="Observability Worker" icon="chart-line">
    Batches and sends analytics events.

    ```tsx theme={null}
    const config = {
      observability: {
        useWorker: true, // Enable worker mode
        workerConfig: {
          batchSize: 50,
          batchIntervalMs: 5000,
        },
      },
    };
    ```

    **Operations:**

    * Event batching
    * API request queuing
    * Data compression
    * Retry logic
  </Tab>

  <Tab title="DOM Capture Workers" icon="camera">
    Generates screenshots efficiently.

    ```tsx theme={null}
    // Multiple workers for parallel processing
    // Automatic load balancing
    // CSP-compliant Data URL approach
    ```

    **Operations:**

    * DOM traversal
    * Style computation
    * SVG generation
    * HTML rendering
  </Tab>
</Tabs>

### Worker Benefits

<Steps>
  <Step title="Non-Blocking UI" icon="bolt">
    Main thread remains responsive during heavy operations
  </Step>

  <Step title="Parallel Processing" icon="layer-group">
    Multiple operations execute simultaneously
  </Step>

  <Step title="Automatic Fallback" icon="shield">
    Gracefully degrades to main thread if workers unavailable
  </Step>

  <Step title="CSP Compliance" icon="lock">
    Uses Data URLs instead of blob URLs for security
  </Step>
</Steps>

## Audio-Aware Capture

Screen capture automatically adapts based on audio state to prevent stuttering.

### How It Works

```tsx theme={null}
const config = {
  captureConfig: {
    enableAudioAdaptation: true, // Default: true
    quality: 0.8,
  },
};
```

<Columns cols={2}>
  <Card title="Normal Mode">
    **When:** No audio playing

    * High-frequency captures (100ms-2s)
    * Full quality rendering
    * Maximum detail capture
  </Card>

  <Card title="Audio Mode">
    **When:** Agent speaking

    * Reduced captures (500ms-5s)
    * Optimized quality
    * Prevents audio stuttering
  </Card>
</Columns>

### Configuration

```tsx theme={null}
// Fine-tune audio adaptation
const config = {
  captureConfig: {
    enableAudioAdaptation: true,
    
    // Normal mode settings
    normalInterval: 1000,      // ms between captures
    normalQuality: 0.9,        // JPEG quality
    
    // Audio mode settings
    audioInterval: 3000,       // ms between captures
    audioQuality: 0.7,         // Reduced quality
  },
};
```

## Critical DOM Renders

Ensures fresh visual context at important conversation moments.

### Automatic Triggers

Critical renders happen automatically at:

<Steps>
  <Step title="Speech Boundaries" icon="microphone">
    * User starts speaking
    * User stops speaking
    * Agent starts response
    * Agent completes response
  </Step>

  <Step title="Interruptions" icon="hand">
    * User interrupts agent
    * Agent pauses for user
    * Conversation direction changes
  </Step>

  <Step title="State Changes" icon="refresh">
    * Page navigation
    * Major UI updates
    * Form submissions
    * Modal opens/closes
  </Step>
</Steps>

### Benefits

<Check>
  Agent always has current visual context
</Check>

<Check>
  No stale UI information during conversations
</Check>

<Check>
  Captures happen even when regular capture is throttled
</Check>

<Check>
  Minimal performance impact with smart timing
</Check>

## Memory Management

Optimize memory usage for long-running sessions.

### Automatic Cleanup

```tsx theme={null}
// SAMMY Three automatically manages:
- Old screen captures
- Audio buffers
- Processed frames
- Event queues
```

### Manual Optimization

```tsx theme={null}
import { useSammyAgentContext } from '@sammy-labs/sammy-three';

function MemoryOptimizedChat() {
  const { clearCache, getMemoryUsage } = useSammyAgentContext();
  
  // Monitor memory usage
  useEffect(() => {
    const interval = setInterval(() => {
      const usage = getMemoryUsage();
      if (usage > 100 * 1024 * 1024) { // 100MB
        clearCache();
      }
    }, 60000); // Check every minute
    
    return () => clearInterval(interval);
  }, []);
  
  return <ChatInterface />;
}
```

## Capture Optimization

### Quality Settings

Balance quality and performance based on your needs.

<CodeGroup>
  ```tsx High Quality theme={null}
  // Best for detailed UIs
  const config = {
    captureConfig: {
      quality: 0.95,
      frameRate: 30,
      resolution: {
        maxWidth: 1920,
        maxHeight: 1080,
      },
    },
  };
  ```

  ```tsx Balanced theme={null}
  // Good quality with better performance
  const config = {
    captureConfig: {
      quality: 0.8,
      frameRate: 15,
      resolution: {
        maxWidth: 1280,
        maxHeight: 720,
      },
    },
  };
  ```

  ```tsx Performance theme={null}
  // Maximum performance for slower devices
  const config = {
    captureConfig: {
      quality: 0.6,
      frameRate: 10,
      resolution: {
        maxWidth: 854,
        maxHeight: 480,
      },
      enableAudioAdaptation: true,
    },
  };
  ```
</CodeGroup>

### Capture Methods Comparison

| Method   | CPU Usage   | Quality  | Best For            |
| -------- | ----------- | -------- | ------------------- |
| `render` | Low-Medium  | High     | Web applications    |
| `video`  | Medium-High | Variable | Full screen capture |

<Tip>
  Use `render` method for most web applications. It's more efficient and provides consistent quality.
</Tip>

## Performance Debugging

### Audio Stutter Analysis

Debug and fix audio performance issues.

```tsx theme={null}
// Enable debugging
const config = {
  debugAudioPerformance: true,
};

// In browser console
audioStutterAnalyzer.setDebugMode(true);
audioStutterAnalyzer.getAnalysis();

// Returns detailed metrics:
{
  stutterCount: 3,
  underruns: [/* timestamps */],
  longCaptures: [/* durations */],
  averageBufferLevel: 0.8,
  correlationScore: 0.7
}
```

### Performance Monitoring

```tsx theme={null}
import { usePerformanceMonitor } from '@sammy-labs/sammy-three';

function PerformanceDebugger() {
  const metrics = usePerformanceMonitor();
  
  return (
    <div className="debug-panel">
      <div>FPS: {metrics.fps}</div>
      <div>Capture Time: {metrics.captureTime}ms</div>
      <div>Memory: {metrics.memoryUsage}MB</div>
      <div>Worker Queue: {metrics.workerQueueSize}</div>
    </div>
  );
}
```

## Optimization Strategies

### Device-Based Configuration

Adjust settings based on device capabilities.

```tsx theme={null}
function getOptimalConfig() {
  const memory = navigator.deviceMemory || 4; // GB
  const cores = navigator.hardwareConcurrency || 4;
  
  if (memory <= 2 || cores <= 2) {
    // Low-end device
    return {
      captureConfig: {
        quality: 0.6,
        frameRate: 10,
        enableAudioAdaptation: true,
      },
      observability: {
        useWorker: false, // Avoid worker overhead
      },
    };
  }
  
  if (memory <= 4 || cores <= 4) {
    // Mid-range device
    return {
      captureConfig: {
        quality: 0.8,
        frameRate: 15,
        enableAudioAdaptation: true,
      },
    };
  }
  
  // High-end device
  return {
    captureConfig: {
      quality: 0.95,
      frameRate: 30,
      enableAudioAdaptation: false, // Can handle both
    },
  };
}
```

### Network Optimization

Reduce bandwidth usage for slow connections.

```tsx theme={null}
function getNetworkOptimizedConfig() {
  const connection = navigator.connection;
  const effectiveType = connection?.effectiveType || '4g';
  
  const configs = {
    'slow-2g': { quality: 0.4, frameRate: 5 },
    '2g': { quality: 0.5, frameRate: 8 },
    '3g': { quality: 0.7, frameRate: 12 },
    '4g': { quality: 0.9, frameRate: 20 },
  };
  
  return {
    captureConfig: configs[effectiveType] || configs['4g'],
  };
}
```

## Best Practices

<Check>
  **Profile First**: Use browser DevTools to identify bottlenecks before optimizing
</Check>

<Check>
  **Start Conservative**: Begin with lower quality settings and increase as needed
</Check>

<Check>
  **Monitor Metrics**: Track performance in production to catch issues early
</Check>

<Check>
  **Test on Target Devices**: Always test on actual devices your users will use
</Check>

<Check>
  **Use Workers**: Keep worker mode enabled unless you have specific reasons not to
</Check>

<Check>
  **Batch Operations**: Group multiple operations together when possible
</Check>

## Performance Benchmarks

### Expected Performance

| Metric               | Good     | Acceptable | Poor    |
| -------------------- | -------- | ---------- | ------- |
| Capture Time         | \< 50ms  | 50-150ms   | > 150ms |
| Audio Latency        | \< 100ms | 100-300ms  | > 300ms |
| Memory Usage         | \< 50MB  | 50-150MB   | > 150MB |
| CPU Usage            | \< 30%   | 30-60%     | > 60%   |
| FPS (during capture) | > 30     | 15-30      | \< 15   |

### Optimization Checklist

<Tabs>
  <Tab title="Initial Setup">
    * [ ] Enable worker mode
    * [ ] Configure audio adaptation
    * [ ] Set appropriate quality
    * [ ] Test on target devices
    * [ ] Monitor initial metrics
  </Tab>

  <Tab title="Fine-Tuning">
    * [ ] Adjust capture intervals
    * [ ] Optimize quality settings
    * [ ] Configure batch sizes
    * [ ] Tune noise gate
    * [ ] Test under load
  </Tab>

  <Tab title="Production">
    * [ ] Enable observability
    * [ ] Set up monitoring
    * [ ] Configure alerts
    * [ ] Plan for scaling
    * [ ] Document settings
  </Tab>
</Tabs>

## Advanced Techniques

### Custom Worker Implementation

```tsx theme={null}
// Create custom worker for specific tasks
class CustomProcessor {
  constructor() {
    this.worker = new Worker(
      URL.createObjectURL(new Blob([`
        self.onmessage = function(e) {
          // Custom processing logic
          const result = processData(e.data);
          self.postMessage(result);
        }
      `], { type: 'application/javascript' }))
    );
  }
  
  process(data) {
    return new Promise((resolve) => {
      this.worker.onmessage = (e) => resolve(e.data);
      this.worker.postMessage(data);
    });
  }
}
```

### Throttling and Debouncing

```tsx theme={null}
import { useThrottle, useDebounce } from '@sammy-labs/sammy-three';

function OptimizedCapture() {
  // Throttle captures to max once per second
  const throttledCapture = useThrottle(captureScreen, 1000);
  
  // Debounce UI updates to reduce re-renders
  const debouncedUpdate = useDebounce(updateUI, 300);
  
  return (
    <div onChange={debouncedUpdate}>
      {/* UI components */}
    </div>
  );
}
```

## Related Features

<Columns cols={3}>
  <Card title="Audio Processing" icon="waveform" href="/features/audio-processing">
    Optimize audio for better performance
  </Card>

  <Card title="Observability" icon="chart-line" href="/features/observability">
    Monitor performance in production
  </Card>

  <Card title="Error Handling" icon="shield-exclamation" href="/features/error-handling">
    Handle performance-related errors
  </Card>
</Columns>