> ## Documentation Index
> Fetch the complete documentation index at: https://docs.sammylabs.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Voice Activity Detection (VAD)

> Configure VAD settings to prevent stuttering and false interruptions caused by background noise in Gemini Live API

<Note>
  The Voice Activity Detection (VAD) system determines when a user is speaking and when they've stopped. Proper configuration is crucial for smooth conversation experiences.
</Note>

## The Problem

<Warning>
  By default, the Gemini Live API's VAD can be overly sensitive to background noise, causing:

  * Interpretation of background sounds as user speech
  * Interruption of the agent mid-response
  * "Stuttering" where the agent repeatedly stops and starts
</Warning>

## The Solution: VAD Presets

We've implemented configurable VAD sensitivity presets to handle different noise environments effectively.

## Available Presets

### Low Sensitivity (Default - Recommended)

<Card title="Low Sensitivity" icon="volume-low">
  **Best for noisy environments or when experiencing stuttering issues.**

  ```javascript theme={null}
  vadConfig: {
    sensitivity: 'low'
  }
  ```

  **Settings:**

  * Start Sensitivity: LOW
  * End Sensitivity: LOW
  * Prefix Padding: 150ms (requires sustained speech)
  * Silence Duration: 500ms (longer pause needed to end turn)

  **Use when:**

  * Working in noisy environments (cafes, open offices)
  * Experiencing frequent false interruptions
  * Background noise is causing stuttering
  * Using speakers instead of headphones (echo/feedback issues)
</Card>

### Medium Sensitivity

<Card title="Medium Sensitivity" icon="volume">
  **Balanced for typical home/office environments.**

  ```javascript theme={null}
  vadConfig: {
    sensitivity: 'medium'
  }
  ```

  **Settings:**

  * Start Sensitivity: MEDIUM
  * End Sensitivity: MEDIUM
  * Prefix Padding: 100ms
  * Silence Duration: 400ms

  **Use when:**

  * Working in moderately quiet environments
  * Want balanced responsiveness
  * Occasional background noise but not constant
</Card>

### High Sensitivity

<Card title="High Sensitivity" icon="volume-high">
  **For very quiet environments where quick response is needed.**

  ```javascript theme={null}
  vadConfig: {
    sensitivity: 'high'
  }
  ```

  **Settings:**

  * Start Sensitivity: HIGH
  * End Sensitivity: HIGH
  * Prefix Padding: 40ms (quick response)
  * Silence Duration: 300ms (quick turn ending)

  **Use when:**

  * Working in silent environments
  * Using high-quality microphone with noise cancellation
  * Need immediate response to speech
  * No background noise present
</Card>

### Custom Settings

<Card title="Custom Configuration" icon="sliders">
  **Fine-tune VAD parameters for your specific needs.**

  ```javascript theme={null}
  vadConfig: {
    sensitivity: 'custom',
    customSettings: {
      startSensitivity: StartSensitivity.START_SENSITIVITY_LOW,
      endSensitivity: EndSensitivity.END_SENSITIVITY_MEDIUM,
      prefixPaddingMs: 200,
      silenceDurationMs: 600
    }
  }
  ```
</Card>

## Implementation Examples

### Basic Usage

```javascript theme={null}
import { SammyProvider } from '@sammy-three/sammy-three';

function App() {
  return (
    <SammyProvider
      config={{
        auth: { /* your auth config */ },
        // Use low sensitivity to prevent stuttering
        vadConfig: {
          sensitivity: 'low'
        }
      }}
    >
      {/* Your app */}
    </SammyProvider>
  );
}
```

### Dynamic Environment Switching

<CodeGroup>
  ```javascript Component theme={null}
  import { useSammyAgent } from '@sammy-three/sammy-three';

  function VADControl() {
    const { updateConfig } = useSammyAgent();
    
    const handleEnvironmentChange = (environment) => {
      let sensitivity;
      switch(environment) {
        case 'noisy':
          sensitivity = 'low';
          break;
        case 'office':
          sensitivity = 'medium';
          break;
        case 'quiet':
          sensitivity = 'high';
          break;
        default:
          sensitivity = 'low';
      }
      
      updateConfig({
        vadConfig: { sensitivity }
      });
    };
    
    return (
      <select onChange={(e) => handleEnvironmentChange(e.target.value)}>
        <option value="noisy">Noisy Environment</option>
        <option value="office">Office</option>
        <option value="quiet">Quiet Room</option>
      </select>
    );
  }
  ```

  ```javascript Environment Presets theme={null}
  import { createAudioConfig } from '@sammy-three/sammy-three';

  // Use office preset
  const audioConfig = createAudioConfig('office');

  // Or noisy environment preset
  const audioConfig = createAudioConfig('noisy');
  ```
</CodeGroup>

## Troubleshooting Guide

### Common Issues

<Tabs>
  <Tab title="Agent Stops Mid-Sentence">
    **Problem:** Agent keeps stopping mid-sentence

    **Solution:** Use 'low' sensitivity preset or increase `prefixPaddingMs` in custom config.
  </Tab>

  <Tab title="Slow Response to Interruptions">
    **Problem:** Agent doesn't respond quickly to interruptions

    **Solution:** Try 'medium' sensitivity or decrease `silenceDurationMs`.
  </Tab>

  <Tab title="Background Noise Interruptions">
    **Problem:** Background typing/clicking causes interruptions

    **Solution:** Use 'low' sensitivity and consider using push-to-talk mode.
  </Tab>

  <Tab title="Cuts Off During Pauses">
    **Problem:** Agent cuts off when user pauses briefly

    **Solution:** Increase `silenceDurationMs` to allow for natural pauses.
  </Tab>
</Tabs>

## Additional Audio Optimizations

### Hardware Solutions

<Columns cols={2}>
  <Card title="Use Headphones" icon="headphones">
    Prevents speaker feedback
  </Card>

  <Card title="Directional Mic" icon="microphone">
    Use a directional microphone
  </Card>

  <Card title="Hardware Cancellation" icon="shield">
    Enable hardware noise cancellation
  </Card>

  <Card title="Acoustic Treatment" icon="home">
    Consider room acoustics
  </Card>
</Columns>

### Browser Noise Suppression

<Info>
  The package includes built-in noise suppression that works alongside VAD:
</Info>

```javascript theme={null}
// In noise-config.ts
noiseSuppression: {
  enabled: true,
  enhancementLevel: 'medium', // 'light', 'medium', or 'aggressive'
  fallbackToBasic: true
}
```

## Testing Your Configuration

<Steps>
  <Step title="Start with Low Sensitivity">
    Begin with the 'low' preset to establish a baseline without stuttering.
  </Step>

  <Step title="Test in Your Environment">
    Make some background noise typical to your environment:

    * Type on your keyboard
    * Move papers around
    * Have background conversations
  </Step>

  <Step title="Gradually Increase Sensitivity">
    If the agent is not responsive enough, try 'medium' sensitivity.
  </Step>

  <Step title="Fine-tune with Custom Settings">
    If presets don't work perfectly, use custom settings to dial in the exact behavior you need.
  </Step>
</Steps>

## Best Practices

<Columns cols={2}>
  <Card title="Default to Low" icon="gauge-low">
    Start with 'low' sensitivity and only increase if needed
  </Card>

  <Card title="Consider Environment" icon="building">
    Choose presets based on your typical working environment
  </Card>

  <Card title="Use Headphones" icon="headphones">
    This prevents feedback loops and false interruptions
  </Card>

  <Card title="Monitor Logs" icon="terminal">
    Check console logs for VAD configuration details:

    ```
    [Agent Config] VAD settings: { preset: 'low', config: {...} }
    ```
  </Card>

  <Card title="Combine with Noise Gates" icon="filter">
    The audio pipeline includes noise gates that work with VAD to filter out background noise
  </Card>

  <Card title="Test Thoroughly" icon="flask">
    Test in your actual working environment, not just quiet spaces
  </Card>
</Columns>

## Migration Guide

<Warning>
  If you're upgrading from a version without VAD configuration:
</Warning>

<Tabs>
  <Tab title="Before (Stuttering)">
    ```javascript theme={null}
    // No VAD configuration available
    // Uses HIGH sensitivity by default
    <SammyProvider config={config}>
    ```
  </Tab>

  <Tab title="After (With VAD Control)">
    ```javascript theme={null}
    // Explicitly set VAD sensitivity
    <SammyProvider 
      config={{
        ...config,
        vadConfig: {
          sensitivity: 'low' // Prevents stuttering
        }
      }}
    >
    ```
  </Tab>
</Tabs>

## VAD Parameters Reference

<Tabs>
  <Tab title="Start Sensitivity">
    Controls how easily VAD detects speech start:

    * **LOW**: Requires clear, sustained speech
    * **MEDIUM**: Balanced detection
    * **HIGH**: Detects even slight sounds
  </Tab>

  <Tab title="End Sensitivity">
    Controls how easily VAD detects speech end:

    * **LOW**: Requires longer silence
    * **MEDIUM**: Balanced detection
    * **HIGH**: Quick to detect speech end
  </Tab>

  <Tab title="Prefix Padding">
    Milliseconds of audio before speech detection:

    * **Higher**: More context, fewer false starts
    * **Lower**: Quicker response
  </Tab>

  <Tab title="Silence Duration">
    Milliseconds of silence to end turn:

    * **Higher**: Allows for natural pauses
    * **Lower**: Quicker turn-taking
  </Tab>
</Tabs>

## Debug Mode

Enable debug logging to see VAD behavior:

```javascript theme={null}
vadConfig: {
  sensitivity: 'low',
  debug: true
}

// Console output:
// [VAD] Speech detected with confidence: 0.92
// [VAD] Silence detected, duration: 450ms
// [VAD] Turn ended after 500ms silence
```

## Conclusion

<Note>
  Proper VAD configuration is crucial for a smooth conversation experience with the Gemini Live API. By defaulting to 'low' sensitivity and providing easy configuration options, we've addressed the common stuttering issues while maintaining flexibility for different use cases.

  **Remember: When in doubt, use 'low' sensitivity** - it's better to require clearer speech than to have constant false interruptions.
</Note>