Audio Processing

SAMMY Three includes advanced audio processing capabilities to ensure high-quality voice interactions in any environment.

Overview

The audio processing system provides multiple layers of enhancement to deliver crystal-clear voice communication.

Noise Suppression

Remove background noise with browser-native or AI-powered suppression

Noise Gate

Software-based filtering to eliminate ambient sounds

Environment Presets

Pre-configured settings optimized for different environments

Configuration

Enable audio processing in your SAMMY Three configuration.

const config = {
  audioConfig: {
    noiseSuppression: {
      enabled: true,
      enhancementLevel: 'medium',
    },
    noiseGate: {
      enabled: true,
      threshold: 0.04,
    },
    environmentPreset: 'office',
  },
};

Noise Suppression

Two powerful noise suppression modes are available to match your needs.

Browser Native
Koala AI

Default Browser Suppression

Uses the browser’s built-in echo cancellation and noise suppression capabilities.

const config = {
  audioConfig: {
    noiseSuppression: {
      enabled: true,
      enhancementLevel: 'light', // 'light', 'medium', 'aggressive'
    },
  },
};

Pros:

No additional configuration required
Works on all modern browsers
Zero latency
Free to use

Cons:

Limited effectiveness in very noisy environments
Quality varies by browser and device

Advanced AI Suppression

Professional-grade noise removal using Koala AI’s neural networks.

const config = {
  audioConfig: {
    noiseSuppression: {
      enabled: true,
      enhancementLevel: 'medium',
      // Koala-specific configuration
      accessKey: process.env.KOALA_ACCESS_KEY,
      modelPath: '/models/koala_model.pv',
    },
  },
};

Pros:

Superior noise removal quality
Consistent across all platforms
Handles extreme noise conditions
Professional broadcast quality

Cons:

Requires Koala access key
Additional model download
Small processing latency

Enhancement Levels

Choose the right suppression level for your environment:

Light

Best for: Quiet offices, home officesMinimal processing that preserves natural voice quality while removing light background noise.

enhancementLevel: 'light'

Medium

Best for: Open offices, cafesBalanced approach that removes moderate noise while maintaining voice clarity.

enhancementLevel: 'medium'

Aggressive

Best for: Noisy environments, public spacesMaximum noise removal that prioritizes voice isolation over natural sound.

enhancementLevel: 'aggressive'

Noise Gate

Software-based noise gate filters out background noise between speech segments.

How It Works

The noise gate acts like an automatic mute button, opening only when you speak and closing during silence to eliminate ambient noise.

const config = {
  audioConfig: {
    noiseGate: {
      enabled: true,
      threshold: 0.04,      // Volume threshold (0-1)
      attackTime: 30,       // How fast gate opens (ms)
      holdTime: 400,        // Hold open during pauses (ms)
      releaseTime: 150,     // How fast gate closes (ms)
    },
  },
};

Configuration Parameters

number

Volume level (0-1) required to open the gate. Lower values are more sensitive.

0.02-0.03: Very sensitive, good for quiet environments
0.04-0.06: Standard setting for most environments
0.07-0.10: Less sensitive, for noisy environments

number

default:30

Time in milliseconds for the gate to fully open when speech is detected.

10-20ms: Very fast, may clip beginning of words
30-50ms: Standard, natural sounding
60-100ms: Slower, smoother transitions

number

default:400

Time in milliseconds to keep gate open after speech stops, preventing choppy audio.

200-300ms: Quick release, good for fast conversations
400-500ms: Standard, handles normal pauses
600-800ms: Longer hold, better for thoughtful speech

number

default:150

Time in milliseconds for the gate to fully close after hold time expires.

50-100ms: Fast close, may sound abrupt
150-200ms: Standard, natural fade
250-400ms: Slow close, very smooth

Environment Presets

Pre-configured audio settings optimized for common environments.

// Quiet office environment
{
  audioConfig: {
    environmentPreset: 'office',
  },
}

// Automatically configures:
// - Light noise suppression
// - Low noise gate threshold (0.03)
// - Standard timing parameters

// Home environment with moderate noise
{
  audioConfig: {
    environmentPreset: 'home',
  },
}

// Automatically configures:
// - Medium noise suppression
// - Medium threshold (0.04)
// - Balanced parameters

// Cafe, public space, or noisy environment
{
  audioConfig: {
    environmentPreset: 'noisy',
  },
}

// Automatically configures:
// - Aggressive noise suppression
// - Higher threshold (0.06)
// - Longer hold times

// Professional recording environment
{
  audioConfig: {
    environmentPreset: 'studio',
  },
}

// Automatically configures:
// - Minimal processing
// - Very low threshold (0.02)
// - Fast response times

// Manual configuration
{
  audioConfig: {
    environmentPreset: 'custom',
    // Specify all parameters manually
  },
}

Audio Debugging

Built-in tools to diagnose and fix audio issues.

Stutter Analyzer

Debug audio stuttering and performance issues:

// Enable performance monitoring
const config = {
  debugAudioPerformance: true,
};

// In browser console:
audioStutterAnalyzer.setDebugMode(true);  // Start collection
audioStutterAnalyzer.getAnalysis();       // View analysis
audioStutterAnalyzer.clear();             // Clear buffer

// The analyzer tracks:
// - Audio underruns (stuttering)
// - Long DOM captures during audio
// - Buffer statistics
// - Correlation between renders and stutters

Common Issues and Solutions

Echo Problems
Choppy Audio
Background Noise
Muffled Voice

Issue: Hearing echo or feedbackSolutions:

// Enable echo cancellation
{
  audioConfig: {
    noiseSuppression: {
      enabled: true,
      // Echo cancellation is automatic
    },
  },
}

Additional Tips:

Use headphones when possible
Reduce speaker volume
Increase distance between mic and speakers

Issue: Words getting cut offSolutions:

// Adjust noise gate timing
{
  audioConfig: {
    noiseGate: {
      enabled: true,
      threshold: 0.03,     // Lower threshold
      attackTime: 20,      // Faster opening
      holdTime: 500,       // Longer hold
    },
  },
}

Issue: Too much background noiseSolutions:

// Use aggressive filtering
{
  audioConfig: {
    environmentPreset: 'noisy',
    // or
    noiseSuppression: {
      enabled: true,
      enhancementLevel: 'aggressive',
    },
  },
}

Issue: Voice sounds unnatural or muffledSolutions:

// Reduce processing
{
  audioConfig: {
    noiseSuppression: {
      enabled: true,
      enhancementLevel: 'light',
    },
    noiseGate: {
      enabled: false, // Try disabling gate
    },
  },
}

Best Practices

Test in Target Environment: Always test audio settings in the actual environment where the agent will be used

Start with Presets: Use environment presets as a starting point, then fine-tune if needed

Monitor Performance: Enable debug mode during development to catch audio issues early

User Feedback: Provide visual feedback for volume levels and mute status

Fallback Options: Have a text input fallback for environments where audio isn’t suitable

Advanced Configuration

Combining Multiple Techniques

Layer different audio processing techniques for optimal results:

const config = {
  audioConfig: {
    // Start with a preset
    environmentPreset: 'office',
    
    // Override specific settings
    noiseSuppression: {
      enabled: true,
      enhancementLevel: 'medium',
    },
    
    // Fine-tune noise gate
    noiseGate: {
      enabled: true,
      threshold: 0.045,
      holdTime: 450,
    },
  },
};

Dynamic Adjustment

Adjust audio settings based on user feedback or environment detection:

function adjustAudioForEnvironment(noiseLevel: 'low' | 'medium' | 'high') {
  const presets = {
    low: { environmentPreset: 'office' },
    medium: { environmentPreset: 'home' },
    high: { environmentPreset: 'noisy' },
  };
  
  // Update configuration dynamically
  updateAgentConfig({
    audioConfig: presets[noiseLevel],
  });
}

Platform Considerations

Audio processing behavior may vary across different platforms and browsers. Always test on your target platforms.

Browser Support

Feature	Chrome	Firefox	Safari	Edge
Native Suppression	✅	✅	✅	✅
Echo Cancellation	✅	✅	✅	✅
Noise Gate	✅	✅	✅	✅
Koala AI	✅	✅	⚠️	✅

Safari may require additional permissions for advanced audio processing features.

VAD Configuration

Configure voice activity detection settings

Performance Optimization

Optimize audio processing for better performance

Get Started

Features

Integrations

Audio Processing

Audio Processing

Overview

Noise Suppression

Noise Gate

Environment Presets

Configuration

Noise Suppression

Default Browser Suppression

Advanced AI Suppression

Enhancement Levels

Noise Gate

How It Works

Configuration Parameters

Environment Presets

Audio Debugging

Stutter Analyzer

Common Issues and Solutions

Best Practices

Advanced Configuration

Combining Multiple Techniques

Dynamic Adjustment

Platform Considerations

Browser Support

VAD Configuration

Performance Optimization

​Audio Processing

​Overview

Noise Suppression

Noise Gate

Environment Presets

​Configuration

​Noise Suppression

​Default Browser Suppression

​Advanced AI Suppression

​Enhancement Levels

​Noise Gate

​How It Works

​Configuration Parameters

​Environment Presets

​Audio Debugging

​Stutter Analyzer

​Common Issues and Solutions

​Best Practices

​Advanced Configuration

​Combining Multiple Techniques

​Dynamic Adjustment

​Platform Considerations

​Browser Support

​Related Features

VAD Configuration

Performance Optimization

Audio Processing

Overview

Configuration

Noise Suppression

Default Browser Suppression

Advanced AI Suppression

Enhancement Levels

Noise Gate

How It Works

Configuration Parameters

Environment Presets

Audio Debugging

Stutter Analyzer

Common Issues and Solutions

Best Practices

Advanced Configuration

Combining Multiple Techniques

Dynamic Adjustment

Platform Considerations

Browser Support

Related Features