Performance Optimization
Achieve optimal performance with SAMMY Three’s built-in optimization strategies and worker-based architecture.
Overview
SAMMY Three is designed for high performance with multiple optimization layers working together.
Worker Architecture
Offload heavy processing to background workers
Audio-Aware Capture
Intelligent throttling during audio playback
Critical Renders
Automatic capture at conversation boundaries
Worker Architecture
SAMMY Three uses multiple workers to prevent main thread blocking.
Worker Types
Canvas Worker
Observability Worker
DOM Capture Workers
Handles image encoding and compression.// Automatic - no configuration needed
// Processes screen captures in background
// Falls back to main thread if unavailable
- JPEG/PNG encoding
- Image compression
- Resolution scaling
- Format conversion
Batches and sends analytics events.const config = {
observability: {
useWorker: true, // Enable worker mode
workerConfig: {
batchSize: 50,
batchIntervalMs: 5000,
},
},
};
- Event batching
- API request queuing
- Data compression
- Retry logic
Generates screenshots efficiently.// Multiple workers for parallel processing
// Automatic load balancing
// CSP-compliant Data URL approach
- DOM traversal
- Style computation
- SVG generation
- HTML rendering
Worker Benefits
Non-Blocking UI
Main thread remains responsive during heavy operations
Parallel Processing
Multiple operations execute simultaneously
Automatic Fallback
Gracefully degrades to main thread if workers unavailable
CSP Compliance
Uses Data URLs instead of blob URLs for security
Audio-Aware Capture
Screen capture automatically adapts based on audio state to prevent stuttering.
How It Works
const config = {
captureConfig: {
enableAudioAdaptation: true, // Default: true
quality: 0.8,
},
};
Normal Mode
When: No audio playing
- High-frequency captures (100ms-2s)
- Full quality rendering
- Maximum detail capture
Audio Mode
When: Agent speaking
- Reduced captures (500ms-5s)
- Optimized quality
- Prevents audio stuttering
Configuration
// Fine-tune audio adaptation
const config = {
captureConfig: {
enableAudioAdaptation: true,
// Normal mode settings
normalInterval: 1000, // ms between captures
normalQuality: 0.9, // JPEG quality
// Audio mode settings
audioInterval: 3000, // ms between captures
audioQuality: 0.7, // Reduced quality
},
};
Critical DOM Renders
Ensures fresh visual context at important conversation moments.
Automatic Triggers
Critical renders happen automatically at:
Speech Boundaries
- User starts speaking
- User stops speaking
- Agent starts response
- Agent completes response
Interruptions
- User interrupts agent
- Agent pauses for user
- Conversation direction changes
State Changes
- Page navigation
- Major UI updates
- Form submissions
- Modal opens/closes
Benefits
Agent always has current visual context
No stale UI information during conversations
Captures happen even when regular capture is throttled
Minimal performance impact with smart timing
Memory Management
Optimize memory usage for long-running sessions.
Automatic Cleanup
// SAMMY Three automatically manages:
- Old screen captures
- Audio buffers
- Processed frames
- Event queues
Manual Optimization
import { useSammyAgentContext } from '@sammy-labs/sammy-three';
function MemoryOptimizedChat() {
const { clearCache, getMemoryUsage } = useSammyAgentContext();
// Monitor memory usage
useEffect(() => {
const interval = setInterval(() => {
const usage = getMemoryUsage();
if (usage > 100 * 1024 * 1024) { // 100MB
clearCache();
}
}, 60000); // Check every minute
return () => clearInterval(interval);
}, []);
return <ChatInterface />;
}
Capture Optimization
Quality Settings
Balance quality and performance based on your needs.
// Best for detailed UIs
const config = {
captureConfig: {
quality: 0.95,
frameRate: 30,
resolution: {
maxWidth: 1920,
maxHeight: 1080,
},
},
};
Capture Methods Comparison
| Method | CPU Usage | Quality | Best For |
|---|
render | Low-Medium | High | Web applications |
video | Medium-High | Variable | Full screen capture |
Use render method for most web applications. It’s more efficient and provides consistent quality.
Audio Stutter Analysis
Debug and fix audio performance issues.
// Enable debugging
const config = {
debugAudioPerformance: true,
};
// In browser console
audioStutterAnalyzer.setDebugMode(true);
audioStutterAnalyzer.getAnalysis();
// Returns detailed metrics:
{
stutterCount: 3,
underruns: [/* timestamps */],
longCaptures: [/* durations */],
averageBufferLevel: 0.8,
correlationScore: 0.7
}
import { usePerformanceMonitor } from '@sammy-labs/sammy-three';
function PerformanceDebugger() {
const metrics = usePerformanceMonitor();
return (
<div className="debug-panel">
<div>FPS: {metrics.fps}</div>
<div>Capture Time: {metrics.captureTime}ms</div>
<div>Memory: {metrics.memoryUsage}MB</div>
<div>Worker Queue: {metrics.workerQueueSize}</div>
</div>
);
}
Optimization Strategies
Device-Based Configuration
Adjust settings based on device capabilities.
function getOptimalConfig() {
const memory = navigator.deviceMemory || 4; // GB
const cores = navigator.hardwareConcurrency || 4;
if (memory <= 2 || cores <= 2) {
// Low-end device
return {
captureConfig: {
quality: 0.6,
frameRate: 10,
enableAudioAdaptation: true,
},
observability: {
useWorker: false, // Avoid worker overhead
},
};
}
if (memory <= 4 || cores <= 4) {
// Mid-range device
return {
captureConfig: {
quality: 0.8,
frameRate: 15,
enableAudioAdaptation: true,
},
};
}
// High-end device
return {
captureConfig: {
quality: 0.95,
frameRate: 30,
enableAudioAdaptation: false, // Can handle both
},
};
}
Network Optimization
Reduce bandwidth usage for slow connections.
function getNetworkOptimizedConfig() {
const connection = navigator.connection;
const effectiveType = connection?.effectiveType || '4g';
const configs = {
'slow-2g': { quality: 0.4, frameRate: 5 },
'2g': { quality: 0.5, frameRate: 8 },
'3g': { quality: 0.7, frameRate: 12 },
'4g': { quality: 0.9, frameRate: 20 },
};
return {
captureConfig: configs[effectiveType] || configs['4g'],
};
}
Best Practices
Profile First: Use browser DevTools to identify bottlenecks before optimizing
Start Conservative: Begin with lower quality settings and increase as needed
Monitor Metrics: Track performance in production to catch issues early
Test on Target Devices: Always test on actual devices your users will use
Use Workers: Keep worker mode enabled unless you have specific reasons not to
Batch Operations: Group multiple operations together when possible
| Metric | Good | Acceptable | Poor |
|---|
| Capture Time | < 50ms | 50-150ms | > 150ms |
| Audio Latency | < 100ms | 100-300ms | > 300ms |
| Memory Usage | < 50MB | 50-150MB | > 150MB |
| CPU Usage | < 30% | 30-60% | > 60% |
| FPS (during capture) | > 30 | 15-30 | < 15 |
Optimization Checklist
Initial Setup
Fine-Tuning
Production
Advanced Techniques
Custom Worker Implementation
// Create custom worker for specific tasks
class CustomProcessor {
constructor() {
this.worker = new Worker(
URL.createObjectURL(new Blob([`
self.onmessage = function(e) {
// Custom processing logic
const result = processData(e.data);
self.postMessage(result);
}
`], { type: 'application/javascript' }))
);
}
process(data) {
return new Promise((resolve) => {
this.worker.onmessage = (e) => resolve(e.data);
this.worker.postMessage(data);
});
}
}
Throttling and Debouncing
import { useThrottle, useDebounce } from '@sammy-labs/sammy-three';
function OptimizedCapture() {
// Throttle captures to max once per second
const throttledCapture = useThrottle(captureScreen, 1000);
// Debounce UI updates to reduce re-renders
const debouncedUpdate = useDebounce(updateUI, 300);
return (
<div onChange={debouncedUpdate}>
{/* UI components */}
</div>
);
}
