Audio Analysis

Oscilla can analyze live audio input from a microphone and publish signal values to the ParamBus. This enables scores that respond to sound - triggering cues based on loudness, brightness, or attack transients.

Enabling Audio Analysis

Click the microphone icon in the toolbar (in the audio tools cluster) to start audio analysis. The browser will request microphone permission on first use.

When active, the button shows an "active" state and audio signals are published continuously.

Published Signals

Audio analysis publishes the following signals under mic.*:

Signal	Range	Description
`mic.amp`	0-1	RMS amplitude (overall loudness)
`mic.peak`	0-1	Peak amplitude
`mic.centroid`	0-1	Spectral centroid (brightness - higher = brighter)
`mic.onset`	0-1	Onset detection (spikes to 1 on attacks, decays)
`mic.low`	0-1	Low frequency energy (~0-300Hz)
`mic.mid`	0-1	Mid frequency energy (~300-2000Hz)
`mic.high`	0-1	High frequency energy (~2000Hz+)
`mic.pitch`	Hz	Detected pitch in Hz (0 if no clear pitch)
`mic.note`	0-127	MIDI note number (60 = middle C)
`mic.pitchConf`	0-1	Pitch detection confidence

Using with Conditional Cues

Combine audio signals with the if: parameter to create sound-reactive scores:

audio(src:accent.wav, if:mic.amp>0.7)
scale(dur:2, if:mic.onset>0.5)
nav(target:loud, if:mic.amp>=0.8)
fade(target:glow, to:1, dur:0.1, if:mic.high>0.6)

Examples

Amplitude-gated playback

audio(src:response.wav, if:mic.amp>0.5)

Audio file plays only when input exceeds threshold.

Onset-triggered animation

rotate(dur:0.5, values:[0,90], if:mic.onset>0.8)

Element rotates on sharp attacks.

Brightness-responsive navigation

nav(target:bright, if:mic.centroid>0.7)
nav(target:dark, if:mic.centroid<=0.3)

Score branches based on timbral brightness.

Multi-band response

scale(dur:0.2, values:[1,1.5,1], if:mic.low>0.6)
color(to:#ff0000, dur:0.1, if:mic.high>0.7)

Different visual responses to bass vs treble.

Pitch-based navigation

nav(target:high, if:mic.pitch>500)
nav(target:low, if:mic.pitch<200)

Score branches based on sung/played pitch.

MIDI note triggers

audio(src:c4.wav, if:mic.note==60)
audio(src:g4.wav, if:mic.note==67)

Trigger samples when specific notes are detected (60 = middle C, 67 = G4).

Pitch register response

scale(dur:0.5, if:mic.note>72)   // above C5
fade(to:0.5, if:mic.note<48)     // below C3

Different visual responses to pitch register.

Technical Notes

Analysis runs at display frame rate (~60fps)
Signals are smoothed to reduce jitter
Onset detection uses amplitude change threshold with decay
Microphone is not routed to speakers (analysis only)
Signals persist at 0 when analysis is stopped

Programmatic Control

import { startAudioAnalysis, stopAudioAnalysis, toggleAudioAnalysis } from './control/audioAnalysis.js';

// Start analysis
await startAudioAnalysis();

// Stop analysis
stopAudioAnalysis();

// Toggle
await toggleAudioAnalysis();

// Also available on window for debugging
window.oscillaAudioAnalysis.start();
window.oscillaAudioAnalysis.stop();