Audio Analysis

Oscilla can analyze live audio input from a microphone and publish signal values to the ParamBus. This enables scores that respond to sound - triggering cues based on loudness, brightness, or attack transients.

Enabling Audio Analysis

Analysis is started by the adc() cue, named after the adc~ object in Pd and Max.

adc()

The browser asks for microphone permission the first time it runs. From then on the signals below are published continuously, and any cue in the score can bind to them.

Form	Effect
`adc()`	start listening; publish under `adc.*`
`adc(uid:room)`	publish under `room.*` instead, so two scores can keep their sources apart
`adc(off:1)`	stop listening
`adc(ch:l)`	listen to the left channel only (`l`/`left`/`0`)
`adc(ch:r)`	…or the right (`r`/`right`/`1`)

Firing adc() again while it is already running does nothing.

Stereo

ch: picks one channel of the input. A stereo pair is two cues with two uids, the way adc~ gives you two outlets to patch separately:

adc(ch:l, uid:left)
adc(ch:r, uid:right)

synth(uid:s1, freq: follow(left.pitch, 90, 2000))
scale(uid:box, sx: map(right.amp, 0, 0.4, 0.5, 3))

Both taps share one getUserMedia, so the performer is asked for the microphone once however many channels the score listens to. Each keeps its own onset envelope and its own smoothed amplitude and pitch — they are independent sources, not two views of one.

Without ch: the analyser mixes the input down itself, which is the right reading for a mono microphone and is what happened before channels existed. On a mono input, asking for ch:r logs a warning and listens to channel 0, rather than silently reporting nothing. adc(uid:left, off:1) stops just that tap; a bare adc(off:1) releases the input entirely.

The parameter is off: rather than stop: because stop is a cue name in the grammar and would be read as one.

Published Signals

Audio analysis publishes the following signals under adc.* (or under <uid>.* if you passed uid:):

Signal	Range	Description
`adc.amp`	0-1	RMS amplitude (overall loudness)
`adc.peak`	0-1	Peak amplitude
`adc.centroid`	0-1	Spectral centroid (brightness - higher = brighter)
`adc.onset`	0-1	Onset detection (spikes to 1 on attacks, decays)
`adc.low`	0-1	Low frequency energy (~0-300Hz)
`adc.mid`	0-1	Mid frequency energy (~300-2000Hz)
`adc.high`	0-1	High frequency energy (~2000Hz+)
`adc.pitch`	Hz	Detected pitch in Hz (0 if no clear pitch)
`adc.note`	0-127	MIDI note number (60 = middle C)
`adc.pitchConf`	0-1	Pitch detection confidence

Using with Conditional Cues

Combine audio signals with the if: parameter to create sound-reactive scores:

audio(src:accent.wav, if:"adc.amp>0.7")
scale(dur:2, if:"adc.onset>0.5")
fade(target:glow, to:1, dur:0.1, if:"adc.high>0.6")

Examples

Amplitude-gated playback

audio(src:response.wav, if:"adc.amp>0.5")

Audio file plays only when input exceeds threshold.

Onset-triggered animation

rotate(dur:0.5, values:[0,90], if:"adc.onset>0.8")

Element rotates on sharp attacks.

Multi-band response

scale(dur:0.2, values:[1,1.5,1], if:"adc.low>0.6")
color(to:#ff0000, dur:0.1, if:"adc.high>0.7")

Different visual responses to bass vs treble.

MIDI note triggers

audio(src:c4.wav, if:"adc.note==60")
audio(src:g4.wav, if:"adc.note==67")

Trigger samples when specific notes are detected (60 = middle C, 67 = G4).

Pitch register response

scale(dur:0.5, if:"adc.note>72")   // above C5
fade(to:0.5, if:"adc.note<48")     // below C3

Different visual responses to pitch register.

Technical Notes

Analysis runs at display frame rate (~60fps)
Signals are smoothed to reduce jitter
Onset detection uses amplitude change threshold with decay
Microphone is not routed to speakers (analysis only)
Signals persist at 0 when analysis is stopped

Programmatic Control

adc() is the normal route. These entry points exist for extensions and for debugging from the live console.

import { startAudioAnalysis, stopAudioAnalysis, toggleAudioAnalysis } from './control/audioAnalysis.js';

// Start analysis
await startAudioAnalysis();

// Stop analysis
stopAudioAnalysis();

// Toggle
await toggleAudioAnalysis();

// Also available on window for debugging
window.oscillaAudioAnalysis.start();
window.oscillaAudioAnalysis.stop();