Audio Analysis

Oscilla can analyze live audio input from a microphone and publish signal values to the ParamBus. This enables scores that respond to sound - triggering cues based on loudness, brightness, or attack transients.

Enabling Audio Analysis

Analysis is started by the adc() cue, named after the adc~ object in Pd and Max. Place it on an element like any other cue:

adc()

The browser asks for microphone permission the first time it runs. From then on the signals below are published continuously, and any cue in the score can bind to them.

Form	Effect
`adc()`	start listening; publish under `adc.*`
`adc(uid:room)`	publish under `room.*` instead, so two scores can keep their sources apart
`adc(off:1)`	stop listening
`adc(ch:l)`	listen to the left channel only (`l`/`left`/`0`)
`adc(ch:r)`	…or the right (`r`/`right`/`1`)
`adc(trig:click)`	start on click rather than when the playhead arrives

Firing adc() again while it is already running does nothing, so it is safe to place one on every page of a multi-page score.

You can also place a mic from the + button in the top bar — choose mic in the Create Element panel and set the uid, channel and size there. That writes exactly the same cue, with a small microphone drawn on the score so the input is visible as an object you can patch from. See the DSL inspector.

Stereo

ch: picks one channel of the input. A stereo pair is two cues with two uids, the way adc~ gives you two outlets to patch separately:

adc(ch:l, uid:left)
adc(ch:r, uid:right)

synth(uid:s1, freq: follow(left.pitch, 90, 2000))
scale(uid:box, sx: map(right.amp, 0, 0.4, 0.5, 3))

Both taps share one getUserMedia, so the performer is asked for the microphone once however many channels the score listens to. Each keeps its own onset envelope and its own smoothed amplitude and pitch — they are independent sources, not two views of one.

Without ch: the analyser mixes the input down itself, which is the right reading for a mono microphone and is what happened before channels existed. On a mono input, asking for ch:r logs a warning and listens to channel 0, rather than silently reporting nothing. adc(uid:left, off:1) stops just that tap; a bare adc(off:1) releases the input entirely.

The parameter is off: rather than stop: because stop is a cue name in the grammar and would be read as one.

Earlier versions started analysis from a microphone button in the toolbar and published under mic.*. Both are gone: use adc(), and rename any mic.amp in an old score to adc.amp. A stale mic. reference is now reported at load rather than silently producing nothing.

Published Signals

Audio analysis publishes the following signals under adc.* (or under <uid>.* if you passed uid:):

Signal	Range	Description
`adc.amp`	0-1	RMS amplitude (overall loudness)
`adc.peak`	0-1	Peak amplitude
`adc.centroid`	0-1	Spectral centroid (brightness - higher = brighter)
`adc.onset`	0-1	Onset detection (spikes to 1 on attacks, decays)
`adc.low`	0-1	Low frequency energy (~0-300Hz)
`adc.mid`	0-1	Mid frequency energy (~300-2000Hz)
`adc.high`	0-1	High frequency energy (~2000Hz+)
`adc.pitch`	Hz	Detected pitch in Hz (0 if no clear pitch)
`adc.note`	0-127	MIDI note number (60 = middle C)
`adc.pitchConf`	0-1	Pitch detection confidence

Using with Conditional Cues

Combine audio signals with the if: parameter to create sound-reactive scores:

audio(src:accent.wav, if:"adc.amp>0.7")
scale(dur:2, if:"adc.onset>0.5")
nav(scroll@loud, if:"adc.amp>=0.8")
fade(target:glow, to:1, dur:0.1, if:"adc.high>0.6")

Examples

Amplitude-gated playback

audio(src:response.wav, if:"adc.amp>0.5")

Audio file plays only when input exceeds threshold.

Onset-triggered animation

rotate(dur:0.5, values:[0,90], if:"adc.onset>0.8")

Element rotates on sharp attacks.

Brightness-responsive navigation

nav(scroll@bright, if:"adc.centroid>0.7")
nav(scroll@dark, if:"adc.centroid<=0.3")

Score branches based on timbral brightness.

Multi-band response

scale(dur:0.2, values:[1,1.5,1], if:"adc.low>0.6")
color(to:#ff0000, dur:0.1, if:"adc.high>0.7")

Different visual responses to bass vs treble.

Pitch-based navigation

nav(scroll@high, if:"adc.pitch>500")
nav(scroll@low, if:"adc.pitch<200")

Score branches based on sung/played pitch.

MIDI note triggers

audio(src:c4.wav, if:"adc.note==60")
audio(src:g4.wav, if:"adc.note==67")

Trigger samples when specific notes are detected (60 = middle C, 67 = G4).

Pitch register response

scale(dur:0.5, if:"adc.note>72")   // above C5
fade(to:0.5, if:"adc.note<48")     // below C3

Different visual responses to pitch register.

Technical Notes

Analysis runs at display frame rate (~60fps)
Signals are smoothed to reduce jitter
Onset detection uses amplitude change threshold with decay
Microphone is not routed to speakers (analysis only)
Signals persist at 0 when analysis is stopped

Programmatic Control

adc() is the normal route. These entry points exist for extensions and for debugging from the live console.

import { startAudioAnalysis, stopAudioAnalysis, toggleAudioAnalysis } from './control/audioAnalysis.js';

// Start analysis
await startAudioAnalysis();

// Stop analysis
stopAudioAnalysis();

// Toggle
await toggleAudioAnalysis();

// Also available on window for debugging
window.oscillaAudioAnalysis.start();
window.oscillaAudioAnalysis.stop();