video()

Purpose: Spawn and control HTML5 video overlays during a score.

Required

• file: Filename with optional extension (.mp4 default if missing). Supports mp4|webm|ogg|mov|m4v, case-insensitive (Clip.MP4 works). src: is accepted as an alias. A filename containing spaces must be quoted: file:"My Clip.mp4" — better still, rename the file without spaces.

File Resolution

Videos are resolved in order:

1. Project video directory: scores/<project>/video/<filename>
2. Shared video directory: shared/video/<filename>

If the file is found in neither, the console logs an error naming both paths that were tried — check there first when a video cue "does nothing". Filenames are case-sensitive on the server: Lum.mp4 will not match lum.mp4.

Getting the file into the project

The DSL editor's video file field has a Browse… button: pick any video on your system and it is copied into the project's video/ folder and the field filled in — no manual file management. If a file with the same name already exists there, the existing file is referenced instead.

Placement & Positioning

• layer: front (default) | back — back paints the video behind the score's SVG elements at full opacity, with clicks passing through to the score. Works fixed, fullscreen, and scroll-anchored. The score background must be transparent over the video area — an opaque background rect in the SVG will hide a background video (in Inkscape: no page background / remove the backdrop rect there).
• location: fixed (default) | scroll
  • fixed → pinned to viewport.
  • scroll → the video is placed into the score and rides its scroll and zoom, sitting where its target was when the cue fired.
• target: <elementId> — centers the video on the target (if not fullscreen).
• offsetX: / offsetY: offsets applied after centering.
• follow:1 — with location:scroll, re-anchors the video to the target every frame, so a trans(), o2p(), rotate() or scale() running on the target carries the video with it. Without it the placement is a one-off read of the target's position. The target has to be reachable by plain id, so put the animation on a wrapping group and point target: at a child:

  <g id="carrierGroup" data-oscilla="trans(to:[wp1], dur:8, trig:playhead)">
    <rect id="carrier" x="0" y="100" width="320" height="180" fill="none"/>
  </g>
  

Size

• fit: — what happens when the video's aspect ratio doesn't match its box (most visible with size:fs on a mismatched screen):
  • contain (default) → letterbox/pillarbox bars, nothing cropped
  • cover → scales up to fill the screen, cropping the overflowing edges — the usual choice for backdrop/atmosphere video
  • fill (alias stretch) → stretches to exactly fit, distorting the aspect ratio
• w: / h: — preferred method for setting dimensions (e.g., w:960, h:540)
  • location:fixed → viewport pixels.
  • location:scroll → score units, so a video with w:320, h:180 exactly fills a 320×180 element drawn in the score, at any zoom.
• size: — alternative formats:
  • fs or fullscreen → covers viewport at (0,0) with 100vw×100vh and passes clicks through by default.
  • <W> (e.g. 640) → width in px, height 180px default.
• center:1 — centers video on viewport (useful with location:fixed)

Audio (Default Muted)

• audio: 0|1|true|false — default is muted. Set audio:1 (or true) to unmute.

Spawning / Reuse

• Retriggering a cue restarts its existing video (seeks back to in: and plays) — multiple triggers never stack copies on top of each other.
• The instance identity is the uid: when present, else file + target.
• new:1 → explicitly opt into a new concurrent instance per trigger (layered simultaneous copies — see the demo's "Concurrent" example).
• Every instance receives data-uid for tracking; data-key = file_target.

Timing & Playback

• in: seconds (seek start)
• out: seconds (optional fade-out scheduling; pairs well with fadeOut)
• hold: seconds (auto-remove after this duration, regardless of loop)
• loop: 0 = infinite; N = loop count; omit = play once
• speed: playback rate (e.g. 0.5, 1.25)
  • Slow motion looks jerky when the maths run out of frames: browsers hold frames longer rather than inventing new ones, so effective fps = source fps × speed (25 fps at speed:0.5 → 12.5 fps). For smooth slow motion use a high-fps source, or pre-render the slowdown with motion interpolation and play it at speed:1:

    ffmpeg -i clip.mp4 -vf "setpts=2.0*PTS,minterpolate=fps=30:mi_mode=mci:mc_mode=aobmc:vsbmc=1" -an clip_half.mp4
    

• opacity: 0..1 (default 1)
• fadeIn: seconds; fadeOut: seconds

Glitch grains — in:, seg:, speed: as lists or patterns

The same forms as audio(): in: a list or pattern gives several start points, seg: is the length of each grain from its start point, speed: a pattern gives each grain its own rate. All three are drawn per grain on the single video element — a seek, never a re-create — and loop: counts grains (0 = endless):

video(file:clip, size:fs, fit:cover, in:Prand(2, 9, 15, 31, inf), seg:0.4, speed:Prand(0.5, 1, 2, inf), loop:60)
video(file:clip, w:640, h:360, in:[2, 9, 15], seg:Pseq(0.4, 0.4, 0.8, inf), loop:0)

Two limits, both the browser's: no backwards grains (negative playbackRate is unsupported for <video> — negative speeds play forwards), and seek latency is set by the file's keyframe spacing — a normal encode lands each jump on the nearest keyframe and decodes forward, which stutters at grain rates above ~1–2 Hz. For fast grains make an all-intra copy, where every frame is a keyframe:

ffmpeg -i clip.mp4 -c:v libx264 -x264-params keyint=1:min-keyint=1 -crf 23 -an clip_glitch.mp4

(demo-video's "Glitch" box uses lum_glitch.mp4, a 10 s all-intra excerpt.)

Modal / UI Features

• closeBtn:1 — adds a close button (×) in the top-right corner
  • Fullscreen videos show the close button by default (it is the only touch-device dismissal when clicks pass through). It fades to a faint glyph after a few seconds but stays tappable. Disable with closeBtn:0 for performances.
• backdrop:1 — adds a semi-transparent dark overlay behind the video
• controls:1 — shows native video controls (play/pause, scrubber, etc.)

Triggering

• By default a video cue fires when the playhead crosses its element, and is also click-triggerable.
• trig:click — click-only: the playhead passes over the element without firing it.
• trig:manual — never auto-fires; trigger via the live console or button().
• click:0 — the score element ignores clicks entirely (playhead-only).
• revtrig: — also fire when the playhead crosses in reverse (1 = trailing edge, 2 = leading edge), tdelay: — delay (s) after trigger.

All of these — and every other parameter on this page — are editable as form fields in the DSL editor (click the element with the inspector active); new video cues start prefilled with trig:playhead, w:480, h:270, fadeIn:0.5.

Interaction

• Fullscreen (size:fs): uses pointer-events:none so the score behind remains draggable/clickable.
  • Add clickable:1 to allow clicks on the fullscreen video (e.g., to close on click).
• Non-fullscreen videos remain clickable; single-click removes them by default.
• Clicking the backdrop (if enabled) also closes the video.

Removal

• Ends when:
  • playback completes (no loop) or
  • loop count is reached or
  • hold expires (fading first if fadeOut: is set) or
  • user clicks (non-fs, or fs with clickable:1) / clicks the backdrop or close button or
  • user presses Escape — dismisses the most recent cue video (keyboard devices) or
  • user taps the corner × (shown by default on fullscreen videos — the touch-device equivalent of Escape).

Video on the control plane

A video reports where it has got to, and can be driven while it plays:

video(uid:film, file:lum, w:640, h:360, speed: follow(fader.t, 0.5, 2))
rotate(uid:r1, dir:1, dur: follow(film.t, 10, 1))

speed is clamped to what browsers actually accept (0.0625–16) and made positive — a <video> cannot play backwards. A bound opacity owns the value: fadeIn/fadeOut are a timed ramp on the same property, so the ramp is switched off rather than left to fight the signal.

Position is reported per decoded frame where the browser supports it, falling back to about four times a second — and the slower path keeps reporting in a background tab, where the per-frame one stops.

Until now a follow() written on a video() was silently ignored: video builds its parameters differently from every other cue, and the conversion that turns follow(...) into a binding never ran on them.

Demo Project

The bundled demo-video project walks through every feature on one score: windowed, modal, and fullscreen spawning, size variants, audio, speed/opacity, looping, trim + timed removal, concurrent instances (new:1), target anchoring with location:scroll, follow:1 on a trans()-driven frame, and instance reuse (hot-updating a running video by retriggering the same file + target).

Examples

1. Fullscreen, pass-through clicks, but allow click-to-close

   cue:video(file:intro.mp4,size:fs,clickable:1,audio:1,fadeIn:0.5)
   

1b. Fullscreen backdrop that fills the screen edge-to-edge (no bars)

video(file:atmosphere.mp4, size:fs, fit:cover, loop:0, fadeIn:2, closeBtn:0)

2. Windowed, anchored to a target, follows scroll, infinite loop

   cue:video(file:clip.webm,target:markerA,location:scroll,size:640x360,loop:0,opacity:0.9)
   

2b. Windowed, riding a target that trans() is moving

cue:video(file:clip.webm,target:carrier,location:scroll,w:320,h:180,follow:1,hold:12)

carrier is the inner rect; the animated group around it carries trans(…) in data-oscilla. The group's translate moves the frame and follow:1 keeps the video on it. The video is sized in score units (w:/h: under location:scroll), so it fills the frame exactly.

3. Spawn a second independent instance via uid

   cue:video(file:cam.mp4,uid:stageLeft,in:5,fadeIn:1,fadeOut:1,hold:20)
   

4. Force new instance without specifying uid

   cue:video(file:teaser.mp4,new:1,in:2,out:10,fadeIn:0.5,fadeOut:0.5,speed:1.25)
   

5. Modal video with close button and backdrop

   cue:video(file:tutorial.mp4,w:960,h:540,location:fixed,closeBtn:1,backdrop:1,controls:1,fadeIn:0.3)
   

6. Clickable SVG element that triggers a modal video

   Use button() with mode:overlay to make an SVG element trigger a video while keeping the SVG visible:

   <g id="button(trigger:video(file:tutorial.mp4,w:960,h:540,location:fixed,closeBtn:1,backdrop:1),mode:overlay,tooltip:Tutorial)">
     <rect x="0" y="0" width="50" height="48" rx="4" style="fill:#4a90d9"/>
     <text x="25" y="35" style="text-anchor:middle;fill:#fff">?</text>
   </g>
   

Note: size:fs always positions at (0,0) and ignores target geometry; all other sizes center on target (or the cue position) with optional offsetX/offsetY. Default audio is muted; use audio:1 to unmute. By default, same file+target cues reuse the existing element unless uid or new:1 is supplied.

Tip: use ← → or ↑ ↓ to navigate the docs