About Gesture Synth Weld
Gesture Synth Weld is a free browser gesture synthesizer — a web instrument that turns hand movement into music. Open the homepage, allow the webcam (or use your mouse), and move: horizontal position picks a note, vertical position opens and closes a filter, and a thumb-to-index pinch gates the sound. There is nothing to install, no account to create, and no video ever leaves your device.
Why this project exists
Most browser music toys fall into two camps: polished audio demos with weak controls, or clever camera experiments with no privacy story and no fallback when a webcam is missing. We wanted a gesture synth that takes both sides seriously — a real instrument with keys, scales, and two voices, wrapped in a page that explains itself honestly and treats the camera as yours, not ours.
We are not a music-technology institute or a venture-backed startup. We are a small project that enjoys building careful web instruments, and we put our effort into the details that matter for trust: local-only processing, a clear privacy policy, and controls that behave the same every time.
How it works technically
Hand tracking
Camera mode uses the MediaPipe Hand Landmarker, a machine-learning model that finds 21 landmarks per hand. The model runs as WebAssembly directly in your browser tab — frames are analyzed on your device and discarded immediately. Thumb-to-index distance, normalized by hand size, drives the note gate with hysteresis so notes trigger cleanly instead of flickering.
Sound engine
Audio is synthesized with the Web Audio API. Each of the two voices uses two detuned oscillators feeding an amplitude envelope and a low-pass filter. Hand or pointer X is quantized to degrees of the selected scale (major pentatonic, major, or natural minor) across two octaves; Y sweeps the filter cutoff between roughly 180 Hz and 7.5 kHz. Everything you hear is generated live — there are no samples and no audio input.
Recording
The Record button taps the synth's output through a MediaStreamDestination and pairs it with a canvas-drawn visualization (waveform, current note, watermark), capturing up to 15 seconds as a video file (.mp4 or .webm, depending on the browser). The clip is created and saved locally; the site has no upload endpoint for it.
Who it is for
- The curious — anyone who wants to feel what a gesture-controlled instrument is like in under a minute.
- Music learners — scale quantization makes it a gentle way to hear how keys and scales relate.
- Performers and tinkerers — two voices, 12 root keys, and four oscillator presets leave room to experiment.
- Teachers — nothing to install and no accounts make it usable on shared classroom machines.
Our commitments
Three promises, each matched to how the code actually behaves: the instrument is free with no feature paywall; the camera is local-only (see our privacy policy); and the page works without a webcam through mouse and touch mode. If any of that ever changes, this page and the privacy policy will change first.
Questions or ideas? Get in touch — we read everything.