Read the manual.

Installation & setup

System requirements

• macOS 10.13 or later
• DAW supporting VST3 or AU format (Ableton Live 11+, Logic Pro, FL Studio 20+, etc.)
• Webcam with 720p or higher resolution
• USB 2.0 or better bandwidth
• Adequate lighting for hand tracking

For VST3

• Download the Nimbus VST3 bundle
• Navigate to ~/Library/Audio/Plug-Ins/VST3/
• Copy CV MIDI Controller.vst3 to this folder
• Restart your DAW

For AU

• Download the Nimbus AU bundle
• Navigate to ~/Library/Audio/Plug-Ins/Components/
• Copy CV MIDI Controller.component to this folder
• Restart your DAW

First launch

Open your DAW, load Nimbus as an instrument or MIDI effect, and grant camera permissions when prompted. The hand visualizer should display with your webcam feed active.

Getting started

Workspace layout

The Nimbus interface is divided into three main sections:

Left panel — Source pads

• 4 Sound Pads (2×2 grid)
• 4 Sliders (2×2 grid)
• 4 Knobs (2×2 grid)
• Keyboard
• Arp Stream
• Strings
• Dead Zone

Center — Hand visualizer

• Live hand tracking display
• 0–400px coordinate space
• Gesture visualization
• Interactive instrument placement

Right panel — Settings

• Hand color selection
• Tracking smoothing slider
• CV filters (brightness, contrast)
• Camera selection
• Scene management

Basic workflow

• Drag an instrument from the left panel onto the hand visualizer
• Release to place the instrument at that location
• Right-click on the instrument to edit its properties
• Perform gestures with your hands to trigger the instrument
• Save your scene using the Scene menu

The hand visualizer

What you're seeing

• Two hands rendered in real-time as skeletal structures (up to 2 hands tracked simultaneously)
• Finger bones connecting 21 landmark points on each hand
• Hand joints shown as colored circles (larger at wrist, smaller at fingertips)
• Grid background for reference positioning
• Placed instruments as colored circles or lines

Coordinate system

• X-axis (horizontal): 0 (left) → 1.0 (right)
• Y-axis (vertical): 0 (top) → 1.0 (bottom)
• Z-axis (depth): tracked but not visually displayed; used for collision detection

Interaction tips

• Keep hands 18–24 inches from the camera for best tracking
• Ensure good lighting on your hands
• The hand visualizer updates at 30Hz
• Landmarks are smoothed with adjustable EMA (exponential moving average)

Gesture mappings

Hand openness

Fist (fully closed) outputs CC value 0. Open palm (fully extended) outputs CC value 127. Assigned to the "Openness" gesture slot. Use it for filter cutoff, reverb wet/dry, or volume.

Hand height (Y position)

Hand at top of frame outputs CC 127. Hand at bottom outputs CC 0. Inverted so raised hand equals high value. Assigned to the "Height" gesture slot.

Hand X position (horizontal)

Left hand: leftmost = 0, rightmost = 127. Right hand: mirrored. Useful for pan, filter frequency sweep, or effect parameters.

Hand distance (two hands)

Hands together outputs CC 0. Hands apart (opposite edges) outputs CC 127. Requires both hands active and not in the dead zone. Great for stereo width, delay feedback, or reverb decay.

Pinch detection

Thumb to index finger distance < 45% of hand size activates the pinch. Also works with thumb to middle finger. While pinching, all CC values for that hand freeze (pinch lock) — useful for holding a parameter value.

Snap detection

Triggered when thumb flicks up while middle finger flicks down simultaneously. Hand must be mostly closed; opposing vertical velocities > 14px/frame. 500ms cooldown between snaps. Visual feedback: orange ripple effect at trigger point.

Custom gestures

Right-click "Record" in settings to capture a pose. Choose between trigger mode (toggle on/off) or scale mode (continuous 0–127). Detection uses RMS distance from the template pose with an adjustable threshold. Assign any custom CC number and range.

Instruments

Sound pads

Trigger audio samples and MIDI notes by touching virtual pads with your index fingertip. 4 pads arranged in a 2×2 grid. Drag from the left panel to place.

• Position: X/Y coordinates (0–1.0 normalized)
• Size: pad radius in pixels (default 40px)
• MIDI Note: 0–127 (default 60 = Middle C)
• MIDI Channel: 0–15 (default 0)
• Velocity: 0–127 (default 127)
• Polyphonic: up to 4 pads simultaneously

Sliders

Output continuous CC values based on hand position along a linear track. Pinch with thumb + index, then move fist up/down to modulate. CC output updates in real-time (0–127).

Knobs

Output continuous CC values based on rotational hand movement around the knob center. Place hand over knob, rotate wrist to modulate. 360° rotation maps to full 0–127 CC sweep.

Piano keyboard

Curl individual fingers to play notes on a virtual keyboard overlay. Each finger maps to a key. Place on the canvas and interact with fingertips.

Arp laser streams

BPM-synced laser beams that fire MIDI notes when your hand passes through armed pulses. 16-step sequencer with per-step note assignment via a built-in piano picker. Set time division (1/4, 1/8, 1/16, 1/32).

Guitar strings

6 virtual strings with chord switching. One hand pinches like holding a pick and strums up/down across strings. The other hand signs numbers 1–5 to switch between 5 assignable chords from a 12-chord library (C, Cm, D, Dm, G, Em, E, Am, A, F, Bm, B). Chords persist until a new number is shown — fist is neutral.

Dead zone

Drag a resizable exclusion zone onto the canvas where hand tracking is completely ignored. No MIDI, no triggers, nothing. Useful for resting hands or isolating performance areas.

Advanced features

Collision detection

Z-axis (depth) proximity prevents accidental triggers. Hand must be within 100px depth of pad center. Useful for avoiding false positives in live performance.

Motion trails

Visual history of hand movement (5 frames). Helps visualize gesture speed and direction. Toggle in Settings → Trails Enabled.

Hand color selection

Customize hand skeleton color for visual contrast. Available colors: Blue, Red, Green, Cyan, Magenta, Yellow. Helps distinguish left/right hand in dual-hand performances.

Settings & customization

Tracking smoothing

Controls the EMA (Exponential Moving Average) filter applied to hand landmarks.

• Low (0.2): responsive but jittery
• Medium (0.5): balanced (recommended)
• High (0.9): smooth but latent

CV filters

• Brightness boost: increases video brightness (useful in dim lighting)
• Contrast boost: increases color separation (helps hand detection in low contrast)
• Trails enabled: toggle motion trails visualization on/off

Camera selection

Dropdown menu lists all available cameras: built-in webcam (default), external USB camera, and virtual camera apps (OBS, etc.).

Scene management

A scene is a complete configuration of all placed instruments, their positions, properties, and assignments.

Save a scene

• Place and configure all instruments
• Menu → Save Scene
• Enter scene name (e.g., "Acid Solo", "Ambient Pad", "Drum Break")
• Configurations are persisted on disk

Load a scene

• Menu → Load Scene
• Select from list of saved scenes
• All instruments restore to their saved positions and properties

The default scene loads on plugin startup. Create separate scenes for different songs or sets and name them descriptively. Backup scenes by exporting plugin state.

Tips & tricks

Performance

• Keep hands 18–24 inches from camera for optimal tracking
• Bright, even lighting on hands improves detection
• Plain, contrasting background (not same color as hands)
• Let detector warm up for 2–3 seconds before performing

Gestures

• Bring thumb and index tips close (almost touching) for reliable pinch
• Fully close all fingers for clean fist detection
• Spread all fingers wide for maximum openness value
• Practice the snap flick motion slowly at first — speed comes with muscle memory

Workflow

• Use a DAW MIDI monitor to verify CC messages are sending
• Practice your performance without audio first
• Use DAW's MIDI learn to auto-assign CCs to parameters
• Regularly export your scenes and custom gesture data

Troubleshooting

Hand not tracking

• Check camera permissions (System Preferences → Security & Privacy → Camera)
• Restart plugin/DAW
• Try different camera (Settings → Camera dropdown)
• Ensure adequate lighting
• Move closer to camera (12–24 inches optimal)

Tracking is jittery

• Increase Tracking Smoothing slider
• Improve lighting (reduce shadows on hands)
• Clean camera lens
• Move camera away from light sources

Gestures not triggering

• Check hand is not in dead zone
• Verify instrument is placed (white dot indicator shows)
• For sliders: ensure you're using pinch + fist, not open hand
• For pads: use index fingertip to touch
• Check DAW MIDI monitor to see if CC messages are being sent

MIDI not reaching DAW

• Verify MIDI channel matches your DAW track
• Check DAW input filter (some DAWs ignore default channels)
• Use MIDI learn to assign CC to DAW parameters
• Verify plugin is on a MIDI track, not audio track

Camera crashes or freezes

• Restart DAW
• Unplug and re-plug USB camera (if external)
• Close other apps using camera
• Update macOS and DAW to latest versions

Snap detection not working

• Practice the flick motion — thumb and middle must move opposite directions
• Increase Tracking Smoothing slightly
• Ensure hand is mostly closed (openness < 35%)
• Verify 500ms has passed since last snap (cooldown)

MIDI CC reference

Keyboard shortcuts

Specifications

• Tracking rate: 30Hz
• Hand landmarks: 21 per hand × 2 hands
• Coordinate space: 0–400px (normalized to 0–1.0)
• MIDI latency: < 1 frame (~33ms)
• Supported MIDI channels: 0–15
• Max simultaneous CC updates: 12 per frame
• Max placed instruments: unlimited (performance dependent)
• Audio formats: WAV, AIFF, MP3, OGG, FLAC