VidDancer VST: Audio Workstation Video Sampler / Morpher / Editor
Get creative with video and MIDI in your DAW...
Dance it slow, dance it quick, dance it forward, dance it back.
Compose music to your morphed video and save as a complete audio/visual production.
Features
- The VidDancer VST effect plugin plays video forwards or backwards at variable speeds, and jumps between chosen cue offsets; all under MIDI control.
- The video soundtrack is similarly morphed in sync with the visuals, if required.
- As with conventional effect plugins VidDancer accepts audio input, which it sums with the video soundtrack.
- On project completion, the audio and visuals are captured through the plug-in and exported to a final video file.
- VidDancer is controlled by MIDI notes configured for variable positive or negative playback speeds and for selected cue offset jumps.
- Mapping of MIDI notes is user configurable like a VST sample player or drum machine.
- VidDancer is available as VST3 or AU2 plug-in and is compatible with Windows, macOS and Linux DAWs.
Usage Notes
System Requirements - Microsoft Windows
Windows version 7 or later. DAW with VST3 support. Sample rate must be 44100 or 48000. Audio buffer/block size should be in the range 512 to 1024 samples.
System Requirements - Apple OSX/macOS
OSX/macOS 10.13 or later (Intel and ARM CPUs are both supported). DAW with VST3 or AU2 support. Sample rate must be 44100 or 48000. CoreAudio buffer/block size should be in the range 512 to 1024 samples.
VidDancerVST is configured as an 'Effect' plug-in
VidDancerVST is unusual because it is an 'Effect' plug-in that requires MIDI input. It should be configured so that your final project audio passes through it, so the audio can be captured into the final finished video.
The setup for routing MIDI to an Effect plug-in will vary between DAWs. There are template projects for 10 different DAWs included in the product distribution .zip file. There are also written configuration instructions for those various DAWs.
Extra configuration texts -
Pro Tools: Avid_Pro_Tools.txt
Tracktion Waveform: Tracktion_Waveform.txt
MUTools MULab: MUTools_MULab.txt
SoundBridge: SoundBridge.txt
Renoise: Renoise.txt
MIDI event logging
VidDancer logs MIDI events as the host playhead passes over them. When you are working in your DAW, and moving the playhead around, the plug-in calculates the correct position in the video by scanning those logged events. This means that you should not need to constantly re-play the whole project from time zero to sync the video.
Tempo changes in Ableton Live:
VidDancerVST will not work correctly in Ableton Live when there are tempo changes in the project. A fixed tempo must be used due to a limitation of the plug-in interface provided by this DAW.
Synchronisation in Bitwig Studio:
Bitwig Studio has a minor issue when used for composing to video. When the transport is stopped this host does not report changed playhead position to plug-ins. It is a limitation of the plug-in interface provided by this particular DAW. A workaround using a Bitwig Extension is discussed here: Bitwig Page
Synchronisation in Cakewalk Sonar and MOTU Digital Performer:
In these DAWs it is recommended to use the 'Sync to PPQ position' Settings option. With this option the plug-in will synchronise when the timeline is moved by clicking or dragging within the project.
Avid Pro Tools:
Avid do not openly license access to their AAX software development kit (SDK), so it is not currently possible to develop an AAX version of VidDancer. A workaround is to host VidDancer inside an instance of
DDMF Metaplugin effect.
How to configure:
Avid_Pro_Tools.txt
Cubase (Windows 10) graphics sub-system:
Cubase may display the following message in Studio Setup: Video Player "Please update your graphics card or switch to a more effective graphics card". This indicates a problem in the graphics sub-system and Cubase may block the VidDancer plug-in from obtaining a valid graphics context.
Acoustica Mixcraft, Zynewave Podium and LMMS:
In these hosts the VidDancerVST option 'Sync to PPQ position' MUST BE SET. Without this setting VidDancer cannot synchronise to the transport timeline. LMMS also requires 'Sync VST plugins to host playback'.
Step By Step
- Configure VidDancer following instructions in the distribution download and notes above: "VidDancerVST is configured as an 'Effect' plug-in".
- Click on 'Load' to select your chosen video. VidDancer creates two work files from that video and notifies you of their full paths. Button 'Properties' is available to view statistics about your chosen video and also reports the work file paths.
- Select 'MIDI Mode: Free Note Input' (button lights green). Use your MIDI controller keyboard to play the video forwards and backwards at various speeds. Edit the key assignment values to set playback speeds and 'Jump To' positions in the video. Make notes of the timestamp values at important cue positions in the video. If you don't like the default keyboard assignments different keys can be set in 'Settings'. 'Play' and 'Stop' keys can be used to nudge playback to locate cue frame timestamps.
- Having familiarised yourself with the video, you may like to rehearse some key sequences like you would if rehearsing a melody on a piano.
- Select 'MIDI Mode: Host Sequence'. Edit, or record in, MIDI sequences to play the video as required for your project.
- Compose your project audio to sync with the video playback.
- Having composed your video and audio it is now time to 'Capture' (record) the whole performance. Click on 'Capture' and you are prompted to place the host playhead at project start and then set the transport running. VidDancer records the audio and video frame sequence into memory until the transport is stopped. It is possible to capture just a small part of the project by starting the transport at an offset position and stopping before the project end. VidDancer reports the current captured length, which is now available for 'Export'.
- Click on 'Export' and VidDancer prompts you for an export file name. The plug-in extracts the required video frames from your original video file (not from the work files) and merges them with the recorded audio to make your final audio/visual production. The video is encoded to the same frame size as the original. There are options in 'Settings' to do fast or more compressed (lower quality) encoding if required.
Hints and Tips
Pausing playback:
'Jump To' events stop playback at the assigned offset. This is so you can present a fixed image for a while, if required. Place a 'Play' event just after the 'Jump To' to immediately resume playback from the new offset.
Multiple clips in several different files:
There is always the option to start a project by first editing all the clips into one video (using one of the many available video editors). Just drop them in as they are, in no particular order. Load that video into VidDancer and assign 'Jump To' keys for each of the clip offsets. You can then experiment with playing the clips however you like, slowed down, speeded up, forwards, reversed, repeated, shortened.
Using DAW markers:
All the various DAWs have the option to set markers at key positions in the project. It is a good idea to place 'Jump To' events at each of these locations (followed by 'Play' events as required). Then, each time you audition a section of the project, start playback just to the left of the marker. This will ensure you are always viewing your video correctly synced to the project.
Using VidDancer as a simple film scoring playback device:
Insert VidDancer on an audio track and set a MIDI track to control it from. In the plug-in set 'MIDI Mode: Host Sequence' and load your video file. In the MIDI track place, at the project start offset, an E4 ('Jump To' 0.00) followed immediately by a C3 ('Play' +1.00). Start the transport and VidDancer logs the 2 midi events as the playhead passes over. You can now move the playhead around in your project and VidDancer will sync the video to follow it.