Hi everyone,
Over the last few months I’ve been developing a VR camera directing workflow for iClone, originally for a personal animated project that I’m working on. Its been very useful for me so I was thinking perhaps other iClone users might enjoy it too.
When I started production, I experimented with some of the existing virtual production and live performance capture solutions available in Unreal Engine and Unity. While they worked well for certain tasks, I found they weren’t really the right environment for the type of complex cinematography I wanted to create. I prefer blocking out scenes, animating characters, and editing performances in iClone, but I wanted a more natural way to operate cameras. Though the final project is rendered in Unity, I felt the Live Performance tools occur too late in the pipeline. For example when characters need to look somehere specific in relation to the camera.
That led me to develop a VR Camera Director workflow specifically for iClone.
The idea is simple: instead of keyframing camera movements traditionally, you put on a VR headset and physically operate the camera in real time. The system streams between iClone and a Unity VR application, allowing you to record natural handheld-style camera movement directly into iClone. Im also working on an IOS version that works in a similar way.
Some of the features currently working include:
- Real-time camera control in VR
- Recording camera animation directly into iClone
- Handheld camera-style movement
- First-person and over-the-shoulder style shots
- Adjustable camera rig positions
- Quest 3 support
- Live video and audio monitoring from iClone inside VR
- Automatic take management
- Multiple camera takes stored in the scene
The goal is to make virtual cinematography feel more like operating a real camera rather than manually editing keys on a timeline.
A few examples of the types of shots I’ve been using it for:
- Handheld documentary-style camera work
- Character-follow shots
- Walking and tracking shots
- First-person point-of-view sequences
- Cinematic camera performances that feel less robotic than traditional keyframing
One particularly enjoyable aspect is that the camera can be attached to different parts of the VR rig. For example, you can create a head-mounted POV shot, hold the camera out in front of you like a real operator, or position it lower for more cinematic tracking shots. It produces camera motion that feels much closer to how a real camera operator would work. If your scene,has audio, the audio is also streaming into your headset with low latency.
One limitation is that zoom recording is currently disabled during recording because of restrictions in the current iClone Python API, although focal length can still be adjusted outside of recording.
At the moment I’m simply trying to determine whether this is something the community would find useful before deciding whether to continue developing it into a commercial product.
Would this be something you’d consider using in your workflow?
I’d love to hear:
- What types of shots you’d use it for
- Which VR headsets you’d want supported
- Features you’d consider essential
If there’s enough interest I’d be happy to share videos and demonstrations of the workflow.
Thanks!