Creating a 360 Stereo Video in 3 (kind of) easy steps
February 14, 2017
As part of the release of the Daydream VR headset, Google brought us on to create a 360 video to live within Daydream demos in-stores across the world.
The video, a combination 360 live action footage and animated graphics,was created to show what is possible inside the realm of Daydream.
The video concept we designed included a montage of 360 live action, original content footage. All we needed was to decide how to shoot it. As it turns out, that decision took quite a bit of R&D (not that we mind!). We came away with an exciting film, and a host of best practices for creating a 360 film.
Because our video concept included a montage of 360 live action footage, we needed to plan and shoot original content. Below is what we learned during our process.
Step 1: R&D
Stereo vs Mono
We delved headfirst into our research, beginning with the stereo vs mono debate. After taking in the pros and cons of stereo vs mono - the depth of footage in stereo outweighed the cons associated with stereo shooting, and so we decided to shoot with the JUMP camera as it is the only 360 rig that shoots stereo footage.
Stereo gives much more depth to the scene.
Step 2: Shooting with the JUMP Camera
If you have never shot with the JUMP Camera before, one fundamental key to keep in mind is to understand the limitations of your equipment before the day of the shoot. Just like any shoot there are many variables that come into play, time of day, inside vs outside, number of people in scene, how close the action happens to camera, and etc. Testing camera height can help you set up the scene going into the shoot.
Beware of your surroundings
One aspect that makes a 360 shoot different then other shoots is everything is in sight. You can’t hide “behind the scenes” because there isn’t a “behind the scenes.” This, in combination with the lack of realtime feedback for the JUMP means more detailed setups and shot lists.
If you get too close the stitching starts to break
Technology can only do so much
There is no true live feedback out yet for the JUMP Camera. Unlike traditional film shoots, you can’t get a live feed out of the camera. Our best solution for this was to create a rig consisted of four Go-Pro cameras on top of the JUMP Camera. The cameras would record the scene and we would then do a quick stitch on site.
Having an SSD drive to transfer footage can save you hours because you’re usually transferring around 500 GBs.
Google created an app allowing you to upload your footage to their server. They stitch the footage for you with a turnaround time of 1 to 3 days. Google sends back high res footage plus low res proxies to optimize your post workflow.
Step 3: Post Workflow for 360 Video
Workflow: There is no good workflow yet ¯\_(ツ)_/¯
At the time we are writing this, no 360 VR workflow is perfect and most likely will require beta software, plugins and custom scripting. In our case, using the JUMP camera removed the need for stitching. We used Premiere for editing and Mettle plugin for live proxy preview in the Oculus Rift as well as light compositing. After Effects was used in a more traditional manner for masking and rotoscopy. Cinema4D and CV-VRcam along with photogrammetry captured on set allowed for stereoscopic vfx compositing over the JUMP cam footage.
Everything takes more time, plan ahead.
Producing 360 VR content is tedious and even the most mundane task can be more time consuming than anticipated. The amount of time, either for testing or actual production, needs to be identified early on and accounted for. For editing, the need to proxy all the footage adds an extra step, usually not necessary when delivering for a mobile platform. Compositing and post needs to be done on the full res footage.
To give a sense of scale, the bitrate coming out of 6K stereoscopic JUMP has 2.265 Billion pixels per second, versus 62 Million pixels per seconds for 1080 30fps footage.
Testing out the capabilities and limitations of 360 film with the JUMP rig was an exciting and lesson-filled experience for us. We hope you find our learnings useful, and if you have tips of your own share them here! firstname.lastname@example.org