Multi-Device Storyboards for Cinematic Narratives in VR

The first part of my PhD was looking at enabling storyboards for stereo3D environments. With the interest in VR growing, and more people trying to create stories in the VR environment, it seemed a logical next step to start exploring a similar storyboarding concept, but in the VR environment.

The first challenge was that nobody really knew what it meant to storyboard in VR. People created 2D storyboards, or individual drawings, but there was no standard on what it meant to storyboard for VR.

Since storyboard artists are highly trained and incredibly fast when it comes to drawing on 2D surfaces, it made logical sense to have the artist work on a 2D surface, and have the director view and comment on the scene in VR.

Understanding what I was trying to achieve, the question was how to orchestrate the solution.

I ended up using a similar approach to the layer approach I used with my stereo tool, where I treated the 360° environment as a set of concentric cylinders that I could unrolled to display on the 2D sketch surface.

The next challenge was that since the artist and director couldn't see each other (where they were looking, or to what they might be referring), how were they going to easily talk about aspects of the storyboard. Several approaches were used to help with this matter.

First, an overhead view was introduced, which provided the artist with reference to how they were placing drawings in the 3D volume. It also allowed the ability to show where the person in VR was looking.

In the radial map image above, the gray circle in the middle is where the person in VR is standing, and the purple wedge shows where they are looking. The large gray wedge shows where the artist is currently focused, with the blue and yellow lines indicate drawn content in the scene.

On the sketch surface, a red bar at the bottom of the drawing volume was introduced, which showed where the person in VR was currently looking.

In the VR environment, a rig was introduced that showed both where the user was looking and the depth of the current sketch cylinder. In the image below, the rig is highlighted in purple on the right.

The next question to answer was what should a storyboard look like in VR?

For the sketch surface, since they were unwrapped cylinders, displaying them in strips in a vertical list seemed the logical approach. Unfortunately, it wasn't as simple as that. First, each cylinder has a different area; secondly, in VR one has a default direction they're facing in a scene - how does one represent this factor?

For the issue of the unwrapped cylinders, I decided to warp each layer so that they all aligned to the back (largest) panel. Next I implemented a "focus" box on the strip that showed the default user direction. To help emphasise this direction, I implemented a gradient shading over the rest of the panel, with the area exactly behind the user being the darkest.

In the VR environment, a flat 2D image didn't seem like the right approach.

In the end, I settled with laying out a series of mini dioramas of the sketch cylinders around the user. Since it was possible to draw above and below the ground, the ground plane was indicated with a ring around the edge. A backdrop was also introduced to make it easier to see the content - this created a type of "container" around the sketched diorama.

Finally, the user could manipulate the diorama, bringing it closer and twisting it to view it from different angles.

This project was an interesting challenge, with many unique problems to solve. In the end, I only scratched the surface of what I had in mind, and the different possibilities to explore for creating storyboards for a VR environment...a challenge that still hasn't really been solved to this day.

Return to Portfolio