Screen capture in AR and VR experiences

Augmented and Virtual reality amazing experiences from a first person point of view. However, watching someone play AR and VR experience from outside is all the more interesting. While these provide a unique experience to it’s viewers, they also give an ability to create content for different perspectives.

Virtual Reality was extensively used to take the best shots for Lion King movie. It allowed director to view test the movie from different angles before finalising the best shot.

The main topic of the article however is for layman users to utilise the function in their every day device like phone or VR headset.

The current screen capture mechanisms

Both AR and VR apps have capabilities to capture photo and video from a first person or user point of view. Below is an example of Emirates VR experience.

And similarly you can view and capture the AR experience through device.

However, these captures are from the user point of view which limits the nature of AR and VR. AR and VR experiences are spatial in nature. Meaning experience spans in 360 degrees around the agency.

Oculus Quest allows to record the experience as screen capture.

Need for captures from different position and angles

Having a capture mechanism from angles and position other than the agency will work wonders. Most of the images that we see online depict the experiences from different spot than the user and shows users wearing the headset or holding the phone. This allows to give a good context of the experience in form of user posture and action and evoke different emotions.

Having a mechanism to set up camera as a default OS option in the apps for AR and VR screen captures from different positions and angles will allow content creators to create variations of the scene. This setup is similar to different camera angles used in movies to evoke emotions for the viewers.

A plan for camera setup and it’s capture in live shoot

What’s needed?

The capture system should allow it’s user to set up

• More than one capture camera
• The capture camera can take either photo or video
• The capture camera can capture either 2d or 360
• The capture camera can be set in different positions in the space
• The capture camera can be set in different angles
• The output of capture camera can either be stitched sequentially or put in a grid layout
• Ability to superimpose a human character avatar based on controller positions. The avatar and controllers can be turned on or off
• Annotate the capture before saving
• Crop and trim the capture before saving

Same scene different emotions

Cinematography in Hurt Locker