What Is VR Eye Tracking and How Does It Work?

To create compelling virtual reality (VR) experiences, sensors must track and collect information from the sense organs, like the eyes and ears, in addition to tracking the head. While VR tracking typically focuses on head tracking, developments in eye tracking are at the forefront of VR research.

Eye tracking in itself is not a new technology, but the development of small cameras led to the rapid growth of eye-tracking techniques using video. These cameras are small enough to fit into a VR headset, monitoring eye movements during a user’s experience of a VR world. VR eye tracking collects specific data on where the subject is relative to the 3D of the VR world. 

For anyone interested in getting started in VR, it’s helpful to understand the types of eye movement that are important to track. Let’s explore how VR eye tracking works, its implementations, and its challenges to get a broader perspective. 

Types of eye movements to track

The eyes move in six ways that are important for VR eye tracking. Let’s take a look at each kind of movement:

• Saccades: Quick eye movements of the fovea (which creates great visual detail) that help the eye fixate on the details as it transitions from one object to another. These movements happen unconsciously and sometimes consciously when you focus on something particular. 

• Smooth pursuit: Slow rotation of the eye to track moving objects to reduce the perceived motion blur of the objects. This movement helps the eye with image stabilization.

• Vestibulo-ocular reflex: Involuntary reflex motion of the eyes to keep focus on objects when the head is angling back and forth. The eyes counteract the angle of the head to maintain focus. This motion is important for VR to track as it is also vital for image stabilization.

• Optokinetic reflex: Occurs when a fast object moves by, allowing the eyes to focus on particular features of the object. From this reflex, your eyes move back and forth on details of a moving train, moving between saccade and smooth pursuit movements.

• Vergence: Moves the eyes to create a single image of one object they both focus on. This occurs as convergence or divergence and happens when an object is closer and farther away. The movements help give a sense of distance an object is away. 

• Microsaccades: Complex, involuntary eye movements of less than one degree that help the eye fixate on particular objects, improve acuity, and help with perception issues. Microsaccades are the least known eye movement and are under research.

What is VR eye tracking used for?

For VR interaction, eye tracking has applications in VR display and rendering, user interaction, and collaborative environments. Immersive VR has a definition close to that of human vision but presents an implementation challenge due to the high data rates needed to create it. Eye tracking provides a solution to this rendering since the eye’s region of visual acuity through the fovea is only 5.2 degrees. 

VR eye tracking makes implementing this life-like definition possible by focusing the highest-definition regions on that of the fovea. It accomplishes this by tracking where the subject looks, a process called foveated rendering. It also saves on computing power by rendering everything of the same quality. 

VR eye tracking addresses vergence-accommodation conflict (VAC) issues, which leads to discomfort, headaches, and eye strain. Recall that vergence moves the eyes to create a unique image on which they both focus. However, new research into varifocal displays uses VR eye tracking to manage vergence issues by tracking the eye’s vergence. 

VR eye tracking has potential implementations in how users interact in virtual environments, which can help if users have motor disabilities or if a virtual environment wants to free up controller usage.  VR eye tracking helps with selection techniques using the eyes, creating hands-free movement within virtual environments, and system interaction with just eye movements. 

Lastly, VR eye tracking creates realistic user interaction by creating realistic renderings of the human face, which creates accurate construction of user avatars. Humans give communication cues through their eyes, and VR eye tracking shows where you look and point with just your gaze. 

How does VR eye tracking work?

VR eye tracking works by understanding the various eye movements and sending data to VR applications. VR eye tracking uses three methods:

• Electro-oculography (EOG)

• Scleral search coils

• Video oculography (VOG)

Let’s take a closer look at how each VR eye tracking method works. 

Electro-oculography (EOG)

EOG uses electrodes placed into contact points where the VR headset meets the face. Its function measures the eye's orientation by measuring the voltage across the positively charged cornea to the negatively charged retina. Measuring positive and negative charges allows the VR software to generate a map of the eye’s orientation. It has imprecision but is the only method to measure the eyes while the user has them closed. 

Scleral search coils

The scleral search coils method offers a high degree of accuracy. It uses a contact with a wire loop embedded in it. Electrical currents move through the coils as the eyes move, reading the eyes' vertical, horizontal, and rotational movements. While highly effective, it is difficult to use in conjunction with a VR headset but has value as a measurement against other eye-tracking methods.

Video oculography (VOG)

The most common tracking method is VOG, which uses images captured by cameras in the VR headset. Once captured, it transmits an analysis of each image to the VR headset by locating the position of the pupil and other eye features. The quality of the captured images and the way the cameras mount to the headset determine the use of this method.  

Who uses VR eye tracking?

VR developers, the marketing, health care, and education industries benefit from VR eye tracking. An obvious use of VR eye tracking is for developers to improve VR systems' performance, latency, resolution, comfort, and user-to-user interactions. Three other industries that use or could use VR eye tracking as research develops include marketing, health care, and education.

Marketing

VR eye tracking has applications in consumer experience research (CX) by examining where the viewer's eyes go during the pre-purchase, purchase, and post-purchase stages. VR eye tracking helps examine how advertisements work on the consumer, with VR and AR as advertising mediums having better engagement than traditional media sources. In the pre-purchase and purchase stages, VR eye tracking gives insight into how consumers interact with products and where their eyes gravitate based on product branding. 

Health care

VR eye tracking has diagnostic, therapeutic, and interactive applications in health care settings. In detecting certain neurological diseases, VR eye tracking can accomplish diagnostic tasks for tracking objects and looking for abnormal eye movements. In therapeutic settings, eye tracking and VR help with exposure therapy to see how a subject’s gaze responds to triggering scenes. 

Education

Examination of a learner’s experience with new material is possible with VR eye tracking by a researcher. Eye tracking in education environments focuses on temporal and spatial movements to examine how long a learner looks at a particular area and where the learner looks. While VR eye tracking is a valuable tool for perceiving learner engagement, it is a new technology that requires more research for accurate predictions. 

Challenges of VR eye tracking

Implementing VR eye tracking has its challenges. Three challenges of VR eye tracking include the following:

• Technological challenges

• Data and security challenges

• Safety challenges

Let’s examine these challenges within those three main categories more closely. 

Technological challenges

VR eye tracking has technological limitations, including the quality of data eye tracking systems collect. Four of these issues are in spatial precision, spatial accuracy, high latency, and low sampling rates, all of which lead to issues in the VR experience for the user. Additionally, issues of how long calibration takes might dissuade users from doing so, leading to a decay in tracking quality. 

Data and security challenges

VR eye tracking collects a range of data that leads to security challenges. Eye tracking data contains a range of personal information from cognitive states, information on mental illness or disorders, personality traits, and other sensitive data like age, ethnicity, drug usage, emotions, and fears.  Additionally, iris patterns are one of the best ways to biometrically identify you, which leads to data concerns of identity theft for the infrared (IR) images produced by eye tracking systems. 

With the essential personal information found within VR eye tracking, methods to degrade, sensor, and delete this information require further research to ensure the privacy and safety of VR users. 

Safety challenges

VOG VR eye trackers use IR light and near IR light to produce the contrast necessary for eye tracking. Lengthy exposure to this light can damage the cornea, iris, and pupil. As the implementation of eye tracking in VR increases, further research into prolonged exposure is needed. 

Getting started with Coursera

VR eye tracking is at the forefront of new developments in VR, aiding rendering and creating more effective VR experiences. To learn more about how VR works and how to create your own experiences, try the Virtual Reality Specialization from the University of London on Coursera.