Go to the NASA Homepage
 
Advanced Controls and Displays Left-Side Header Image
Research Sidebar Header
Advanced Controls and Displays Image Collage
Research header
Improved Temporal Response in Virtual Environments through System Hardware and Software Reorganization
Abstract
End-to-end latency, the time delay from input user action until the consequences of that activity are first available for display, continues to be a major limitation in virtual environment (VE) system performance. This overall latency stems from the time lags present in the individual components of the VE system. These include the time for transduction and internal processing by input sensors (e.g., displacement trackers, gloves, etc.), data transfer from the sensors to the host computer, update of the simulation output by the application(s) in response to user input, and, finally, image rendering and display.

Our efforts have been directed toward reducing the latency of our VE system and increasing its overall update rate. Our computer hardware includes a four-CPU (R4400) SGI Onyx with Reality Engine II graphics running under the IRIX 6.2 operating system. We use Sense8's WorldToolKit 8.0 for developing and running VE applications and Polhemus's Fastrak sensors as displacement input devices. For portions of this work, we have also installed an ASO interface in our Onyx. Because similar hardware and software already find widespread use, the techniques and results described here will be broadly applicable and transferable to other VE systems.

In this work, we considered a series of hardware and software reconfigurations. Our baseline was the typical implementation in which a single Fastrak instrument with multiple sensors, connected to the computer via an RS-232 serial interface, is polled at each update of the application program. In subsequent configurations, the portions of the code responsible for communicating with the sensors and handling Fastrak data were extracted and implemented as separate UNIX processes which transfer data to the main VE application via shared memory. Additionally, by operating the sensors in continuous streaming data mode rather than the original polled method, the new Fastrak driver processes run asynchronously with respect to the VE application. Other enhancements that we have investigated include the use of multiple Fastrak instruments (each with only one sensor) and high bandwidth ASO serial communication between the Fastrak and the computer.

End-to-end latency and update rate for a simple (1000 polygon) test VE were determined for each configuration. Update rates were measured by time stamping each cycle of the application program. Latency was measured by comparing the time of the Fastrak sensor's mechanical interruption of a first photodetector against a second photodetector's timing of a resultant VE event that was registered on a CRT.

Beginning from an average latency of approximately 65 msec and an update rate of 20 Hz, our most successful implementation to date has an update rate of 60 Hz (the maximum achievable with our graphics display hardware) and an approximate displacement latency of 30 msec.

One reason why even a small amount of latency are not desirable in a virtual environment is that it has an adverse effect on the performance of a person engaging in a time-critical activity (realtime path-tracing, for instance). Here are two quicktime videos of a subject attempting to hold a virtual object inside a virtual frame. The first was recorded with the baseline latency of the system; the second was with six frames of extra latency added to the system.
Click to view Quicktime movie of Zero-Frame Latency run
Zero-Frame Latency run
(Quicktime - 1.3MB)
Click to view Quicktime movie of Six-Frame Latency run
Six-Frame Latency run
(Quicktime - 1.2MB)
Go to the First Gov Homepage
Go to the NASA - National Aeronautics and Space Administration Homepage
Curator: Phil So
NASA Official: Brent Beutter
Last Updated: August 15, 2019