Motional has made its nuReality virtual reality (VR) environments open source to help accelerate research on the interaction between autonomous vehicles (AV) and pedestrians.
Motional says nuReality is a proprietary set of VR experiences it is using in its expressive robotics research to learn how to train robots to respond in the environment similar to a person.
The company has developed nuReality to understand how expressive behaviours by AVs like flashing lights and deliberate sounds such as exaggerated braking can aid in human-machine communication with pedestrians and signal the vehicle's intentions.
Using expressive behaviours to help AVs communicate with pedestrians in crossing situations enables consumers to more quickly understand the intent of AVs and feel more confident in their decisions, the company adds.
The VR environment has an animation file of an AV that comes with a side mirror and roof-mounted Lidar sensors and no visible occupants. A second file shows a human-driven model in which a driver is looking ahead and remains motionless during the intersection.
The files contain vehicle animation scenarios that see both vehicles stopping at an intersection followed by two more where they do not stop. Another situation includes an AV using expressive behaviour such as a light bar or sounds to signal its intentions.
The virtual environments include road and building texturing, parked cars, swaying tree, birds chirping, cars driving by and people talking.
According to Motional, these details enhance place illusion and allow users to sense spatial presence within the virtual environment – giving the impression that they are standing on an actual street.
The company claims that this “VR immersion experience” was so convincing that it provoked several participants to elicit instinctively angry reactions including swearing and making gestures toward vehicles that did not stop for them.
The nuReality files can be adapted and used in a variety of applications so that others can expand on Motional's work in expressive robotics.
In 2019, Motional made the nuScenes autonomous driving dataset available to help further research that seeks to bring safe AVs to streets and communities faster.
