About VR + Non-player characters

The article from UploadVR discusses a groundbreaking demo where the developer of Modbox integrated Windows speech recognition, OpenAI’s GPT-3 AI, and Replica’s natural speech synthesis. This combination resulted in what can be described as one of the first artificially intelligent virtual characters in the realm of VR. Modbox, a multiplayer game creation sandbox with SteamVR support, showcased this innovation, hinting at the potential future of non-player characters (NPCs) in gaming.

Features of VR + Non-player characters

  1. Integration of Multiple Technologies: The demo seamlessly combines Windows speech recognition, OpenAI’s GPT-3 AI, and Replica’s natural speech synthesis. This trio offers a unique experience, allowing users to interact with virtual characters in a more realistic manner.
  2. Conversations with Virtual Characters: Starting at around 4 minutes and 25 seconds into the demo video, viewers can witness conversations with two virtual characters. This showcases the potential for real-time interaction with NPCs that can understand and respond to user queries.
  3. Cloud-based Services: Both GPT-3 and Replica operate as cloud-based services. This setup currently results in a delay between the user’s question and the NPC’s response. However, the future might see on-device models that can eliminate this delay, similar to some smart home devices by Google and Amazon.
  4. Direct Interaction with NPCs: Unlike traditional video games and VR experiences where interactions with characters are limited to pre-written dialog trees, this technology allows users to directly speak to virtual characters. The responses are convincing and not restricted to predefined scripts.
  5. Use of Transformer Model: The breakthrough in machine learning that made this possible is the Transformer model, which focuses on the concept of attention. OpenAI’s GPT series, especially GPT-3, utilizes this approach, enabling more human-like interactions.

Additional Features

  1. Evolution of Language Models: The article traces the evolution of language models, from Google’s introduction of the Transformer model to OpenAI’s development of GPT-2 and GPT-3. GPT-3, with its 175 billion parameters, can produce results almost indistinguishable from human writing.
  2. Potential Limitations: While GPT-3 offers impressive capabilities, it doesn’t truly “understand” in the conventional sense. It can sometimes produce nonsensical or inappropriate results. Addressing these limitations, such as implementing a “common sense check” mechanism, will be crucial for its broader consumer application.