MOKUKU is more than just a VR/AR system—it’s a complete interactive platform inside vehicles. To bring our AI-driven, passenger-focused experiences to life, we designed a modular, robust, and high-performance hardware stack. In this post, we’ll dive into the key components and architecture that make MOKUKU possible.

1. Core Processing: ESP32 Controllers

At the heart of MOKUKU are ESP32-based microcontrollers, chosen for their:

  • Low power consumption
  • Built-in Wi-Fi and Bluetooth
  • Real-time control for displays, sensors, and audio

ESP32 handles:

  • Reading CAN bus and OBD-II vehicle data
  • Driving animations and interactive displays
  • Audio input/output control
  • Communication with AI servers for passenger interaction

We also explored ESP32-C3 variants to handle specialized tasks, ensuring smooth integration with sensors and VR tracking systems.

2. Visual Display System: Dual Circular Screens

MOKUKU’s “face” is two independent circular displays:

  • Left & Right displays: Can show dynamic expressions, vehicle info, or interactive content.
  • Animation playback: Supports sequence-frame animations from SD cards for rich visuals.
  • Independent or combined mode: Can display separate info or form a cohesive animated face.

This display system allows us to blend vehicle telemetry with playful interactions, creating a unique passenger experience.

3. Audio System: Talking and Listening

Sound is key to immersion. MOKUKU uses:

  • ICS-43434 Digital Microphone for high-quality voice input.
  • ES8311 Audio Codec for stereo sound output (speakers or headphones).
  • WebSocket streaming: Sends audio to AI servers in real-time for responsive conversations.

This enables voice commands, AI conversation, and auditory feedback to passengers while driving.

4. Vehicle Data Integration

To make experiences responsive to real-world movement, MOKUKU taps into the vehicle:

  • CAN bus / OBD-II sensors: Speed, RPM, acceleration, braking, and steering inputs.
  • IMU / Gyroscope data: Enhances VR/AR 6DOF tracking.

This lets our system react to driving conditions, enabling virtual pets to move, animations to adjust, and VR environments to match car motion.

5. Media Management & Expansion

MOKUKU is designed for modular content management:

  • SD Card storage: Holds animations, VR assets, and configuration files.
  • Web-based interface: Upload, organize, and manage media without physically removing SD cards.
  • Extensibility: Hardware allows adding sensors, cameras, or new displays as the system evolves.

6. Summary

The hardware behind MOKUKU is a careful balance of:

  • Real-time control (ESP32)
  • Immersive visuals (dual displays + VR)
  • Responsive audio (mic + codec + AI)
  • Vehicle integration (CAN/OBD/IMU)

This foundation enables interactive, passenger-centric experiences—from virtual pets and gamified content to full VR/AR in-car adventures.