Gesture Truck Challenge

client:

HK Science Museum

role:

Concept, UI, animation.

problem:

Translate complex mechanical movements to intuitive human gestures.

result:

Iinteractive system enabling seamless human-machine collaboration.

skills:

UX, motion, system calibration.

Context

Featured in the "Glorious Voyage" exhibition at the Hong Kong Science Museum, this project provided an interactive gaming experience for players aged 8 to 20+. It demonstrated how natural gestures can collaborate with electronic devices to navigate physical environments.

Challenge

The primary challenge was establishing a seamless mapping system between physical gestures and mechanical actions. We needed to ensure that untrained visitors could master moving, rotating, and lifting controls quickly without cognitive overload during the exhibition.

Approach

We implemented "paper" for movement and "stone" for rotation, alongside vertical gestures for the robotic arm. Real-time recognition was programmed to drive the motorized truck, while a clean ui provided immediate feedback to confirm the player’s intent.

Impact

The installation successfully transformed complex technology into an accessible educational experience. Participants gained practical understanding of human-machine collaboration by navigating physical obstacles, showcasing the potential of gesture control in spatial design and interactive storytelling.

Previous project

Heading

Interactive Wayfinding

learn more

Next project

Heading

270° Immersive Room

learn more