Language Matrix

Module 6 “Design and Research of User Experience” · with Michela Guizzetti, Floor den Hertog, Thanh My Luong and Mischa Telman

The idea

Many Ukrainian refugees in Dutch shelters barely speak Dutch, and the shelters are often cut off from the neighbourhood around them. Our interviews with refugees and locals in Amsterdam-Noord showed both groups wanted more contact. The language barrier was the blocker.

So we turned the barrier into the game. A main screen asks multiple-choice questions in several languages. The answers appear on LED-matrix answer boxes spread around the room, and all players have to run to their box and press the same answer at the same time. Your mother tongue becomes an advantage, and not understanding something starts a conversation.

What I built

• The answer boxes: each one is an ESP32 with an LED matrix display and buttons, programmed in C++. They show letters, pictograms, a screensaver, and animations for correct or wrong answers.
• The connection: the boxes talk to the Processing game on the main screen over Wi-Fi with MQTT, in both directions: game phase and right/wrong one way, connection status and button presses the other.
• Research and testing: interviews with refugees and locals, paper (lo-fi) prototypes, and user tests comparing an English-only and a multilingual version.

Result

The multilingual version clearly worked better. Players leaned on each other, discussed answers and described it as a fun way to meet new people. In testing the buttons sometimes lost connection, which taught me a lot about making wireless hardware reliable. Next steps would be more cultural content (food, traditions) and a version that doesn’t rely on running, so it works for all ages.