Gwanghwa Su: Machine learning based XR Monument
Delivered Gwanghwasu in Gwanghwa Sidae:The Age of Light, the national XR project series, incorporating AI deep learning solutions and AR technology
Client
Ministry of Culture, Sports and Tourism / KOCCA
Deliverable
UX Design, Interaction Design, Project Coordination
Role
UX Designer(Mobile, HMD), Project Coordinator
Ideation Process
The Ask: The request for proposal from KOCCA and the Ministry of Culture, Sports, and Tourism provided clear guidelines for the project. Specifically, the objective was to develop a digital forum that could effectively share the ideas, philosophies, and concerns of people around the world. To meet the technical requirements, the forum had to be an interactive, big data-based AR monument that leveraged SNS data and 5G technology. Finally, the forum needed to be accessible across a variety of devices, including mobile phones (iOS & Android), tablets, kiosks, and a type of HMD, which required us to prepare at least four different types of software clients. These specifications allowed us to focus our efforts and deliver a product that met the needs of our clients and their users.
The Idea: Our core concept for the project draws inspiration from the movie "Inside Out" and utilizes an AI classification method to analyze and visualize people's thoughts and emotions.
When a user creates and uploads a post to the cloud, our AI engine analyzes and classifies it into one of seven emotions, including happiness, anger, love, desire, worry, discontent, and sadness. The emotion is then visualized as a colored orb floating in the AR space, allowing users to see and interact with emotions in a novel and engaging way. This approach offers a unique perspective on how we can visualize and understand human emotions, and we believe it has the potential to inspire and inform users in meaningful ways.
Planning & Kick-off: To ensure the success of the project, I began by researching and identifying the necessary features and technologies. Given that the project required machine learning and data visualization capabilities, I reached out to several startups that specialize in AI technology, as well as a team of media artists who have experience working with data visualization.
With the support of these professionals, we were able to kick off the project successfully. As the project coordinator, I managed a team of 50+ specialists and oversaw the entire development process, ensuring that the project was completed on time and within budget. This involved coordinating and communicating with all stakeholders, managing project resources, and resolving any issues that arose during development. Through effective planning and coordination, we were able to deliver a high-quality product that exceeded our clients' expectations.
Conceptualization: Given that the project involved multiple teams of developers, designers, and artists, I recognized the need for a clear and comprehensive system architecture diagram. To facilitate collaboration and alignment across the different parties, I created a rough blueprint that allowed all team members to visualize the big picture and understand how their individual contributions fit into the broader project.
As someone without a coding background, this blueprint was especially helpful for me in communicating with the various types of developers and engineers involved in the project. By establishing a shared understanding of the project's goals and requirements, we were able to work more efficiently and effectively, delivering a high-quality product that met our clients' needs and expectations.
Design Process
Technologies & Challenges: As previously mentioned, this project was technology-heavy, requiring expertise in various areas such as SNS data crawling, machine learning (AI classification), web3D (Three.js), data visualizations, webAR & Vuforia engine (image and cylinder tracking), 3D model animation, and HMD augmentation (Nreal).
One of the biggest challenges we faced was achieving AR stability and visibility for this site-specific outdoor project. Despite encountering issues until the very end, it was a valuable opportunity to learn what visual elements and features work successfully in outdoor AR environments.
Another primary challenge was dealing with different SNS platforms' data policies, as we wanted to display meaningful posts from general users while adhering to guidelines and filtering out irrelevant spam. To ensure a clean and unbiased user experience across different cultural and political contexts, we applied filter words and created an admin platform to filter out inappropriate posts.
Coordinating a project involving multiple advanced technologies required me to quickly grasp and understand different contexts to communicate efficiently with various teams.
Design & Execution: Due to the non-disclosure agreement, I am unable to provide detailed information about the execution process. However, I have attached design output images for your reference. Please feel free to contact me if you have any questions or would like to know more about the design process.
Lesson& Result: Overall, this was a challenging project that presented many technical difficulties and required several design changes within tight timelines. However, it provided an excellent opportunity to learn about the integration of machine learning and data visualization with AR technology. The project commenced in February 2022 and if you wish to learn more about the project, you can visit the Gwanghwasu project website at https://gwanghwasu.gwanghwasidae.kr/.
