Duration: 4 months
Thesis Log: Link in Medium
Project Type: Thesis
Visual Programming Language, Gen AI, User Research, Experience Design
_____________________________________________________________________________
What is harmonyAssistant?
An OpenAI based AI chatbot assistant
Only in Max/Msp(visual language programming software for audio visual design)
Problem Statement
“How can Gen AI play as an assistant in visual programming language design software like Max/Msp?
Novice designers are experiencing long and shallow learning curve. They would be able to create more fascinating work by getting hints from Gen AI bot before and during programming. In order to make this bot, large study data and patches would need to be fed into AI for training and improve onboarding flow .”
Artist Statement
I love the beauty and funky nature of electronic based art work, but also as a novice user in visual language coding I struggled a lot when I was doing my very first project. Along the way I saw a lot of other designers who are new to this programming language were in my shoes. We hope there would be some helper to guide us through. In 2023 I met Gen-AI and while using it I thought that I could merged AI and Max to craft something truly special for beginners. I was inspired by the stunning capabilities of AI in studying and analyzing large database and write codes with its interpretation. I designed harmonyAssistant to create a hop on stool for all novice designers could utilize Max and create the audio visual in their heads. Imagine playing a puzzle map where every detail whispers of possibility and innovation. This work sits where art and tech meet, with this stool I want to see more groundbreaking audiovisual pieces produced by novice designers.
This is the basic form of harmonyAssistant. Top box use for prompt and lower one is for answer.
User Research
To understand the user more in onboarding flow, I draw out their process and decision points in flow chart. Identified two most important change maker spaces. One is after first opening the software and second is after tutorial exploration. These two opportunity points led to my future user research on obstacles user met in these two specific areas.
