Google Scholar  /  Twitter

Cheng-Zhi Anna Huang 黃成之

In Fall 2024, I started a faculty position at Massachusetts Institute of Technology (MIT), with a shared position between Electrical Engineering and Computer Science (EECS) and Music and Theater Arts (MTA). For the past 8 years, I have been a researcher at Magenta in Google Brain and then Google DeepMind, working on generative models and interfaces to support human-AI partnerships in music making.

I am the creator of the ML model Coconet that powered Google’s first AI Doodle, the Bach Doodle. In two days, Coconet harmonized 55 million melodies from users around the world. In 2018, I created Music Transformer, a breakthrough in generating music with long-term structure, and the first successful adaptation of the transformer architecture to music. Our ICLR paper is currently the most cited paper in music generation.

I was a Canada CIFAR AI Chair at Mila, and continue to hold an adjunct professorship at University of Montreal. I was a judge then organizer for AI Song Contest 2020-22. I did my PhD at Harvard University, master’s at the MIT Media Lab, and a dual bachelor’s at University of Southern California in music composition and CS.

Research Interests

I’m interested in taking an interaction-driven approach to designing Generative AI, to enable new ways of interacting with music (and AI) that can extend how we understand, learn, and create music. I aim to partner with musicians, to design for the specificity of their creative practice and tradition, which inevitably invites new ways of thinking about generative modeling and Human-AI collaboration.

I propose to use neural networks (NNs) as a lens onto music, and a mirror onto our own understanding of music. I’m interested in music theories and music cognition of NNs and for NNs, to understand, regularize and calibrate their musical behaviors. I aim to work towards interpretability and explainability that is useful for musicians interacting with the AI system. I envision working with musicians to design interactive systems and visualizations that empower them to understand, debug, steer, and align the generative AI’s behavior.

I’m also interested in rethinking generative AI through the lens of social reinforcement learning (RL) and multi-agent RL, to elicit creativity not through imitation but through interaction. This framework invites us to consider how game design and reward modeling can influence how agents and users interact. I envision a jam space, where musicians and agents can jam together, and researchers can swap in their own generative agents and reward models, similar to OpenAI’s Gym. The evaluation is not only on the resulting music, but also on the interactions, how well agents support other players. I’m also interested in efficient machine learning, to build instruments and agents that can run in real-time, to enable Human-AI collective improvisation.

We have multiple Postdoc positions open for Fall 2025 at MIT Music Technology. I'm particularly interested in thinking about multi-agent reinforcement learning, human-ai communication, interpretability, and visualization, in the context of designing interactive systems that enable musicians to jam with generative agents.

For PhD, apply through MIT EECS (by Dec 1st). For indicating "research field of interest" in the application form, if we don't have music technology included in the dropdown menu yet, you can choose any field you're most interested in, such as (but not limited to) ML General Interests, Natural Language and Speech Processing, Reinforcement Learning, Deep Learning, Human Computer Interaction, or Cognitive AI, etc. We'll reach out if additional information is needed. You'll probably hear back from MIT in February. Best of luck!

This website was built thanks to the help of this source code.