The Flaming Lips & Fruit Genie at Google I/O

We partnered with Google’s Magenta team and The Flaming Lips to create an AI-assisted performance as part of the headline concert at I/O 2019. The result was Fruit Genie: a real-time intelligent musical instrument which combines Magenta’s Piano Genie model with a physical interface consisting of fruit.

Challenge

We knew we wanted to integrate AI into the work of one of the most innovative bands in the music industry, The Flaming Lips. Luckily, we had a great source of inspiration in the catalog of existing work by the Magenta team.

Strategy

After a day of brainstorming, The Flaming Lips kept coming back to a program called Piano Genie for both its novel input and AI-powered dynamic output. Piano Genie is a real-time system which translates inputs on a simple controller into plausible music.This made it a natural pairing with band’s iconic live experience which consists of real-time musical and visual elements.

Execution

We didn’t have the actual Piano Genie button-based hardware in our studio that day, but we did have an enterprising electrical engineer named Nico who said he could put something together. He quickly got the hardware up and running that could send MIDI to the Piano Genie software, but we didn’t have any large buttons on hand. In a stroke of genius (or maybe desperation), Nico grabbed a plate of fruit, inserted some leads into each piece, and used the natural capacitance to turn grapes, oranges, and pears into “buttons.” Now, whenever you touched a piece of fruit, it generated a MIDI trigger which the Piano Genie software then turned into a musical note.

With the Fruit Genie in their arsenal, The Flaming Lips began experimenting and writing a new song. Meanwhile, we started to think about the performance at I/O. We wanted to draw on the tradition of the Lips’ larger-than-life interactive shows, so we took some inspiration from the band’s history of throwing out large inflatables to the crowd and ran with it. Using some of Wayne’s designs, we created oversized, fruit-shaped balloons and outfitted them with lights, accelerometers, and wireless communication capabilities.

Whenever someone in the crowd batted a balloon up into the air, it sent a signal to the Piano Genie software to play a note. Together, The Flaming Lips, a bowl of fruit, Google AI, and the audience at I/O played a song like no other.

Fruit Genie is open source. Check out the instructions on GitHub