mAIking The Band, Part 2

_{This is part two of mAIking the band. Check out the first part here.}

In part 1, we discussed band names, and how we might write their stories with AI text models. We had a system that was working pretty well for that, so we moved on to the next element that we had agreed could define a band:

• Visual style, primarily captured through things like album covers

February 2019 was a good month indeed for fans of generative models, as StyleGAN was released just days before GPT-2. The groundbreakingly good image synthesis that StyleGAN was capable was a perfect candidate to make visuals in our quest to make an AI band. So, Chris trained a new model on a dataset of hundreds of thousands of album covers.

The results shocked us - from a distance, the AI-generated covers were by and large indistinguishable from a real, human-created cover. But of course, on closer inspection you can tell something is up; there aren't actual words (or often, letters), humans are recognizably human-shaped but distressingly lack facial features - the kinds of things you might see in a dream.

With a trained model ready, we wondered how we might be able to connect music with these generated covers. I had recently gotten a Roli Seaboard, which is notable for having a continuous pitch axis, instead of discrete keys - perfect to explore the continuous latent space of our GAN. Hacking things together one afternoon, I got our gan to run in real time responding to MIDI messages from the keyboard.

Once we had the code in place to interact with StyleGAN models in real time, all bets were off. I began building strange things that I wanted to exist, such as a "face theremin" that would use your webcam and pose detection as input to a GAN that generated faces:

The time of experimenting with faces peaked with a Build Your Own Astronaut experience I made for IDEO Chicago's portfolio review gallery walk. Colleagues were invited to use an XY pad to find a face they liked.

With that face, you would give your person a name, then a GPT-2-generated biography would help to create an astronaut baseball card in which the generated face was face-swapped onto an astronaut portrait.

This wasn't the end of the StyleGAN shenanigans - check out StyleGAN MTV for more!