February 17, 2021This week we were asked to create an application that can be run from the command line and that uses text-to-speech functionality.
Inspired by Valentine’s Day, classical counterpoint, and tech-celebrity romance, my project juxtaposes Elon Musk’s tweets via synthesized audio with music by his partner in life/crime, Grimes. The result is a playful composition reminiscent of glitch poetics.
Basic Flow of The Application
- Call to Twitter’s API
- Request data (the most recent 40 tweets by Elon Musk)
- Twitter returns the data
- Package the data into a single string
- Send the string to AWS Polly to request an .mp3 file of the synthesized voice
- Send a request to AWS Polly for metadata (returned as JSON) that correlates to the start and end times (duration) of the speech
- The JSON data is saved into 2 .txt files; one file contains the timecodes of the speech in milliseconds, the other contains the text
ffmpeg -loglevel quiet -i elon.mp3 -y elon.wav
- The .mp3 file is converted to a .wav file
chuck --dac6 patch.ck
- Run a script in ChucK (audio programming language) that imports the .txt files and converts their content into arrays
- Create a generative musical composition combining the synthesized voice and an audio recording of Grimes - Oblivion
Written by David Currie, a current student at NYU ITP. Follow me on Twitter