Integrating Music and Genetics through
Sonification and Data-Driven Music Composition

Abstract

This paper presentation will describe a semester-long interdisciplinary research experience for university-level student composers and biologists that is co-taught by a music professor and a biology professor. A part of the Mutational Music Project, this unique beyond-the-classroom experience integrates scientific research in genetics with creative activity in music technology. Composers use techniques from the fields of sonification, algorithmic composition, and data-driven music to assist the biologists in the sonic realization of their projects. Working in groups that pair composers with biologists, the students are asked to create a project that addresses the following question: In what way(s) can basic processes of genetics and evolutionary biology (especially mutation) be effectively represented through musical processes? The presentation will provide an overview of the course and discussion of the technological tools and methodologies employed, as well as selected project examples.

Handout

Presentation Handout (pdf)

University of South Carolina Courses

DUDYCHA BIOL 599
Topics in Biology: Chords and Codons, MW 2:20-3:35 pm
Coker Life Science Building, R202
Syllabus: pdf

Instructors

Interdisciplinary Research Experience Overview

The schedule below lists the combined activities from BIOL 599 & MUSC 540/(737). For a complete listing of activities see the Spring 2020 BIOL 599 and MUSC 540 syllabi (above), respectively.

SPRING 2020

The biologists and composers meet in their assigned groups from this point forward.

Final Report (Due: Wed., May 6, at noon)

Music Lectures

The three music lectures covered:

Lecture 1: Music as Organized Sound

• Music as "organized sound" (Varese and Wen-Chung 1966)
• Music as: perceived, encoded script, number, and data (LaRue 1970; Hofstadter 1979)
• Musical Instrument Digital Interface – www.midi.org
• Algorithmic composition (Nierhaus 2009; McLean and Dean 2018)
• Mozart's musical dice game (Gardner 1974)
• Computational thinking in music (Wing 2006; Edwards 2011)
  

Lecture 2: Sonification and Data-Driven Music

• The harmony of the spheres (Goodstein 2003)
• Sonification (Kramer 1994, Kramer et al. 1997; Hermann et al. 2011; Worrall 2009 & 2019)
• Data-driven music (Scaletti 2016; Vickers 2016)
• Gene music (Hayashi and Munakata 1984; Munakata and Hayashi 1995)
• Also: Takahashi and Miller 2007; McCormack et al. 2009; Taylor 2017; Temple 2017
  

Lecture 3: Mutational Music Project Ideas

• "Music as a gradual process" (Reich 1968)
• Visualizing data (Koblin 2009; Kuchera-Morin 2009)
• Survey of previous research in music and genetics (Bain)
• Parameter-mapping sonification using Cycling '74's Max & MIDI (Bain)
• Cycling 74's Max – https://cycling74.com/products/max

Student Projects

The research experience was offered during the Spring 2018 and Spring 2020 terms. The collaborative biologist-composer research groups produced the following projects:

Tools

The software tools used to create these projects include:

• Notation Programs
  Avid's Sibelius <https://www.avid.com/music-writing-software>
  MakeMusic's Finale <https://www.finalemusic.com>
  MuseScore <https://musescore.org/en>
  Steinberg's Dorico <https://new.steinberg.net/dorico/>

• DAWs
  Ableton Live <https://www.ableton.com/en/>
  Apple's Logic Pro <https://www.apple.com/logic-pro/>
  Reason Studios' Reason <https://www.reasonstudios.com/en/reason>

• Programming Languages
  Cycling '74's Max <https://cycling74.com/products/max>
  IRCAM's OpenMusic <http://repmus.ircam.fr/openmusic/home>

• Audio Editing/Recording
  Audacity <https://www.audacityteam.org>
  Avid's ProTools <https://www.avid.com/pro-tools>

Two Student Project Examples

EXAMPLE 1. Algorithmically-derived jazz from amino acid data
by biologists Kate Bothe & Michelle St. John, and composers Bryce Owens & Graeme Rosner

Video description:
One of two jazz compositions produced for this project, the following blues by Graeme Rosner was created from the AVPR1A gene, a gene associated with musical creativity. {YouTube: https://youtu.be/oEARsVYB1mk}

Paul Rosner, drums
Nate Lee, trombone
Graeme Rosner, flügelhorn

You can learn more about the composer's process at:

https://www.graemerosner.com/avpr1a-blues

Data credit:
UniProt.org, UniProtKB - P37288 (V1AR_HUMAN), Protein: Vasopressin V1a receptor; Gene: AVPR1A;
Available at: https://www.uniprot.org/uniprot/P37288#sequences

EXAMPLE 2. EEG Sonification: What Does Parkinson's Sound Like?
by biologists Abby Askins & Jack Gabel, and composers Te-Wei Huang & Jesse Kaiser

Audio recording description:
"This piece is a sonification of brain activity from five patients with Parkinson's disease and five healthy controls. You are hearing a composite of ten EEGs, with the pitch of the gamelan clustered for the two groups. Differences in the pitch represent the 'freezing of gait' that each of these Parkinson's patients experienced. This symptom has been shown to attenuate theta-band power and increase beta-band power at the Cz electrode during pedaling exercise. The audio is based on voltage measured from the Cz electrode and only sonifies a small portion of the total data displayed in the graphs."

Excerpt: A mp3 recording of the first 20 sec. of the 30 min. sonification

Data Credit:
EEG data obtained from: Singh, A., Cole, R. C., Espinoza, A. I., Brown, D., Cavanagh, J. F., & Narayanan, N. (2020). "Frontal theta and beta oscillations during lower-limb movement in Parkinson’s disease." Clinical Neurophysiology.

References

Ben-Tal, Oded and Jonathan Berger. 2004. "Creative Aspects of Sonification." Leonardo 37/3 (June 2004): 229–233.

Brooker, Robert J. 2009. Genetics: Analysis & Principles, 3rd ed. New York: McGraw Hill. {GB}

Burk, Phil, Larry Polansky, Douglas Repetto, Mary Roberts and Dan Rockmore. 2011. Music and Computers: A Theoretical and Historical Approach, Archival Version. Available online at: <http://musicandcomputersbook.com>.

Clark, Mary Ann, Matthew Douglas, and Jung Choi. Biology, 2/e. Houston, TX: OpenStax (Rice University). Available online at: <https://openstax.org/details/books/biology-2e>.

Edwards, Michael. 2011. "Algorithmic Composition: Computational Thinking in Music." Communications of the ACM 54/7: 58–67. Available online at: <https://cacm.acm.org/magazines/2011/7/109891-algorithmic-composition/fulltext>.

Gardner, Martin. 1974. "Mathematical Games: The arts as combinatorial mathematics, or how to compose like Mozart with dice." Scientific American 231/6 (December 1974): 132–137.

Goodstein, David L. and California Institute of Technology. 2003. Video. Episode 26 "The Harmony of the Spheres." from The Mechanical Universe...and Beyond. DVD. South Burlington, VT: Annenberg/CPB Project. Available online at: <https://www.youtube.com/watch?v=f3lwtHRLK6w>.

Hayashi, Kenshi and Nobuo Munakata. 1984. "Basically musical." Nature 310 (July 12, 1984): 96.

Hermann, Thomas, Andy Hunt and John G. Neuhoff, eds. 2011. The Sonification Handbook. Berlin: Logos Publishing House. Available online at: <https://sonification.de/handbook/>.

Hofstadter, Douglas. 1999/1979. Gödel, Escher, Bach: An Eternal Golden Braid, Twentieth Anniversary Edition. New York: Basic Books. {GB}

Koblin, Aaron. 2011. Flight Patterns (2009). In "Visualizing ourselves...with crowd-sourced data." TED 2011.
Available online at: <https://www.ted.com/talks/aaron_koblin_visualizing_ourselves_with_crowd_sourced_data/up-next?language=en>.

Kramer, Gregory, ed. 1994. Auditory Display: Sonification, Audification, and Auditory Interfaces. Proceedings Volume XVIII in the Sante Fe Institute Studies in the Sciences of Complexity. Reading, MA: Addison-Wesley. {GB}

Kramer, G., B. Walker, T. Bonebright, P. Cook, J. Flowers, N. Miner, and J. Neuhoff. 1997. "Sonification Report: Status of the field and research agenda," Technical Report. International Community for Auditory Display. Available online at: <http://www.icad.org/websiteV2.0/References/nsf.html>.

Kuchera-Morin, JoAnn. 2009. "Stunning data visualization in the AlloSphere." TED 2009.
Available online at: <https://www.ted.com/talks/joann_kuchera_morin_stunning_data_visualization_in_the_allosphere?c=322345>.

LaRue, Jan. 1970/2011. Guidelines for Style Analysis, Expanded 2nd ed. Warren, MI: Harmonie Park Press. {GB}

McCormack, Jon, Alice Eldridge, Alan Dorin, and Peter Mcilwain. 2009. "Generative Algorithms for Making Music: Emergence, Evolution, and EcoSystems." In The Oxford Handbook of Computer Music, edited by Roger T. Dean, pp. 354-379. New York: Oxford University Press. {GB}

McLean, Alex and Roger T. Dean, eds. 2018. The Oxford Handbook of Algorithmic Music. New York: Oxford University Press. {GB}

Munakata, Nobuo and Kenshi Hayashi. 1995. "Gene Music; Tonal assignments of Bases and Amino Acids." In Visualizing Biological Information, edited by Clifford A. Pickover, pp. 72–83. New York: World Scientific. {GB}

Nierhaus, Gerhard. 2009. Algorithmic Composition: Paradigms of Automated Music Generation. New York: Springer. {GB}

Reich, Steve. 2004. "Music as a Gradual Process." In Writings on Music, 1965-2000, edited by Paul Hillier. New York: Oxford University Press, pp. 34-36. {GB}

Scaletti, Carla. 2018. "Sonification ≠ Music." In The Oxford Handbook of Algorithmic Music, edited by Roger T. Dean and Alex McLean, pp. 363–385. New York: Oxford. {GB}

Takahashi Rie and Jeffrey H. Miller. 2007. "Conversion of amino-acid sequence in proteins to classical music: search for auditory patterns." Genome Biology 8/5, Article 405 (2007). Available online at: <https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1929127/>.

Taylor, Stephen Andrew. 2017. "From Program Music to Sonification: Representation and the Evolution of Music and Language.” The 23rd International Conference on Auditory Display (ICAD 2017)." {SMARTech}

Temple, Mark D. 2017. "An auditory display tool for DNA sequence analysis." BMC Bioinformatics 18/221 (2017). Available online at: <https://bmcbioinformatics.biomedcentral.com/articles/10.1186/s12859-017-1632-x>.

Varèse, Edgard and Chou Wen-Chung. 1966. "The Liberation of Sound." Perspectives of New Music 5/1 (Autumn - Winter, 1966): 11–19.

Vickers, Paul. 2016. "Sonification and Music, Music and Sonification." The Routledge Companion to Sounding Art, edited by Marcel Cobussen, Vincent Meelberg, and Barry Truax, pp. 135-144. New York: Routledge. {GB}

Wing, Jeannette M. 2006. "Computational Thinking." Communications of the ACM 49/3 (March 2006): 33–35. Available online at: <https://www.cs.cmu.edu/~15110-s13/Wing06-ct.pdf>.

Worrall, David. 2009. "An Introduction to Data Sonficiation." In The Oxford Handbook of Computer Music, edited by Roger T. Dean, pp. 354-379. New York: Oxford University Press. {GB}

Worrall, David. 2019. Sonification Design: From Data to Intelligible Soundfields. New York: Springer. {GB}

Acknowledgements

Special thanks to all of the students who participated in the Spring 2018 and Spring 2020 interdisciplinary research experiences.

The Mutational Music Project is the broader impact component of the National Science Foundation (NSF) grant Mutational variance of the transcriptome and the origins of phenotypic plasticity (NSF award #1556645), Jeff Dudycha, principle investigator. Reginald Bain is the other senior person on the grant.