Association for Technology in Music Instruction (ATMI) National Conference

Virtual Conference

October 11, 2020

Integrating Music and Genetics through
Sonification and Data-Driven Music Composition

Reginald Bain, Professor
Composition and Theory
School of Music
University of South Carolina
813 Assembly St.
Columbia, SC 29208 USA
rbain@mozart.sc.edu

Abstract

This paper presentation will describe a semester-long interdisciplinary research experience for university-level student composers and biologists that is co-taught by a music professor and a biology professor. A part of the Mutational Music Project, this unique beyond-the-classroom experience integrates scientific research in genetics with creative activity in music technology. Composers use techniques from the fields of sonification, algorithmic composition, and data-driven music to assist the biologists in the sonic realization of their projects. Working in groups that pair composers with biologists, the students are asked to create a project that addresses the following question: In what way(s) can basic processes of genetics and evolutionary biology (especially mutation) be effectively represented through musical processes? The presentation will provide an overview of the course and discussion of the technological tools and methodologies employed, as well as selected project examples.

Handout

Presentation Handout (pdf)

University of South Carolina Courses

DUDYCHA BIOL 599
Topics in Biology: Chords and Codons,
MW 2:20-3:35 pm
Coker Life Science Building, R202
Syllabus: pdf

BAIN MUSC 540/(737)
(Advanced) Projects in Computer Music
, TBA
Music Building, Computer Music Studio B, R011
Syllabus: pdf

Instructors

Jeff Dudycha, Professor
Department of Biological Sciences
College of Arts and Sciences
University of South Carolina
dudycha@biol.sc.edu
Website: www.tangledbank.org
Reginald Bain, Professor
Composition & Theory
School of Music
University of South Carolina
rbain@mozart.sc.edu
Website: reginaldbain.com

Interdisciplinary Research Experience Overview

The schedule below lists the combined activities from BIOL 599 & MUSC 540/(737). For a complete listing of activities see the Spring 2020 BIOL 599 and MUSC 540 syllabi (above), respectively.

SPRING 2020

Introductory Lectures/Activities
Mon., Jan. 13 Bio 1: Course Introduction
Wed., Jan. 15 Music 1: Music as Organized Sound
Wed., Jan. 22 Bio 2: Genetics Review
Mon., Jan. 27 Music 2: Sonification and Data Driven Music
Wed., Jan. 29 Bio 3: Mutation
Mon., Feb. 3 Music 3: Mutational Music Project Ideas
Meet the Composers
Wed., Feb. 5 Meet the Composers
Project Brainstorming
Mon., Feb. 17 Project Brainstorming
Wed., Feb. 19 Bio Group 1-5 Consultations
Feb. 20 - March 3 Bio-Music Group 1-5 Meetings

The biologists and composers meet in their assigned groups from this point forward.

Project Work, Progress Reports, and Consultations
Wed., March 4 Bio Group 1-5 Status Reports
Wed., March 18 Written Progress Reports/Consultations
Wed., March 25 Written Progress Reports/Consultations
Mon., April 6 Written Progress Reports/Consultations
Mon., April 13 Written Progress Reports/Consultations
Presentation
Mon., May 4 Bio Group 1-5 Project Presentations

Final Report (Due: Wed., May 6, at noon)

Music Lectures

The three music lectures covered:

Lecture 1: Music as Organized Sound

Lecture 2: Sonification and Data-Driven Music

Lecture 3: Mutational Music Project Ideas

Student Projects

SPRING 2020

Group Biologists Composers Title/Description
1 Libby Davenport
Patrick Lawson
Ian Jones
Jacob Wylie
The Harmonic Balance of Eat or Be Eaten
2 Kate Bothe
Michelle St. John
Bryce Owens
Graeme Rosner
Algorithmically-derived jazz from amino acid data
3 Jacob Brock
Dexter Reasons
Elizabeth Greener
Hunter Vowell
Mutations Sonified in a Fugue
4 Rishi Suresh
Frank Webb
Andrew Gretzinger
Peter Underhill
SoniPhylogenies:
Cytochrome B Sonification using BLOSUM
5 Abby Askins
Jack Gabel
Te-Wei Huang
Jesse Kaiser
What Does Parkinson's Sound Like?

SPRING 2018

Group Biologists Composers Title/Description
1 Lauren Huffmire
Kathryn Metts
Thomas Palmer
Morgan Soard
A genetic sequence is directly mapped to a chord
progression while implementing the properties of
various mutations
2 Zach Spicer
Matthew Waller
Ryan Williams Waltz Toward Disaster: A Representation of the
Accumulation of Mutations Over Time
3 Rachel May
Joel Strom
Michael VanBuhler
Robert Wilkenson
A familiar melody is altered according to the rules of
genetic mutation
4 Lexi Dickson
Olivia Harris
Jacob Wylie Hearing the Silent: Musically Expressing
Intronic Mutations

Tools

Software tools used in the student projects include:


Two Student Project Examples


EXAMPLE 1. Algorithmically-derived jazz from amino acid data
by biologists Kate Bothe & Michelle St. John, and composers Bryce Owens & Graeme Rosner

Video description:
One of two jazz compositions produced for this project, the following blues by Graeme Rosner was created from the AVPR1A gene, a gene associated with musical creativity. {YouTube}

Video: YouTube, https://youtu.be/oEARsVYB1mk


Paul Rosner, drums
Nate Lee, trombone
Graeme Rosner, flügelhorn

You can learn more about the composer's process at:

https://www.graemerosner.com/avpr1a-blues

Data credit:
UniProt.org, UniProtKB - P37288 (V1AR_HUMAN), Protein: Vasopressin V1a receptor; Gene: AVPR1A;
Available at: https://www.uniprot.org/uniprot/P37288#sequences


EXAMPLE 2. EEG Sonification: What Does Parkinson's Sound Like?
by biologists Abby Askins & Jack Gabel, and composers Te-Wei Huang & Jesse Kaiser

Video description:
"This piece is a sonification of brain activity from five patients with Parkinson's disease and five healthy controls. You are hearing a composite of ten EEGs, with the pitch of the gamelan clustered for the two groups. Differences in the pitch represent the 'freezing of gait' that each of these Parkinson's patients experienced. This symptom has been shown to attenuate theta-band power and increase beta-band power at the Cz electrode during pedaling exercise. The audio is based on voltage measured from the Cz electrode and only sonifies a small portion of the total data displayed in the graphs."


Video: YouTube, https://youtu.be/bqvwL6ubEpo

Data Credit:
EEG data obtained from: Singh, A., Cole, R. C., Espinoza, A. I., Brown, D., Cavanagh, J. F., & Narayanan, N. (2020). "Frontal theta and beta oscillations during lower-limb movement in Parkinson’s disease." Clinical Neurophysiology.




References

Ben-Tal, Oded and Jonathan Berger. 2004. "Creative Aspects of Sonification." Leonardo 37/3 (June 2004): 229–233.

Brooker, Robert J. 2009. Genetics: Analysis & Principles, 3rd ed. New York: McGraw Hill.

Burk, Phil, Larry Polansky, Douglas Repetto, Mary Roberts and Dan Rockmore. 2011. Music and Computers: A Theoretical and Historical Approach, Archival Version. Available online at: <http://musicandcomputersbook.com>.

Clark, Mary Ann, Matthew Douglas, and Jung Choi. Biology, 2/e. Houston, TX: OpenStax (Rice University). Available online at: <https://openstax.org/details/books/biology-2e>.

Edwards, Michael. 2011. "Algorithmic Composition: Computational Thinking in Music." Communications of the ACM 54/7: 58–67. Available online at: <https://cacm.acm.org/magazines/2011/7/109891-algorithmic-composition/fulltext>.

Gardner, Martin. 1974. "Mathematical Games: The arts as combinatorial mathematics, or how to compose like Mozart with dice." Scientific American 231/6 (December 1974): 132–137.

Goodstein, David L. and California Institute of Technology. 2003. Video. Episode 26 "The Harmony of the Spheres." from The Mechanical Universe...and Beyond. DVD. South Burlington, VT: Annenberg/CPB Project. Available online at: <https://www.youtube.com/watch?v=f3lwtHRLK6w>.

Hayashi, Kenshi and Nobuo Munakata. 1984. "Basically musical." Nature 310 (July 12, 1984): 96.

Hermann, Thomas, Andy Hunt and John G. Neuhoffeds. 2011. The Sonification Handbook. Berlin: Logos Publishing House. Available online at: <https://sonification.de/handbook/>.

Hofstadter, Douglas. 1979. Gödel, Escher, Bach: An Eternal Golden Braid. New York: Basic Books.

Koblin, Aaron. 2011. Flight Patterns (2009). In "Visualizing ourselves...with crowd-sourced data." TED 2011.
Available online at: <https://www.ted.com/talks/aaron_koblin_visualizing_ourselves_with_crowd_sourced_data/up-next?language=en>.

Kramer, Gregory, ed. 1994. Auditory Display: Sonification, Audification, and Auditory Interfaces. Proceedings Volume XVIII in the Sante Fe Institute Studies in the Sciences of Complexity. Reading, MA: Addison-Wesley.

Kramer, G., B. Walker, T. Bonebright, P. Cook, J. Flowers, N. Miner, and J. Neuhoff. 1997. "Sonification Report: Status of the field and research agenda," Technical Report. International Community for Auditory Display. Available online at: <http://www.icad.org/websiteV2.0/References/nsf.html>.

Kuchera-Morin, JoAnn. 2009. "Stunning data visualization in the AlloSphere." TED 2009.
Available online at: <https://www.ted.com/talks/joann_kuchera_morin_stunning_data_visualization_in_the_allosphere?c=322345>.

LaRue, Jan. 1970/2011. Guidelines for Style Analysis, Expanded 2nd ed. Warren, MI: Harmonie Park Press.

McCormack, Jon, Alice Eldridge, Alan Dorin, and Peter Mcilwain. 2009. "Generative Algorithms for Making Music: Emergence, Evolution, and EcoSystems." In The Oxford Handbook of Computer Music, edited by Roger T. Dean, pp. 354-379. New York: Oxford University Press.

McLean, Alex and Roger T. Dean, eds. 2018. The Oxford Handbook of Algorithmic Music. New York: Oxford University Press.

Munakata, Nobuo and Kenshi Hayashi. 1995. "Gene Music; Tonal assignments of Bases and Amino Acids." In Visualizing Biological Information, edited by Clifford A. Pickover, pp. 72–83. New York: World Scientific.

Nierhaus, Gerhard. 2009. Algorithmic Composition: Paradigms of Automated Music Generation. New York: Springer.

Reich, Steve. 2004. "Music as a Gradual Process." In Writings on Music, 1965-2000, edited by Paul Hillier. New York: Oxford University Press, pp. 34-36.

Scaletti, Carla. 2018. "Sonification ≠ Music." In The Oxford Handbook of Algorithmic Music, edited by Roger T. Dean and Alex McLean, pp. 363–385. New York: Oxford.

Takahashi Rie and Jeffrey H. Miller. 2007. "Conversion of amino-acid sequence in proteins to classical music: search for auditory patterns." Genome Biology 8/5, Article 405 (2007). Available online at: <https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1929127/>.

Taylor, Stephen Andrew. 2017. "From Program Music to Sonification: Representation and the Evolution of Music and Language.” The 23rd International Conference on Auditory Display." Pennsylvania State University.

Temple, Mark D. 2017. "An auditory display tool for DNA sequence analysis." BMC Bioinformatics 18/221 (2017). Available online at: <https://bmcbioinformatics.biomedcentral.com/articles/10.1186/s12859-017-1632-x>.

Varèse, Edgard and Chou Wen-Chung. 1966. "The Liberation of Sound." Perspectives of New Music 5/1 (Autumn - Winter, 1966): 11–19.

Vickers, Paul. 2016. "Sonification and Music, Music and Sonification." The Routledge Companion to Sounding Art, edited by Marcel Cobussen, Vincent Meelberg, and Barry Truax, pp. 135-144. New York: Routledge.

Wing, Jeannette M. 2006. "Computational Thinking." Communications of the ACM 49/3 (March 2006): 33–35.

Worrall, David. 2009. "An Introduction to Data Sonficiation." In The Oxford Handbook of Computer Music, edited by Roger T. Dean, pp. 354-379. New York: Oxford University Press.

Worrall, David. 2019. Sonification Design: From Data to Intelligible Soundfields. New York: Springer.

Acknowledgements

Special thanks to all of the students who participated in the Spring 2018 and Spring 2020 interdisciplinary research experiences.

The Mutational Music Project is the broader impact component of the National Science Foundation (NSF) grant Mutational variance of the transcriptome and the origins of phenotypic plasticity (NSF award #1556645), Jeff Dudycha, principle investigator. Reginald Bain is the other senior person on the grant.

NSF logo


Updated: June 20, 2021

Reginald Bain | University of South Carolina | School of Music
https://in.music.sc.edu/fs/bain/atmi20/