Music notation 2.0


random trip report
THE FOLLOWING ISN'T FINISHED YET. I NEED TO MAKE SOME EXAMPLES.

Introduction

There are many ways in which technology, such as computers and hi-res displays, could be used to enhance how we learn and perform music. Here I'm concerned with improving the "scores" used while practicing and performing music, and classical piano music in particular.

Structured scores

Traditional classical music scores, consisting of black-and-white printed pages, are static and graphical. They use Western Music Notation (WMN), the system of staffs, note-heads, stems, ties, slurs, dots, dynamic markings, etc. This system has remained static for 200+ years, although some contemporary composers have introduced ad-hoc notations for things like indeterminacy and clusters.

To date, the main applications of technology (computers and the Internet) to music scores are:

  • Web archives of PDF images of scores (IMSLP).
  • Display of scores on tablets (forScore).
  • Programs for score editing and printing: Finale, Sibelius, MuseScore, etc.
  • A standard for machine-readable WMN representation (MusicXML).

Technology can also let us extend the idea of score, in terms of both structure and content. I propose a new type of score called "structured scores", building on WMN and existing technology.

Figure 1: the components of structured scores

The "abstract part" of a structured score consists of:

  • A data structure describing the composition, in a standardized computer-readable format such as MusicXML.
  • Possibly several "annotation layers" indicating dynamics, rhythm variation, articulation, fingerings, footnotes, and other textual annotations. The composer may create an annotation layer, and editors may add another one. Performers can change or remove these layers, as well as adding their own. Annotation layers are also represented in standardized XML.
Software (ideally free and open-source) provides several capabilities:
  • Edit abstract scores; in particular, performers can create and edit annotation layers, using the full set of notational features used by the composer and editor; they'll no longer have to scribble in pencil.
  • "Render" an abstract score to a visual representation or a "concrete score", to be used by the performer during practice and/or performance. The perform can control the parameters of the rendering, e.g., the size, color scheme, spacing between notes and staffs, etc. The concrete score may be interactive - see below.
  • A playback function that renders an abstract score as sound, using synthesized instruments. This is useful for "debugging" scores.
Abstract scores will hopefully become the way music is sold, archived, and shared among performers. When we download from IMSLP or buy a piece of music, we'll get an abstract score.

Our personalized scores will be stored in the cloud. A version control system (ScoreHub?) will let us undo changes and have multiple branches. We'll be able to share and discuss our changes with other performers, including our collaborators in ensemble pieces.

Notation ideas

Now let's think about the notational possibilities enabled by technology. I have some ideas - but first, let me clarify some things:

  • I propose extending WMN, not changing it. In general, I think WMN is great; and we've all made a huge investment in learning it at a neural level.
  • I'm not proposing creating a "standard", or requiring anyone to do anything in particular. I want notational features that help *me* - features that I'll use in adding my own annotations to scores. In the course of doing this I'll talk with other performers who have similar goals; maybe we'll reach a concensus on good ways to do things. We'll make scores with our new notational features, pass them around, and publish them on IMSLP. Maybe some of the features might catch on and become a de-facto standard. In that case composers could use them in their interpretive layers. Maybe at some point some organization will define a true standard.
  • I propose features that allow a more complete and precise specification of nuance than is currently possible. But that doesn't mean I want to eliminate spontaneity and unpredictability from live performance! I want to be able to make a clear record of my interpretive decisions, so that I can share them with collaborators, and so that I'm reminded of them if I put the piece aside for a while and return to it. Also, I want to enable composers to specify nuance to the extent they desire.
  • The figures of merit of a notation are a) its incremental expressive power; b) the ease with which it can be learned, and parsed during performance.
  • Features can be proposed and discussed, but they don't really exist until they're implemented in a score editing/display program (and ideally implemented in its playback function as well, so you can hear what you notate).

OK, now for some possible features. I'm just throwing a bunch of idea out there.

Dynamics

Note-level dynamics (including chord voicing): possibilities:

  • Color-coding: loud in red, soft in green (or some color spectrum). Or loud in black, soft in gray.
  • Note-head size: bigger is louder (already used in WMN sometimes)
  • Pulsed note-heads; deeper or faster pulse = louder.
Traditional scores are black on a white background. If we're going to use color and gray-scale, we might want to consider black backround; it may be easier to visually distinguish colors that way.

Overall dynamics:

  • Vertically scale entire staff, bigger = louder? (might be confusing, especially with abrupt changes).
  • Change color or line weight of entire score.

Grouping

It should be possible to group notes (e.g. fugue themes), and highlight groups in different ways:

  • With a colored background, like a Hi-Liter.
  • Drawing the noteheads differently (color, size, pulsation).

We might want to demarcate phrases (e.g. 8 bar units), perhaps with some indication of shape within a phrase (e.g. the high point). This could be used in "interactive scores" (see below) in various ways, e.g. show only the current phrase, or highlight it in some way.

Rhythm

I want to be able to denote fluctuations in the underlying pulse. One way to do this is with "proporational notation", in which the horizontal space for a note is proportional its notated duration. We can then draw light vertical "grid lines" on quarters or eighths. The distance between lines is the duration of the interval, and the variation in this spacing shows fluctuation.

To make this easier to visually parse, we could tinge these vertical strips with red (faster than mean) to white (mean) to blue (slower than mean). (Although this could conflict with other uses of color.)

It would make sense to center note-heads on the grid line when they occur. Deviations from the underlying pulse (playing ahead of or behind beat) could then be denoted by offsetting note-heads from the grid lines. This is different from measure lines in WMN; if we use this notation, we'd need to change measure lines so that they pass through downbeat note-heads; things would be confusing otherwise.

Articulation

A more precise notation of articulation (staccato/portamento/legato) would be good; an indication of how long each note sounds, or is held down. I don't have any good ideas here.

Performance scores

Until now I've been talking about what I'll call "reference scores" - complete representations of a piece. By "performance score" I mean what's displayed to the performer during performance or practice.

With traditional scores, these are the same thing: performers see 1 or 2 pages at once, and have to keep track of where they are on the page. But with technology, they can be different. For example, a system can "listen" to the performer (via sound or MIDI), keep track of where they are, and display only a subset of the reference score: maybe a few seconds. Advantages:

  • "Now" would always be at the same place on the display.
  • The display can be larger, easier to parse, and can contain more information.
  • No page turns.

Because a performance score changes over time, there are new notational possibilities: for example, the displayed portion could change in size or color to indicate dynamics or tempo.

A performance score doesn't necessarily have to show everything in the reference score. It might show interpretive info and a few key notes. In the extreme case (playing from memory) it could be empty.

A performance score could potentially be any sensory input - 3-D virtual reality, tactile feedback, etc. But for now let's stick with 2-D displays.

Implementing this would require major changes to MuseScore.

Next steps

  • Study MusicXML and design extensions if needed to handle a) annotation layers and b) new notations.
  • Locate and study existing work in this area.

References

  • notat.io Message boards about notation. Some very smart participants.
  • LilyPond: a textual Latex-like notation language, and software that renders it graphically. Feature-rich, and the site explores notational issues. Supports proportional notation and grid lines.
  • MusicXML DTD for MusicXML. Used by various software: https://www.musicxml.com/software/
  • MuseScore: desktop score editor. Open source. Uses MusicXML or its own file format. Used to support LilyPond too.
  • Noteflight: web-based score editor. Open-source.
  • Finale: desktop score editor, commercial.
  • Sibelius: desktop score editor. Commercial, limited version is free. No "export" option AFAIK.
  • ScoreCloud. Web-based score editor. Save to local file only if pay.
  • Flat. Web-based score editor.
  • Smartmusic. Music-teaching software.
  • A course in music notation at Stanford
  • Tomplay: an app follows along with you in the score. They sell the scores, which are in a proprietary format.

Copyright 2020 © David P. Anderson