Musical Nuance Language


random trip report

Listen carefully to Arthur Rubinstein playing a Chopin Mazurka. You'll notice that no two beats have exactly the same duration, and no two notes have exactly the same volume. The performance has "nuance" - complex variations in timing, dynamics, and articulation. This nuance is not represented in the score. If we played the score exactly as written, it would sound like video game music. The beauty and emotion is largely in the nuance.

I have, for a long time, been interested in how to express nuance in computer-based systems for editing and playing music. This arises, in large part, from my desire to use computers to create "prepared performances" (see below) of classical piano music that I'm unable to play physically.

One approach is to integrate nuance with the score itself. I have some ideas about how this might be done. But it's a big project - it would involve major changes to score-editing software like MuseScore. I'm not sure I have the energy to do this, or the ability to get other people to do it. And in the end it might not work very well.

While on my bike today, I thought of a simpler approach: separate the nuance from the score. Apply nuance as a post-processing step, in Unix pipeline style. Express the nuance in a language designed just for that purpose (and not for score representation). I call this Musical Nuance Language (MNL). MNL would let you say things like:

  • Make the 2nd beat of each measure 10% longer.
  • Make the top and bottom notes of a chord 50% louder than the others.
  • Accelerate linearly by 30% from measures 4 to 8.

I call these "nuance primitives". The ones listed above are probably too simplistic; e.g. a good accelerando might not be linear. I assume there's a lot of research on the nuance used by real musicians, and we could use this research to define a good set of primitives.

Primitives might be defined in terms of time periods (e.g. a crescendo over 4 beats) or sequences of notes (e.g. the 2nd and 5th notes in a sequence are emphasized). Primitives may apply to all the notes during a particular period, or to a subset, e.g. to particular single voice. There are lots of details to work out.

To describe nuance compactly you need programming language features: e.g. to do something a certain number of times, or to define a function that takes parameters. So I propose that MNL be based on Python, which is a great language in general and has features that will be useful for MNL. So:

  • An MNL program is a Python program that invokes "nuance primitive" functions, and indicates where they should be applied in a score.
  • The input to an MNL program is a score. The logical choice of format is MusicXML, which can be generated by most score editors (MuseScore, Finale, Sibelius). One requirement: we need to be able to "tag" subsets of the score (e.g. voices) so that MNL can refer to them. I believe that MuseScore lets you do this; if not, it would be easy to add.
  • The output of an MNL program is a note list, with the precise timing and dynamics of each note. The logical file format is MIDI.

Trivial MNL programs could apply to any score: e.g. a crescendo from start to end. But in general MNL programs will work only with specific scores.

Anyway, that's the basic idea. I don't think it would be that hard to implement; I could do the basic framework in a few months, and then other people could flesh it out.

As described here, MNL is purely textual. You have to type in numbers for nuance parameters. This isn't ideal; when you're working with music it's better to have a nice GUI. This would be a version 2 thing.

Prepared performance

Computers can generate reasonable piano sounds (or you can get a MIDI grand). Software can play notes with precise timing and dynamics. So we can imagine creating a musical performance using a computer. You (the "performer") would use some software - perhaps MuseScore and MNL. You'd spend time - perhaps weeks or months - creating a rendition that sounds exactly how you want. You could then play it for a live audience, or put it on the Web. I call this a "prepared performance" (in contrast with a live performance).

When I talk to my pianist friends about prepared performance, their initial reaction is usually dismissive. This is understandable: the act of physically playing a piece is kind of magical: it combines emotion, physicality, listening, and spontaneity. It's hard to imagine that anything you do sitting at a computer could produce anything as meaningful.

But I have a different viewpoint. When I work on a piece, it starts as a mental process. I hear the piece, perhaps from a recording or by reading through it myself. It sits in my mind, sort of gestating. I imagine different ways it might sound, and different moods or feelings it might evoke. This translates into choices in tempo, dynamics, articulation etc. - i.e., nuance. This exploration might take place at the keyboard or away from it.

Eventually I have a mental model of how I want the piece to sound. Then there's the (usually arduous) task of getting my fingers to play in a way that approximates the mental model. And:

  • My fingers have limited strength and coordination, and my technique is not great. So when I practice the piece, I have to find a compromise between the mental model and what my fingers can do. The mental model gets blurry.
  • When I stop practicing the piece, it starts to disappear from my fingers. Soon I can't play it well anymore. If I want to perform it again - or if I have a new idea that changes the mental model - I have to do the practicing all over.

No such problems exist with prepared performance. If the software tools are good, you can realize your mental model precisely. And if you want to rethink the piece five years later, your performance is still there - you can tinker with it immediately, no practice needed.

And I reject the notion that there's something uniquely magical about a live performance. To me, the magic happened long before the performance; it happened as the mental model was forming.

By analogy: a painter forms a mental model of an image, and spends weeks or months realizing it in their studio. The result can be beautiful and can express emotion. We don't demand that the painter create the painting in front of the viewer. Why should we make the same demand of musicians?

Copyright 2021 © David P. Anderson