Formalizing nuance in classical music
|
![]() random trip report |
David P. Anderson (with input from Rich Kraft)
1 May 2022
IntroductionThis essay is concerned with nuance in western classical music. In this context, nuance can be loosely defined as the differences between the score for a work and a rendition of the work, such as a human performance. These differences can be roughly divided into:
We focus on keyboard instruments like piano. Other instruments (and voice) are more complex because a) their notes have additional properties (such as attack and timbre), and b) the properties may change continuously during the note. The ideas presented here apply to such instruments, but would have to be extended to encompass these factors. For most classical music, nuance is a critical component of rendition. An example: "Wasserklavier" by Luciano Berio. The score is here.
Notice that in Grimaux's performance, no two beats have exactly the same duration, and no two notes have exactly the same volume. The nuance of the performance is a major factor in the beauty and expression of the rendition. Where does nuance come from? Some scores have indications of nuance: tempo markings, slurs, crescendo marks, fermatas, pedal markings, etc. These do not completely describe the nuance in a rendition, because:
Some musical styles have associated conventions for nuance. In many styles, for example, upbeat notes are softer, and pieces end with a ritardando. Performers learn these conventions by osmosis. The modern conventions may differ from those of the composition's period. Score markings and stylistic conventions are just guidelines. In the end, nuance is left up to the performer(s). Some nuance may be planned in advance. Some may be spontaneous during a particular performance. Some may be unintended consequences of the performer's technique. Formalizing nuanceI propose creating a "formalism" to describe nuance. This formalism should have these properties:
I have developed a formalism with these properties: Music Nuance Specification (MNS). Its details are described here, and a Python implementation of it is here. Why formalize nuance?To many musicians, nuance is ineffable - it's something magical that happens during performances, and to analyze or formalize it would break that magic spell. This viewpoint is understandable. But as music evolves, and as computers are increasingly important tools for composition, pedagogy, and performance, there are reasons to expand our ability to represent and manipulate nuance: to make it a first-class citizen, along with scores and sounds. Doing so will not replace the human component of nuance, or the spontaneity of performance; rather, it will provide tools that enhance these processes and that enable new ways of making music. Let's assume that we have a formalism describing nuance, and that we have software tools that make it easy to create and edit "nuance specifications" for pieces. These capabilities would have several applications:
CompositionAs a composer writes a piece, using a score editor such as MuseScore or Sibelius, they could also develop a nuance specification for the piece. The audio rendering function of the score editor could use this to produce nuanced renditions of the piece. This would facilitate the composition process and would convey the composer's intentions more clearly to prospective performers. Virtual performancePerformers could create nuanced virtual performances of pieces, in which they render the piece using a computer rather than a traditional instrument. PedagogyA teacher's instruction to a student could be represented as a nuance specification which guides the student's practice. This could be done in various ways. For example, as a student practices a piece they could be shown a "virtual conductor" that expresses (graphically, on a computer display) a simplified representation of the target nuance. Or a "virtual tutor" could make suggestions (musical and/or technical) to the student based on the differences between their playing and the nuance specification. Ensemble rehearsal and practiceWhen an ensemble (say, a piano duo) rehearses together, they could record their interpretive decisions as a nuance specification. They could then use this to guide their individual practice (perhaps using the "virtual conductor" described above). Sharing and archivalWeb sites like IMSLP let people share musical scores and recordings of renditions. Such sites could also include user-supplied nuance descriptions for works. This would provide a framework for sharing and discussing the interpretation of these works.
User interfaces for editing nuanceWhat kind of UI (user interface) would facilitate creating and editing nuance specifications - in particular, for transcribing one's mental model of a performance? This generally involves changing every parameter - start time, duration, volume - of every note. We can imagine a GUI that shows a piano-roll representation of the score and lets you click on notes to change their parameters. This low-level approach would let you do whatever you want, but it would be impossibly tedious. Desirable properties of a UI for editing nuance:
Some general approaches:
A research agenda for musical nuanceHaving a formalism for nuance opens up range of possible research in musicology. The most basic issue is what I'll call the "primitive selection problem". A nuance formalism (like MNS) provides a set of "primitives". Some of these define fluctuations in tempo or volume that affect lots of notes. Others apply random 'jitter' to timing and volume. Others affect sets of notes based on attributes such as position within a measure. Others apply to individual notes. The goal in designing the set of primitives is to find a small "basis set" of transformations, each with a small number of parameters, that can achieve the desired specifications - for example, that can closely approximate typical human performances. MNS, for example, has linear and exponential primitives for tempo change. This was easy to implement - but can these approximate ritardandos and accelerandos in practice? There may be better choices: Bezier curves, trig functions, polynomials. I can imagine a research program to study this, by calculating the nuance in human performances, and finding the primitives that approximate it best. The first step is to automatically extract nuance from human performances:
We can then use software to find a transformation that maps the score to the performance. This tranformation would typically have multiple levels. A first level would model large-scale fluctuations. The second level would take the residue from this, and fit it, possibly with different types of primitives. At some point the residue presumably would be noise-like, and its statistical properties could measured. Each level would consist of a set of primitives. The software would consider various families of primitives: in the case of continuous fluctuations this might include linear, polynomial, exponential, logarithmic, etc. The software would use data-fitting techniques to find an optimal basis set. It may turn out that the optimal set of primitives depends on
There has been some research in this general area. I've looked at some, but haven't found anything useful. Some papers study the statistics of deviation from the score, but not the actual modeling of it. |