Skip to content

research

Directory actions

More options

Directory actions

More options

Latest commit

 

History

History

research

Folders and files

NameName
Last commit message
Last commit date

parent directory

..
README.md
README.md
 
 
nns.md
nns.md
 
 
product_ideas.md
product_ideas.md
 
 
spring_course.md
spring_course.md
 
 
tokenizer.md
tokenizer.md
 
 
vgm_music.md
vgm_music.md
 
 

README.md

Who am I?

  • I'm Vitaly Pavlenko
    • an independent researcher of music theory
    • with a background in creating mobile and web products
    • also with a lot of teaching experience
  • In the past:
    • I got a BSc in Applied Math and CS at MIPT (Moscow)
    • After that, I've studied at Skoltech, ШАД and МКИМ
    • I've created pythontutor.ru, which is a popular Python course that's used by thousands of people every day
    • I've taught:
    • I've mentored teams doing tech and startup projects at MIPT and HSE SPb
    • I've also worked at WhatsApp, ASAP and Jetway
  • Now:
  • I'd be very glad to mentor students willing to do research in computational music theory 👩‍🔬🔬🧑‍🎓
  • t.me/vitalypavlenko

What is music theory?

"music theory is the study of how music makes meaning through its structure"

Claire Boge, quoted in "Teaching Music Theory" by Jennifer Snodgrass

Music theory is our knowledge of what musicians and composers do and do not. Also, it's about what listeners hear.

Music can be roughly split into notes (what keys does a pianist press?) and sound (how producers combine effects to create timbres?). I focus on the notes.

The questions I ask:

  • In which order do notes go together in different music?
  • Are there statistical patterns that musicians regularly apply?
  • Do different composers use different toolboxes? Where does it come from?
  • Do listeners actually know and hear that?
  • Do sequences of notes, chords, motives and form structures represent genres and styles? How?

I care about all tonal musical cultures: not only the classics and not only Western. For me, music theory is 95% theoretical linguistics and 5% mathematics.

Things I don't care about:

  • black box automatic music generation
  • rhythm and drumming
  • atonal music and other avant-garde experiments if they don't either create an audible local language or don't reuse a global language

How to start?

Browse topics below. Pick one or design your own. Reach out to me to brainstorm over the possible work plan. Start. Iterate.

Work principles

  • Do corpus-based research: choose from 10 to 100k pieces and analyze them to find patterns
  • Present the discoveries through web demos rich in audio examples and visual explanations
  • Anti-perfectionism: release and demo as soon as there's something to play with
  • Work in extremely small iterations: one week is great, one day is even better
  • Limit the scope of the very first steps, make a humble MVP on the week one and iterate from there
  • Help others with forking and reproducibility: publish the research code in open source repositories with clear setup instructions
  • Consider making something publishable in real music theory journals like Music Theory Online
  • However, optimize for teaching your friends on what you've discovered
  • Listen to the type of music you're researching, even if it's outside of your comfort zone
  • However, optimize for working on music that you deeply care about
  • Read books and papers to get an idea of how other researchers understand its structure
  • Switch back and forth between manual transcription/annotation and large-scale DS/ML processing
  • Write down questions before the research

Feel free to adjust them to your taste. I'm happy to navigate you towards reaching this style of research.

For the research and DS/ML, you may use any available tools and libraries. For visualizing the results, I personally use Githup Pages + React + TypeScript and I often ask GPT4 for help. I'm happy to help you with this stack (or any other).

Things will be faster if you have some background in music theory. You most likely have one if you've learned to play some instrument before. It's probably enough if you have an idea of what chords are, what's major/minor, what's an octave and a fifth, and if you can potentially play some songs given the chords - maybe on piano, guitar, ukulele or in the DAW. Or at least if you understand how others do that.

The curve to learn other useful concepts like Roman numerals, seventh chords, cadences and modulation can be steep. However, I'm very happy to navigate you through learning it as you proceed with your research.

Topics

Below I give topics which I care about. I'm happy to work out any other research topics based on your personal interests. It's way better if you scratch your own itch and tackle the questions you can't stand being unanswered anymore. Let's make music structure uncovered, understood and clearly visualized, one question at a time.

Jazz harmony

One of the key aspects of functional jazz music is harmony - the chain of chords that is played over and over again in choruses.

Jazz harmony is often pretty regular. Chords can be grouped into blocks like ii-V-I or chain of dominants.

Jazz music is constantly being developed: new composers compose new tunes. Every tune is usually represented as a lead sheet: chords + melody. Chords on a lead sheet help musicians to render a tune. However, they somewhat obscure the original composer's idea, and there may be better way to represent the harmonic structure.

  • Can we trace birth and death of certain lego blocks across history? (We can trace certain chords, for instance)
  • Do composers and subgenres of jazz have certain preferences of lego blocks?
  • Can we visualize harmony of all jazz standards at a single page to uncover patterns?
  • Can the knowledge of lego blocks improve automatic lead sheet generation?
  • For abstract jazz OSTs like Side Pocket 1&2&3, is there a way to credibly type it stylistically?

Jazz solos

Another key aspect of jazz is solo: a chain of phrases that a single musician plays over a chain of chords. Every jazz musician grows his solo vocabulary for the entire life by combining common patterns with personal discoveries and taste. Some researchers like Norgaard or Frieler tried to describe patterns in this activity. However, the clear picture is still absent.

  • Can Frieler's models (or any models from Impro-visor) improve jazz solo transcription?
  • Can Frieler's models help us build a real-time imrpovisation instrument that's fun to use? (Touch screen instruments? Music Mouse-like?)
  • What use can we make of Doug McKenzie's dataset of recorded and annotated solos?

Bass lines

Bass lines are everywhere in Western music. They can easily be separated from the track using demucs and further transcribed using basic-pitch.

A theory for bass lines should be more regular, because, in most cases, a bass line is inferred from a current chord, and in some cases it has a clear structure (riff or shape).

The questions for bass lines:

  • Can we demonstrate that bass lines become complex as a single musician matures? How to measure complexity/creativeness of a musician from album #1 to album #10?
  • Can we make automatic comparative analysis of different bassists playing the same pieces?
  • Are bass lines different for the same genre across different countries? Eg. do Italian metal bassists do the same things which French metal bassists don't do? Is "region" a thing in enthropy (perplexity) growth as we zoom out from a single musician to a whole genre?
  • What's the easiest visual way to represent bass line patterns? Can we show piano roll bass patterns over everynoise?

Expression in classical piano

A pianist improvises by controlling velocities and onsets of all notes written out by composer, without changing the order of the notes. The actual rendition is a product of:

  • muscle abilities and limitations
  • composer's marks on a music sheet
  • cultural inertia of how should a certain piece sound
  • composer's consistent taste across the pieces
  • composer's approach to this individual piece

Can we decompose a certain piano rendition onto these axes?

Can we visualize the real expressiveness of a certain recording right on the score?

What are the patterns of expression for certain pieces/composers/styles in terms of volumes/onsets? How can we simplify and visualize it?

Rock harmony

On the one hand, rock harmony is theorized in several modern books by manual annotation and categorization of chord progressions.

On the other hand, we have a lot of datasets of chords for the long tail of rock music: both uploaded by people and done automatically.

  • Can we use these datasets to generalize our model to other countries / rock bands?
  • Can we show that a harmonic complexity grows as the band gets older?
  • Can we show trends in using certain chord progressions over times and countries?
  • Can we draw state machines for chords for certain bands to compare harmonic languages?

Video game music

Video game music is a diverse representation of mostly Western musical idiom. It's usually short in form and sometimes it's solving certain tasks related to representing a game's content: location, emotion, cultural context. VGM began with a third generation of game consoles, gradually expanding it's capabilities in timbre and number of voices as audio chips matured.

I speculate that if we take the same means of expression - a SN76489 - then we can probably make an argument about the development of harmony, texture, form and style across time. At least by directly comparing two sets of OSTs: Sega Master System (1985 - 1996) and Sega Game Gear (1990 - 1997)

Listen to some https://chiptune.app/

Some research articles:

Film music

Film music has one certain feature that's rarely used in other Western music: non-functional two-chord changes that are usually described as neo-Riemannian or as chromatic mediants.

  • Given an entire film's soundtrack, can we reliably identify these two-chord patterns in the soundtrack?
  • If "yes", can we batch-process a lot of film music to answer the following:
    • is there evolution of usage of them over time?
    • do all composers use them in the same way?
    • do they have stable semantic load? (can we separate "positive"/"negative", sorts of magic?)

Barbershop quartets

Barbershop is a culture that nearly died at the start of the 20th century and then was semi-artificially preserved. It has The Book: "Barbershop Arranging Manual" (1980). Moreover, this is the culture driven by competition with clear rules of contest judging.

  • Can we build a transcription pipeline that yields four barbershop voices?
    • Can we fine-tuning existing splitters/transcribers?
    • Can we improve its accuracy by exploiting the knowledge of these rules?
  • How does The Manual and The Judging Handbook influence modern arrangers?
  • Is the "lock and ring" really visible on the spectrogram? Can we measure the amount of it? Is it absent on multitracks?
  • Do arrangers have their fingerprints by which we can identify them?
  • Can we measure the interestingness of the arrangement?

Arabic music

Traditional Arabic music is a heterophonic, with a single melody elaborated by several instruments. In 20th century a lot of composers created original pieces, including long songs. Instead of using just two scales, as Western music, an Arabic music uses many scales, or rather jins (4 or 5 pitch collections) and modulates between them, creating a narrative through these modulations, much like Western music creates one with chords.

  • Can we make an automatic transcription of a piece, exploiting this knowledge of expected structure?
  • How can we visualize the modulations in the most efficient way? What's the best piano roll for Arabic music?
  • Are networks of maqams not cliques? Are certain edges really absent? Why?
  • Sami Abu Shumays claims that maqam isn't just a tuning, it's a set of melodic patterns which is different between, say, rast and ajam (akin to radif). In cultures where maqamat gets 12-temperized (like rebetiko) and so maqams collapse, is there any way to historically show how maqams merge over generations of composers?

Also, tuning of quartertones is not fixed. Can we say anything about it using cent-accurate tuner algorithms?

Adhans

Adhan is the Islamic call to prayer recited from the mosque five times daily. It's either recorded of performed live by a trained muezzin. It's in a maqam tradition and can be described as a vocal improvisation in a certain scale. In Türkiye every time of a day is recited in a different makam.

What are the precise rules of this improvisation? What's allowed and what's not?

Persian music

Persian traditional radif improvisation is usually described as follows: a musician trained by a certain master glues motives from a radif together.

Given an audio performance, can we highlight the certain radif chunks in it?

Gamelan

Gamelan music is performed on an orchestra of xylophones and gongs. This culture is mostly about preservation of cultural heritage across centuries, and not about the creativeness of individual performers. The rich soundscape is created by many instruments rendering a rather simple melody by rigid rules.

As the rules are known, and multitracks are unavailable, can we build an unsupervised splitter of a mix into different tracks? Once this problem is solved, can we make automatic models for elaborating instruments like gender barung (in the style of Impro-Visor)?