Advanced Functional Programming in Industry

Jose Pedro Magalhaes

November 21, 2014

London, United Kingdom

Jos
e Pedro Magalh
aes

Advanced Functional Programming in Industry, FP Days 2014

1 / 46

Introduction

Haskell: a statically typed, lazy, purely functional language

Modelling musical harmony using Haskell

Applications of a model of harmony:

I
I
I
I
I

Jos
e Pedro Magalh
aes

Musical analysis
Finding cover songs
Generating chords and melodies
Correcting errors in chord extraction from audio sources
Chordifya web-based music player with chord recognition

Advanced Functional Programming in Industry, FP Days 2014

2 / 46

Demo: Chordify

Demo:

http://chordify.net

Jos
e Pedro Magalh
aes

Advanced Functional Programming in Industry, FP Days 2014

3 / 46

Table of Contents
Harmony
Haskell
Harmony analysis
Harmonic similarity
Music generation
Chord recognition: Chordify

Jos
e Pedro Magalh
aes

Advanced Functional Programming in Industry, FP Days 2014

4 / 46

What is harmony?
SDom

IV

Dom

Ton

Ton

V/V

D7

G7

Harmony arises when at least two notes sound at the same time

Harmony induces tension and release patterns, that can be described

by music theory and music cognition

The internal structure of the chord has a large influence on the

consonance or dissonance of a chord

The surrounding context also has a large influence

Jos
e Pedro Magalh
aes

Advanced Functional Programming in Industry, FP Days 2014

5 / 46

What is harmony?
SDom

IV

Dom

Ton

Ton

V/V

D7

G7

Harmony arises when at least two notes sound at the same time

Harmony induces tension and release patterns, that can be described

by music theory and music cognition

The internal structure of the chord has a large influence on the

consonance or dissonance of a chord

The surrounding context also has a large influence

Demo: how harmony affects melody

Jos
e Pedro Magalh
aes

Advanced Functional Programming in Industry, FP Days 2014

5 / 46

Simplified harmony theory I

A chord is a group of tones separated by intervals of roughly the

same size.

All music is made out of chords (whether explicitly or not).

There are 12 different notes. Instead of naming them, we number

them relative to the first and most important one, the tonic. So we
get I, II[, II . . . VI], VII.

A chord is built on a root note. So I also stands for the chord built
on the first degree, V for the chord built on the fifth degree, etc.
So the following is a chord sequence: I IV II7 V7 I.

Jos
e Pedro Magalh
aes

Advanced Functional Programming in Industry, FP Days 2014

6 / 46

Simplified harmony theory II

Models for musical harmony explain the harmonic progression in music:

I

Everything works around the tonic (I).

The dominant (V) leads to the tonic.

The subdominant (IV) tends to lead to the dominant.

Therefore, the I IV V I progression is very common.

There are also secondary dominants, which lead to a relative tonic.

For instance, II7 is the secondary dominant of V, and I7 is the
secondary dominant of IV.

So you can start with I, add one note to get I7 , fall into IV, change
two notes to get to II7 , fall into V, and then finally back to I.

Jos
e Pedro Magalh
aes

Advanced Functional Programming in Industry, FP Days 2014

7 / 46

An example harmonic analysis

SDom

IV

Dom

Ton

Ton

V/V

D7

G7

Piece
PT

PD

PT

IV

V/V

Jos
e Pedro Magalh
aes

II7

V7

D:7

G:7

Advanced Functional Programming in Industry, FP Days 2014

8 / 46

Why are harmony models useful?

Having a model for musical harmony allows us to automatically

determine the functional meaning of chords in the tonal context.
The model determines which chords fit on a particular moment in a
song.

Jos
e Pedro Magalh
aes

Advanced Functional Programming in Industry, FP Days 2014

9 / 46

Why are harmony models useful?

Having a model for musical harmony allows us to automatically

determine the functional meaning of chords in the tonal context.
The model determines which chords fit on a particular moment in a
song. This is useful for:
I

Musical information retrieval (find songs similar to a given song)

Audio and score recognition (improving recognition by knowing

which chords are more likely to appear)

Music generation (create sequences of chords that conform to the

model)

Jos
e Pedro Magalh
aes

Advanced Functional Programming in Industry, FP Days 2014

9 / 46

Table of Contents
Harmony
Haskell
Harmony analysis
Harmonic similarity
Music generation
Chord recognition: Chordify

Jos
e Pedro Magalh
aes

Advanced Functional Programming in Industry, FP Days 2014

10 / 46

Why Haskell?

Haskell is a strongly-typed pure functional programming language:

Strongly-typed All values are classified by their type, and types are
known at compile time (statically). This gives us strong
guarantees about our code, avoiding many common
mistakes.
Pure There are no side-effects, so Haskell functions are like
mathematical functions.
Functional A Haskell program is an expression, not a sequence of
statements. Functions are first class citizens, and explicit
state is avoided.

Jos
e Pedro Magalh
aes

Advanced Functional Programming in Industry, FP Days 2014

11 / 46

Notes

data Root = A | B | C | D | E | F | G
type Octave = Int
data Note = Note Root Octave

Jos
e Pedro Magalh
aes

Advanced Functional Programming in Industry, FP Days 2014

12 / 46

Notes

data Root = A | B | C | D | E | F | G
type Octave = Int
data Note = Note Root Octave
a4, b4, c4, d4, e4, f4, g4 :: Note
a4 = Note A 4
b4 = Note B 4
c4 = Note C 4
d4 = Note D 4
e4 = Note E 4
f4 = Note F 4
g4 = Note G 4

Jos
e Pedro Magalh
aes

Advanced Functional Programming in Industry, FP Days 2014

12 / 46

Melody
type Melody = [Note]
cMajScale :: Melody
cMajScale = [c4, d4, e4, f4, g4, a4, b4]

Jos
e Pedro Magalh
aes

Advanced Functional Programming in Industry, FP Days 2014

13 / 46

Melody
type Melody = [Note]
cMajScale :: Melody
cMajScale = [c4, d4, e4, f4, g4, a4, b4]
cMajScaleRev :: Melody
cMajScaleRev = reverse cMajScale

Jos
e Pedro Magalh
aes

Advanced Functional Programming in Industry, FP Days 2014

13 / 46

Melody
type Melody = [Note]
cMajScale :: Melody
cMajScale = [c4, d4, e4, f4, g4, a4, b4]
cMajScaleRev :: Melody
cMajScaleRev = reverse cMajScale
reverse :: [] []
reverse [ ]
= []
reverse (h : t) = reverse t +
+ [h]
(+
+) :: [] [] []
(+
+) = . . .

Jos
e Pedro Magalh
aes

Advanced Functional Programming in Industry, FP Days 2014

13 / 46

Transposition

Transposing a melody one octave higher:

octaveUp :: Octave Octave
octaveUp n = n + 1
noteOctaveUp :: Note Note
noteOctaveUp (Note r o) = Note r (octaveUp o)
melodyOctaveUp :: Melody Melody
melodyOctaveUp m = map noteOctaveUp m

Jos
e Pedro Magalh
aes

Advanced Functional Programming in Industry, FP Days 2014

14 / 46

Generation, analysis

Building a canon from a melody:

canon :: Melody Melody
canon m = m +
+ canon m

Jos
e Pedro Magalh
aes

Advanced Functional Programming in Industry, FP Days 2014

15 / 46

Generation, analysis

Building a canon from a melody:

canon :: Melody Melody
canon m = m +
+ canon m
Is a given melody in C major?
root :: Note Root
root (Note r o) = r
isCMaj :: Melody Bool
isCMaj = all ( cMajScale) map root

Jos
e Pedro Magalh
aes

Advanced Functional Programming in Industry, FP Days 2014

15 / 46

Details left out

We have seen only a glimpse of music representation in Haskell.
I

Rhythm

Accidentals

Intervals

Voicing

...

A good pedagogical reference on using Haskell to represent music:

http://di.uminho.pt/~jno/html/ipm-1011.html
A serious library for music manipulation:
http://www.haskell.org/haskellwiki/Haskore

Jos
e Pedro Magalh
aes

Advanced Functional Programming in Industry, FP Days 2014

16 / 46

Table of Contents
Harmony
Haskell
Harmony analysis
Harmonic similarity
Music generation
Chord recognition: Chordify

Jos
e Pedro Magalh
aes

Advanced Functional Programming in Industry, FP Days 2014

17 / 46

Application: harmony analysis

Parsing the sequence Gmin C7 Gmin C7 FMaj D7 G7 CMaj :
Piece
PD

PT

S
IV

V/IV
V/I

I7

Vmin

C:7

G:min

ins

S
V/IV
V/I

I7

Vmin

C:7

C:maj

IV

V/V

V7

F:maj

II7

G:7

D:7

G:min

Jos
e Pedro Magalh
aes

Advanced Functional Programming in Industry, FP Days 2014

18 / 46

Table of Contents
Harmony
Haskell
Harmony analysis
Harmonic similarity
Music generation
Chord recognition: Chordify

Jos
e Pedro Magalh
aes

Advanced Functional Programming in Industry, FP Days 2014

19 / 46

Application: harmonic similarity

A practical application of a harmony model is to estimate harmonic

similarity between songs

The more similar the trees, the more similar the harmony

We dont want to write a diff algorithm for our complicated model;

we get it automatically by using a generic diff

The generic diff is a type-safe tree-diff algorithm, part of a students

MSc work at Utrecht University

Generic, thus working for any model, and independent of changes to

the model

Jos
e Pedro Magalh
aes

Advanced Functional Programming in Industry, FP Days 2014

20 / 46

Table of Contents
Harmony
Haskell
Harmony analysis
Harmonic similarity
Music generation
Chord recognition: Chordify

Jos
e Pedro Magalh
aes

Advanced Functional Programming in Industry, FP Days 2014

21 / 46

Application: automatic harmonisation of melodies

Another practical application of a harmony model is to help selecting
good harmonisations (chord sequences) for a given melody:

# $ " "
%$ " "
"" ""

V III

" "
"" """
"

I III

" "

" "

"" ""
" "

"" ""
" "

II IV

III IV

!
!!!
V

We generate candidate chord sequences, parse them with the harmony

model, and select the one with the least errors.

Jos
e Pedro Magalh
aes

Advanced Functional Programming in Industry, FP Days 2014

22 / 46

Visualising harmonic structure

Piece
Phrase
Ton
I: Maj

Dom
Sub

Ton
Dom

I: Maj

C: Maj III: Min IV: Maj II: Dom7 V: Dom7 C: Maj

E: Min

F: Maj D: Dom7 G: Dom7

You can see this tree as having been produced by taking the chords in
green as input. . .

Jos
e Pedro Magalh
aes

Advanced Functional Programming in Industry, FP Days 2014

23 / 46

Generating harmonic structure

Piece
Phrase
Ton
I: Maj

Dom
Sub

Ton
Dom

I: Maj

C: Maj III: Min IV: Maj II: Dom7 V: Dom7 C: Maj

E: Min

F: Maj D: Dom7 G: Dom7

You can see this tree as having been produced by taking the chords in
green as input. . . or the chords might have been dictated by the structure!

Jos
e Pedro Magalh
aes

Advanced Functional Programming in Industry, FP Days 2014

23 / 46

A functional model of harmony

PieceM [PhraseM ]

Jos
e Pedro Magalh
aes

(M {Maj, Min})

Advanced Functional Programming in Industry, FP Days 2014

24 / 46

A functional model of harmony

PieceM [PhraseM ]

(M {Maj, Min})

PhraseM TonM DomM TonM

|
DomM TonM

Jos
e Pedro Magalh
aes

Advanced Functional Programming in Industry, FP Days 2014

24 / 46

A functional model of harmony

PieceM [PhraseM ]

(M {Maj, Min})

PhraseM TonM DomM TonM

|
DomM TonM
TonMaj IMaj
TonMin Im
Min

Jos
e Pedro Magalh
aes

Advanced Functional Programming in Industry, FP Days 2014

24 / 46

A functional model of harmony

PieceM [PhraseM ]

(M {Maj, Min})

PhraseM TonM DomM TonM

|
DomM TonM
TonMaj IMaj
TonMin Im
Min
7
DomM VM
| VM
| VII0M
| SubM DomM
7
| II7M VM

Jos
e Pedro Magalh
aes

Advanced Functional Programming in Industry, FP Days 2014

24 / 46

A functional model of harmony

PieceM [PhraseM ]

(M {Maj, Min})

PhraseM TonM DomM TonM

|
DomM TonM
TonMaj IMaj
TonMin Im
Min
7
VM

DomM
| VM
| VII0M
| SubM DomM
7
| II7M VM

Jos
e Pedro Magalh
aes

SubMaj IIm
Maj
| IVMaj
| IIIm
Maj IVMaj
SubMin IVm
Min

Advanced Functional Programming in Industry, FP Days 2014

24 / 46

A functional model of harmony

PieceM [PhraseM ]

(M {Maj, Min})

PhraseM TonM DomM TonM

|
DomM TonM
TonMaj IMaj
TonMin Im
Min
7
VM

DomM
| VM
| VII0M
| SubM DomM
7
| II7M VM

SubMaj IIm
Maj
| IVMaj
| IIIm
Maj IVMaj
SubMin IVm
Min

Simple, but enough for now, and easy to extend.

Jos
e Pedro Magalh
aes

Advanced Functional Programming in Industry, FP Days 2014

24 / 46

Now in HaskellI
A naive datatype encoding musical harmony:
data Piece = Piece [ Phrase ]
data Phrase where
PhraseIVI :: Ton Dom Ton Phrase
PhraseVI ::
Dom Ton Phrase

Jos
e Pedro Magalh
aes

Advanced Functional Programming in Industry, FP Days 2014

25 / 46

Now in HaskellI
A naive datatype encoding musical harmony:
data Piece = Piece [ Phrase ]
data Phrase where
PhraseIVI :: Ton Dom Ton Phrase
PhraseVI ::
Dom Ton Phrase
data Ton where
TonMaj :: Degree Ton
TonMin :: Degree Ton

Jos
e Pedro Magalh
aes

Advanced Functional Programming in Industry, FP Days 2014

25 / 46

Now in HaskellI
A naive datatype encoding musical harmony:
data Piece = Piece [ Phrase ]
data Phrase where
PhraseIVI :: Ton Dom Ton Phrase
PhraseVI ::
Dom Ton Phrase
data Ton where
TonMaj :: Degree Ton
TonMin :: Degree Ton
data Dom where
DomV7 :: Degree Dom
DomV
:: Degree Dom
DomVII0 :: Degree Dom
DomIVV :: SDom Dom Dom
DomIIV :: Degree Degree Dom

Jos
e Pedro Magalh
aes

Advanced Functional Programming in Industry, FP Days 2014

25 / 46

Now in HaskellI
A naive datatype encoding musical harmony:
data Piece = Piece [ Phrase ]
data Phrase where
PhraseIVI :: Ton Dom Ton Phrase
PhraseVI ::
Dom Ton Phrase
data Ton where
TonMaj :: Degree Ton
TonMin :: Degree Ton
data Dom where
DomV7 :: Degree Dom
DomV
:: Degree Dom
DomVII0 :: Degree Dom
DomIVV :: SDom Dom Dom
DomIIV :: Degree Degree Dom
data Degree = I | II | III . . .

Jos
e Pedro Magalh
aes

Advanced Functional Programming in Industry, FP Days 2014

25 / 46

Now in HaskellII
A GADT encoding musical harmony:
data Mode = MajMode | MinMode
data Piece ( :: Mode) where
Piece :: [ Phrase ] Piece

Jos
e Pedro Magalh
aes

Advanced Functional Programming in Industry, FP Days 2014

26 / 46

Now in HaskellII
A GADT encoding musical harmony:
data Mode = MajMode | MinMode
data Piece ( :: Mode) where
Piece :: [ Phrase ] Piece
data Phrase ( :: Mode) where
PhraseIVI :: Ton Dom Ton Phrase
PhraseVI ::
Dom Ton Phrase

Jos
e Pedro Magalh
aes

Advanced Functional Programming in Industry, FP Days 2014

26 / 46

Now in HaskellII
A GADT encoding musical harmony:
data Mode = MajMode | MinMode
data Piece ( :: Mode) where
Piece :: [ Phrase ] Piece
data Phrase ( :: Mode) where
PhraseIVI :: Ton Dom Ton Phrase
PhraseVI ::
Dom Ton Phrase
data Ton ( :: Mode) where
TonMaj :: SD I Maj Ton MajMode
TonMin :: SD I Min Ton MinMode

Jos
e Pedro Magalh
aes

Advanced Functional Programming in Industry, FP Days 2014

26 / 46

Now in HaskellII
A GADT encoding musical harmony:
data Mode = MajMode | MinMode
data Piece ( :: Mode) where
Piece :: [ Phrase ] Piece
data Phrase ( :: Mode) where
PhraseIVI :: Ton Dom Ton Phrase
PhraseVI ::
Dom Ton Phrase
data Ton ( :: Mode) where
TonMaj :: SD I Maj Ton MajMode
TonMin :: SD I Min Ton MinMode
data Dom ( :: Mode) where
DomV7 :: SD V Dom7 Dom
DomV
:: SD V Maj Dom
DomVII0 :: SD VII Dim Dom
DomIVV :: SDom Dom Dom
DomIIV :: SD II Dom7 SD V Dom7 Dom
Jos
e Pedro Magalh
aes

Advanced Functional Programming in Industry, FP Days 2014

26 / 46

Now in HaskellIII

Scale degrees are the leaves of our hierarchical structure:

data DiatonicDegree = I | II | III | IV | V | VI | VII
data Quality
= Maj | Min | Dom7 | Dim
data SD ( :: DiatonicDegree) ( :: Quality) where
SurfaceChord :: ChordDegree SD

Jos
e Pedro Magalh
aes

Advanced Functional Programming in Industry, FP Days 2014

27 / 46

Now in HaskellIII

Scale degrees are the leaves of our hierarchical structure:

data DiatonicDegree = I | II | III | IV | V | VI | VII
data Quality
= Maj | Min | Dom7 | Dim
data SD ( :: DiatonicDegree) ( :: Quality) where
SurfaceChord :: ChordDegree SD

Now everything is properly indexed, and our GADT is effectively

constrained to allow only harmonically valid sequences!

Jos
e Pedro Magalh
aes

Advanced Functional Programming in Industry, FP Days 2014

27 / 46

Generating harmony

Now that we have a datatype representing harmony sequences, how do

we generate a sequence of chords?

Jos
e Pedro Magalh
aes

Advanced Functional Programming in Industry, FP Days 2014

28 / 46

Generating harmony

Now that we have a datatype representing harmony sequences, how do

we generate a sequence of chords?
QuickCheck! We give Arbitrary instances for each of the datatypes in our
model.

Jos
e Pedro Magalh
aes

Advanced Functional Programming in Industry, FP Days 2014

28 / 46

Generating harmony

Now that we have a datatype representing harmony sequences, how do

we generate a sequence of chords?
QuickCheck! We give Arbitrary instances for each of the datatypes in our
model.
. . . but we dont want to do this by hand, for every datatype, and to have
to adapt the instances every time we change the model. . . so we use
generic programming:

Jos
e Pedro Magalh
aes

Advanced Functional Programming in Industry, FP Days 2014

28 / 46

Generating harmony
Now that we have a datatype representing harmony sequences, how do
we generate a sequence of chords?
QuickCheck! We give Arbitrary instances for each of the datatypes in our
model.
. . . but we dont want to do this by hand, for every datatype, and to have
to adapt the instances every time we change the model. . . so we use
generic programming:
gen :: .(Representable , Generate (Rep ))
Gen

Jos
e Pedro Magalh
aes

Advanced Functional Programming in Industry, FP Days 2014

28 / 46

Generating harmony
Now that we have a datatype representing harmony sequences, how do
we generate a sequence of chords?
QuickCheck! We give Arbitrary instances for each of the datatypes in our
model.
. . . but we dont want to do this by hand, for every datatype, and to have
to adapt the instances every time we change the model. . . so we use
generic programming:
gen :: .(Representable , Generate (Rep ))
[ (String,Int) ] Gen

Jos
e Pedro Magalh
aes

Advanced Functional Programming in Industry, FP Days 2014

28 / 46

Examples of harmony generation

testGen :: Gen (Phrase MajMode )
testGen = gen [("Dom_IV-V", 3), ("Dom_II-V", 4)]
example :: IO ()
example = let k = Key (Note \ C) MajMode
in sample0 testGen >
>= mapM (printOnKey k)

Jos
e Pedro Magalh
aes

Advanced Functional Programming in Industry, FP Days 2014

29 / 46

Examples of harmony generation

testGen :: Gen (Phrase MajMode )
testGen = gen [("Dom_IV-V", 3), ("Dom_II-V", 4)]
example :: IO ()
example = let k = Key (Note \ C) MajMode
in sample0 testGen >
>= mapM (printOnKey k)

> example
[C: Maj, D: Dom7 , G: Dom7 , C: Maj]
[C: Maj, G: Dom7 , C: Maj]
[C: Maj, E: Min, F: Maj, G: Maj, C: Maj]
[C: Maj, E: Min, F: Maj, D: Dom7 , G: Dom7 , C: Maj]
[C: Maj, D: Min, E: Min, F: Maj, D: Dom7 , G: Dom7 , C: Maj]

Jos
e Pedro Magalh
aes

Advanced Functional Programming in Industry, FP Days 2014

29 / 46

Generating a melody for a given harmony

We then generate a melody in 4 steps:
1. Generate a list of candidate melody notes per chord;
2. Refine the candidates by filtering out obviously bad candidates;
3. Pick one focal candidate melody note per chord;
4. Embellish the candidate notes to produce a final melody.

Jos
e Pedro Magalh
aes

Advanced Functional Programming in Industry, FP Days 2014

30 / 46

Generating a melody for a given harmony

We then generate a melody in 4 steps:
1. Generate a list of candidate melody notes per chord;
2. Refine the candidates by filtering out obviously bad candidates;
3. Pick one focal candidate melody note per chord;
4. Embellish the candidate notes to produce a final melody.
These four steps combine naturally using plain monadic bind:
melody :: Key State MyState Song
melody k = genCandidates >
>= refine >
>= pickOne >
>= embellish
>
>= return Song k

Jos
e Pedro Magalh
aes

Advanced Functional Programming in Industry, FP Days 2014

30 / 46

Example I

&
n

?
#

Phrase
Ton
I: Maj

Ton

Dom
Sub

Dom

I: Maj

C: Maj III: Min IV: Maj II: Dom7 V: Dom7 C: Maj

E: Min

Jos
e Pedro Magalh
aes

F: Maj D: Dom7 G: Dom7

Advanced Functional Programming in Industry, FP Days 2014

31 / 46

Example II
&

? #

##

#n

Phrase
Ton
I: Min

Dom
Sub

Ton
Dom

E: Min IV: Min

Sub

I: Min
Dom

E: Min

A: Min IV: Min II: Dom7 V: Dom7

A: Min F] : Dom7 B: Dom7

Jos
e Pedro Magalh
aes

Advanced Functional Programming in Industry, FP Days 2014

32 / 46

Table of Contents
Harmony
Haskell
Harmony analysis
Harmonic similarity
Music generation
Chord recognition: Chordify

Jos
e Pedro Magalh
aes

Advanced Functional Programming in Industry, FP Days 2014

33 / 46

Back to Chordify: chord recognition

Yet another practical application of a harmony model is to improve chord

recognition from audio sources.
Chord candidates
Beat number

1.00 C
1

0.92 C
0.94 Gm
1.00 G
2

0.96 Em
0.97 C
1.00 Em
3

How to pick the right chord from the chord candidate list? Ask the
harmony model which one fits best.

Jos
e Pedro Magalh
aes

Advanced Functional Programming in Industry, FP Days 2014

34 / 46

Chordify: architecture

Frontend
I
I
I
I
I
I

Jos
e Pedro Magalh
aes

Reads user input, such as YouTube/Soundcloud/Deezer links, or files

Extracts audio
Calls the backend to obtain the chords for the audio
Displays the result to the user
Implements a queueing system, and library functionality
Uses PHP, JavaScript, MongoDB

Advanced Functional Programming in Industry, FP Days 2014

35 / 46

Chordify: architecture

Frontend
I
I
I
I
I
I

Reads user input, such as YouTube/Soundcloud/Deezer links, or files

Extracts audio
Calls the backend to obtain the chords for the audio
Displays the result to the user
Implements a queueing system, and library functionality
Uses PHP, JavaScript, MongoDB

Backend
I
I

I
I

Jos
e Pedro Magalh
aes

Takes an audio file as input, analyses it, extracts the chords

The chord extraction code uses GADTs, type families, generic
programming (see the HarmTrace package on Hackage)
Performs PDF and MIDI export (using LilyPond)
Uses Haskell, SoX, sonic annotator, and is mostly open source

Advanced Functional Programming in Industry, FP Days 2014

35 / 46

Chordify: numbers

Online since January 2013

Top countries: US, UK, Germany, Indonesia, Canada

Views: 3M+ (monthly)

Chordified songs: 1.5M+

Registered users: 200K

Jos
e Pedro Magalh
aes

Advanced Functional Programming in Industry, FP Days 2014

36 / 46

How do we handle these visitors?

Single VPS, 6 Intel Xeon cores, 24GB RAM, 500GB SSD, 2TB hard
drive

Jos
e Pedro Magalh
aes

Advanced Functional Programming in Industry, FP Days 2014

37 / 46

How do we handle these visitors?

Single VPS, 6 Intel Xeon cores, 24GB RAM, 500GB SSD, 2TB hard
drive

Single server hosts both the web and database servers

Jos
e Pedro Magalh
aes

Advanced Functional Programming in Industry, FP Days 2014

37 / 46

How do we handle these visitors?

Single VPS, 6 Intel Xeon cores, 24GB RAM, 500GB SSD, 2TB hard
drive

Single server hosts both the web and database servers

Can easily handle peaks of (at least) 700 visitors at a time

Jos
e Pedro Magalh
aes

Advanced Functional Programming in Industry, FP Days 2014

37 / 46

How do we handle these visitors?

Single VPS, 6 Intel Xeon cores, 24GB RAM, 500GB SSD, 2TB hard
drive

Single server hosts both the web and database servers

Can easily handle peaks of (at least) 700 visitors at a time

I
I

Chordifying new songs takes some computing power, but most songs
are in the database already

Jos
e Pedro Magalh
aes

Advanced Functional Programming in Industry, FP Days 2014

37 / 46

How do we handle these visitors?

Single VPS, 6 Intel Xeon cores, 24GB RAM, 500GB SSD, 2TB hard
drive

Single server hosts both the web and database servers

Can easily handle peaks of (at least) 700 visitors at a time

I
I

Chordifying new songs takes some computing power, but most songs
are in the database already

Queueing system for busy periods

Jos
e Pedro Magalh
aes

Advanced Functional Programming in Industry, FP Days 2014

37 / 46

How do we handle these visitors?

Single VPS, 6 Intel Xeon cores, 24GB RAM, 500GB SSD, 2TB hard
drive

Single server hosts both the web and database servers

Can easily handle peaks of (at least) 700 visitors at a time

I
I

Chordifying new songs takes some computing power, but most songs
are in the database already

Queueing system for busy periods

Infrastructure costs are minimal

Jos
e Pedro Magalh
aes

Advanced Functional Programming in Industry, FP Days 2014

37 / 46

Frontend (PHP/JS) and backend (Haskell) interaction

Frontend receives a music file, calls backend with it

Jos
e Pedro Magalh
aes

Advanced Functional Programming in Industry, FP Days 2014

38 / 46

Frontend (PHP/JS) and backend (Haskell) interaction

I
I

Frontend receives a music file, calls backend with it

Backend computes the chords, writes them to a file:

Jos
e Pedro Magalh
aes

Advanced Functional Programming in Industry, FP Days 2014

38 / 46

Frontend (PHP/JS) and backend (Haskell) interaction

I
I

Frontend receives a music file, calls backend with it

Backend computes the chords, writes them to a file:
I

Jos
e Pedro Magalh
aes

1;D:min;0.232199546;0.615328798
2;D:min;0.615328798;0.998458049
...

Advanced Functional Programming in Industry, FP Days 2014

38 / 46

Frontend (PHP/JS) and backend (Haskell) interaction

I
I

Frontend receives a music file, calls backend with it

Backend computes the chords, writes them to a file:
I

1;D:min;0.232199546;0.615328798
2;D:min;0.615328798;0.998458049
...

Frontend reads this file, updates the database if necessary, and

renders the result

Jos
e Pedro Magalh
aes

Advanced Functional Programming in Industry, FP Days 2014

38 / 46

Frontend (PHP/JS) and backend (Haskell) interaction

I
I

Frontend receives a music file, calls backend with it

Backend computes the chords, writes them to a file:
I

1;D:min;0.232199546;0.615328798
2;D:min;0.615328798;0.998458049
...

Frontend reads this file, updates the database if necessary, and

renders the result

Backend is open-source (and GPL3); only option is to run it as a

standalone executable

Jos
e Pedro Magalh
aes

Advanced Functional Programming in Industry, FP Days 2014

38 / 46

The importance of the UI

Lets have a look at four different online services giving you the chords
for a song (Radioheads Karma Police).

Jos
e Pedro Magalh
aes

Advanced Functional Programming in Industry, FP Days 2014

39 / 46

The importance of the UIChordie

Jos
e Pedro Magalh
aes

Advanced Functional Programming in Industry, FP Days 2014

40 / 46

The importance of the UIRiffstation

Jos
e Pedro Magalh
aes

Advanced Functional Programming in Industry, FP Days 2014

41 / 46

The importance of the UIYoutab.me

Jos
e Pedro Magalh
aes

Advanced Functional Programming in Industry, FP Days 2014

42 / 46

The importance of the UIChordify

Jos
e Pedro Magalh
aes

Advanced Functional Programming in Industry, FP Days 2014

43 / 46

Logistics of an internet start-up

Chordify is created and funded by 5 people

Jos
e Pedro Magalh
aes

Advanced Functional Programming in Industry, FP Days 2014

44 / 46

Logistics of an internet start-up

Chordify is created and funded by 5 people

If you can do without venture capital, do it!

Jos
e Pedro Magalh
aes

Advanced Functional Programming in Industry, FP Days 2014

44 / 46

Logistics of an internet start-up

Chordify is created and funded by 5 people

If you can do without venture capital, do it!

You might end up doing more than just functional programming,
though:

I
I

I
I

Jos
e Pedro Magalh
aes

Deciding on what features to implement next

Recruiting, interviewing, dealing with legal issues related to
employment
Taxation (complicated by the fact that we sell worldwide and support
multiple currencies)
User support
Outreach (pitching events, media, this talk, etc.)

Advanced Functional Programming in Industry, FP Days 2014

44 / 46

Logistics of an internet start-up

Chordify is created and funded by 5 people

If you can do without venture capital, do it!

You might end up doing more than just functional programming,
though:

I
I

I
I

Deciding on what features to implement next

Recruiting, interviewing, dealing with legal issues related to
employment
Taxation (complicated by the fact that we sell worldwide and support
multiple currencies)
User support
Outreach (pitching events, media, this talk, etc.)

But its fun, and you learn a lot!

Jos
e Pedro Magalh
aes

Advanced Functional Programming in Industry, FP Days 2014

44 / 46

Summary

Musical modelling with Haskell:

I

A model for musical harmony as a Haskell datatype

Makes use of several advanced functional programming techniques,

such as generic programming, GADTs, and type families

When chords do not fit the model: error correction

Harmonising melodies

Generating harmonies

Recognising harmony from audio sources

Transporting academic research into industry

Jos
e Pedro Magalh
aes

Advanced Functional Programming in Industry, FP Days 2014

45 / 46

Play with it!

http://chordify.net
http://hackage.haskell.org/package/HarmTrace
http://hackage.haskell.org/package/FComp

Jos
e Pedro Magalh
aes

Advanced Functional Programming in Industry, FP Days 2014

46 / 46