{"@attributes":{"version":"2.0"},"channel":{"title":"TomAF","link":"https:\/\/afiodorov.github.io","description":"Laugh with me, smile with me, disagree with me","language":"en-us","pubDate":"Thu, 17 Apr 2025 10:34:47 +0000","lastBuildDate":"Thu, 17 Apr 2025 10:34:47 +0000","item":[{"title":"The Age of Personalized Software: How LLMs Are Changing Programming","link":"https:\/\/afiodorov.github.io\/2025\/04\/17\/personalized-software\/","pubDate":"Thu, 17 Apr 2025 00:00:00 +0000","author":"Tom A. Fiodorov","description":"

Today, I want to talk about the role of LLMs in the software industry.<\/p>\n\n

First, let me start by saying that the magic of LLMs is real; they really do help with writing all kinds of software and with software-related tasks.<\/p>\n\n

But I\u2019m not here to predict the end of programming. The other day, I thought about using generative AI to write some music for me. See, as a child, I wrote a couple of songs whose lyrics I quite liked, but I was never an advanced or diligent enough musician to turn them into professional-sounding pieces. The songs are still around, featuring my voice and a guitar, but they feel more like drafts \u2013 something you\u2019d show a fellow musician before considering it done.<\/p>\n\n

So, the other day, I found this guy on YouTube who was showing how to use various pieces of software to turn lyrics into music, complete with AI singing voices and musical instruments. I was amazed, but once I started paying closer attention to what he was doing, it became increasingly obvious that he\u2019s very knowledgeable about music production. In other words, he knows how to do all of it without AI. Sure, I could probably create something too, but that \u2018something\u2019 would either be far below today\u2019s musical standards or take me an incredible amount of time to produce.<\/p>\n\n

Where was I\u2026 ah yes, software. So, sure, now anyone can feel like a programmer, just like YouTube tutorials make us all feel like electricians, car mechanics, or doctors. And now, with LLMs, we\u2019re all lawyers and accountants too\u2026 right?<\/p>\n\n

Okay, I can fix a plug, tighten a screw, and change the oil in my car. And now, anyone can make a simple snake game using AI. But have you seen the state of professional software? Making money from apps requires hiring graphic designers, UX experts, and who knows what else.<\/p>\n\n

Because people, on average, aren\u2019t going to part with their cash just to use a basic app made by one developer.<\/p>\n\n

But where does this leave us programmers? Well, I still love LLMs because I get to write personalized apps. And I think that\u2019s where things are heading. It\u2019s not just your grandmother getting a custom app from her grandchildren; it\u2019s also your company\u2019s accountant getting a personal app or a plug-in to make their spreadsheet work easier.<\/p>\n\n

The value of each individual app made by this kind of \u201cvibe coding\u201d might be low, but collectively, they add up. Think of the scale \u2013 maybe it aggregates to something significant, perhaps illustrating the kind of value we see in OpenAI\u2019s valuation, just through sheer volume.<\/p>\n\n

So, to conclude, is this the end of programming or the start of something new? I\u2019d say neither.<\/p>\n"},{"title":"Vibe-blogging: Why I Prefer Bottom-Up Programming","link":"https:\/\/afiodorov.github.io\/2025\/04\/15\/vibe-blogging\/","pubDate":"Tue, 15 Apr 2025 00:00:00 +0000","author":"Tom A. Fiodorov","description":"

I\u2019ve been wanting to get back into technical writing, and something has been bouncing around in my head lately: the perennial discussion of top-down versus bottom-up programming approaches.<\/p>\n\n

Let me define my terms, at least for this discussion. I see the top-down approach as emphasizing upfront planning, designing systems based on anticipated user needs and abstract structures. Conversely, I view the bottom-up approach as more discovery-driven \u2013 figuring out the requirements and the best way to implement them during the process, often starting with concrete pieces and building upwards.<\/p>\n\n

I want to be clear: I firmly believe both approaches have their place and are often needed in tandem for large, successful projects. However, personally? I find myself consistently drawn to, and simply enjoying, the bottom-up process much more.<\/p>\n\n

My journey with this preference started early. When I began coding, the landscape felt dominated by a choice: dive into compiled languages or embrace dynamic ones like Python (or JavaScript, pre-TypeScript). Apps were taking off, but jobs often fell into distinct \u201cbackend\u201d or \u201cfrontend\u201d buckets. This meant choices like Java\/C#\/C++ on one side, and PHP or JavaScript on the other. Python felt like something else entirely back then \u2013 often described as \u201cglue code,\u201d useful for scripting, but not quite a heavyweight contender in either the pure backend or frontend space.<\/p>\n\n

Frankly, I disliked Java intensely. To me, it felt like the epitome of the top-down philosophy. While the approach itself isn\u2019t inherently flawed, my perception was that the Java ecosystem often went overboard with abstractions, boilerplate, and scaffolding. It felt heavy and prescriptive. Python, for me, always represented the opposite end of that spectrum \u2013 lean, flexible, and immediate.<\/p>\n\n

Looking back, what kept pulling me towards Python? Time and again, it was its power in specific, practical contexts.<\/p>\n\n

My first real Python \u201caha!\u201d moment came during mathematical research involving computer simulations. I initially started with C++, chasing raw performance \u2013 a common choice when speed is paramount and you think you know exactly what needs to be computed. But I wasn\u2019t experienced enough to make C++ truly sing quickly. Slowly, I discovered NumPy. The revelation wasn\u2019t just about avoiding C++ compilation times; it was about the speed of experimentation. NumPy allowed me to rapidly test different hypotheses, tweak algorithms, and visualize results. The feedback loop was drastically shorter. Eventually, most of the simulation logic migrated to Python\/NumPy, and I rarely needed to touch the C++ code again. This was bottom-up discovery in action \u2013 finding the path through iteration.<\/p>\n\n

The second Python \u201ctrap\u201d I happily fell into was Pandas. It\u2019s worth appreciating Pandas\u2019 origin: Wes McKinney, working in finance, needed better tools for data analysis than Excel or existing libraries offered. He wasn\u2019t necessarily setting out to build a grand, perfectly designed system; he was solving a real, pressing problem for himself. And boy, did that library take off. For me, Pandas is a quintessential example of bottom-up success. It became, and largely remains, the lingua franca of data analysis.<\/p>\n\n

Now, the \u201ctop-down\u201d perspective often criticizes Pandas for its sometimes inconsistent API or its historical quirks (like the way it handled missing values via np.nan, a legacy of being built on top of NumPy). These criticisms have merit. Reading complex, heavily overloaded Pandas code can be challenging. And newer libraries like Polars have emerged, learning from Pandas\u2019 history to offer a cleaner, potentially more consistent API designed from the ground up. I understand the appeal, though habit keeps me reaching for Pandas often.<\/p>\n\n

But this slight digression highlights the core point. Pandas exploded in popularity because it solved immediate, tangible problems effectively, even if its initial form wasn\u2019t perfectly architected. Its \u201cmessiness\u201d is arguably a side effect of its organic, bottom-up growth. The succinctness, despite the quirks, was a massive advantage when you were still discovering what analysis needed to be done.<\/p>\n\n

This leads me to think about programming more broadly. For me, programming is fundamentally an engineering discipline. A programmer is a tool-wielder. If the best way to solve the problem is writing clean, well-structured code following a grand design, great. If it involves wrestling with a library\u2019s API or finding clever workarounds to get the job done now, that\u2019s also part of the game. The focus is on solving the problem.<\/p>\n\n

Think about other foundational pieces of software. Linus Torvalds wrote Git initially to solve his specific, pressing problem of managing the Linux kernel\u2019s source code. Satoshi Nakamoto created Bitcoin to address a specific set of ideas about decentralization and currency (whatever the exact motivation, it wasn\u2019t purely an exercise in abstract system design).<\/p>\n\n

These weren\u2019t born from committees planning abstract interfaces; they were born from individuals or small groups tackling concrete challenges. A programmer takes a problem and solves it using the tools available, including writing code. The bottom-up approach, the journey of discovery, feels incredibly aligned with this pragmatic, problem-solving core of engineering. While top-down planning provides necessary structure, especially at scale, the initial spark often comes from that bottom-up need.<\/p>\n"},{"title":"Releasing LiveTranslate - automatic simultaneous translation","link":"https:\/\/afiodorov.github.io\/2023\/11\/26\/announcing-livetranslate\/","pubDate":"Sun, 26 Nov 2023 00:00:00 +0000","author":"Tom A. Fiodorov","description":"

I am tired of Americanism: consuming in the Anglosphere sometimes makes me feel\nlike all movies are the same. America is a great country for allowing a massive\nquantity of various productions to flourish however due to commercialisation\npressures some creativity is stiffled.<\/p>\n\n

Growing up I enjoyed some Soviet cinema. Most of it was boring however some\nfilms I loved and kept rewatching. However these days I can\u2019t find Russian\nstuff with dubs\/subtitles - so I am forced to watch this stuff alone. And\nsometimes I want to share the experience. This led me to consider the\npossibilities of leveraging large language models (LLMs) to facilitate\ntranslation for such content.<\/p>\n\n

Current transcription technologies like DeepGram offer decent performance, but\nthere\u2019s an evident bias towards Western languages. When it comes to Russian and\nPolish, for instance, inaccuracies are more common. Those inaccuracies could\neasily be fixed by an LLM post-processing the result: I experimented with using\nan LLM to improve the quality of these transcriptions, but real-time processing\nis still beyond our reach due to the response times of models like\nGPT-3.5-turbo (let alone GPT-4).<\/p>\n\n

When it comes to the translation, again LLMs are just not fast enough to\ntranslate real-time, so I am sticking with Google Translate. And surely enough\nI can just preprocessing most content and translate thing myself - but that\u2019s\nmuch more effort than just putting something on.<\/p>\n\n

In the meantime, I\u2019ve found that automatic translation can be quite useful for\nthose who have a basic grasp of the source language. It serves as a helpful\ntool for understanding and learning, similar to how one might pick up slang\nfrom context within a movie. This has proven especially true for me with Polish\ncontent - all of a sudden I found it understandable (a Russian speaker\ntypically doesn\u2019t understand Polish but can read a lot of it without any\nlearning).<\/p>\n\n

Given the current limitations of real-time translation, I\u2019ve decided to release\nmy project, LiveTranslate<\/a>, on GitHub. It\u2019s a practical solution for now,\nparticularly for programmers who might be interested in contributing to its\ndevelopment. The project has potential for further enhancements, especially in\nthe areas of user interface and increased accuracy.<\/p>\n\n

The repository is now available for access and contribution. This release is\nfor those who share an interest in making non-English content more accessible\nand are looking for a way to integrate programming skills with language\ntranslation. The hope is that as LLMs improve in latency, so too will the\ncapabilities of LiveTranslate.<\/p>\n\n

Demo<\/h2>\n\n

\"Demo<\/p>\n\n"},{"title":"Announcing Thoughtful3","link":"https:\/\/afiodorov.github.io\/2022\/12\/29\/announcing-thoughtful3\/","pubDate":"Thu, 29 Dec 2022 00:00:00 +0000","author":"Tom A. Fiodorov","description":"

I spent last 6 days creating Thoughtful3<\/a>.\nIt\u2019s basically Twitter but it is hosted on Ethereum blockchain and thus is very\nexpensive. Like $8 per Tweet that you have to pay the crypto miners to embed\nyour thoughts into the blockchain. I think it\u2019s a feature actually: with such\nhigh prices no problem with spam accounts.<\/p>\n\n

Why? I like the idea of embedding your thoughts into Ethereum forever. Whatever\nI post there is likely to outlive me. Also this is truly censorship resistant.\nSo if you need to complain about evil governments: go ahead (but it\u2019ll cost\nyou $8 per 300 characters). Anyway I\u2019ve always wanted to make a Web3 app, i.e.\na decentralised app. This is my first (Web3 and Thoughtful is the reason for the\nname). Underneath I am using thegraph<\/a>. I used vyper for the\nsmartcontract & pure typescript (no libraries like React) for the front end.\nCode of the project can be found here<\/a>.<\/p>\n\n

I didn\u2019t know much about Metamask (extension that allows you to interact with\nEthereum network) nor modern JavaScript 6 days ago and heavily used ChatGPT3 to\ngenerate boilerplate code for me & used DALL-E to generate a logo.<\/p>\n\n"},{"title":"I am in Lisbon","link":"https:\/\/afiodorov.github.io\/2022\/07\/26\/i-am-in-lisbon\/","pubDate":"Tue, 26 Jul 2022 00:00:00 +0000","author":"Tom A. Fiodorov","description":"

I am on my summer break and it didn\u2019t take me long to get to Lisbon, Portugal.\nI flew from Canary Islands and did not know what to expect. I\u2019ve only been here\n3.5 days but slowly I am forming a picture.<\/p>\n\n

Approaching Lisbon I saw red-bricked roofs of the houses and many hills and it\ninstantly reminded me of Vilnius, Lithuania. However right at the airport\nwaiting for the taxi pick-up I noticed multi ethnic groups waiting and so my\nimpression changed towards either a) yet another Western European city b)\npost-colonial capital. Of course it is a bit of both, but I am currently\nthinking it\u2019s more b) than a).<\/p>\n\n

\"roofs\"<\/p>\n\n

Greeted by the transit person who was responsible for getting me to the hotel\n(most hotel bookings here include a taxi transfer), I found out she\u2019s Brazilian\nand so is the taxi driver. Took her a few minutes of knowing me to complain about\nthe Portuguese: she said she moved from Porto to Lisbon because Portuguese\nthere \u201cdon\u2019t like Brazilians much\u201d. Woaaa. Felt like getting to know a group of\ngood friends who instantly started gossiping about one another.<\/p>\n\n

Checking in at the hotel the receptionist spoke some Spanish to me for about 1\nminute and then he just switched to English without an explanation. The other\nstaff stood by timidly and didn\u2019t say much after the initial \u201chola\u201d. From\nfuture interactions with the locals I am getting the impression that Portuguese\nare much more reserved and timid compared to southern Spanish. Why am I comparing\nthem to Spanish anyway?<\/p>\n\n

\"bridge\"<\/p>\n\n

Before coming here I used to think of Portugal as some kind of Spanish little\nbrother. And it is not - as simple as that. The easiest way to offend everyone\nin this country is to say \u2018gracias\u2019 instead of \u2018obligado\u2019. Portugal has a\ndistinct national identity. Apart from a brief period of the union with\nCastilla they remained largely independent. Even the union was never a conquest\nby the Spanish - it happened because of some blood ties of the royal families.<\/p>\n\n

The similarities I see are more to do with the climate: dinners are very\nlate, people never seem to be in a rush. Some ingredients and dishes are in\ncommon: like the octopus. Although that\u2019s probably because of the shared\ncuisine with their Northern neighbour: Galicia. The languages don\u2019t sound the\nsame at all. In fact many people say the Portuguese phonetically resemble some\nkind of Slavic language. And to my ear this is completely true. The first day I\nkept wondering why everyone is speaking Polish around here. Reasons for the\nsimilarity are coincidental: Slavic & Portuguese are stress-timed languages and\nfeature many sounds like sh\/zh and other constant clusters that make them\nsounds somewhat similar especially if are not paying much attention.<\/p>\n\n

However, there is one similarity in (slang) Spanish and Portuguese languages:\nthe frequency of the AO diphthong. I always wondered why Maluma sings \u2018ya que\nestoy tomA\u2019O\u2019 even though he\u2019s Colombian. I currently suspect this A\u2019O thing is\na Brazilian influence on the Latin Spanish to make it sound cool. Coincidently\nsame A\u2019O diphthong can be heard in the Galician accent, language most similar\nto Portuguese. Satisfying when all pieces fit.<\/p>\n\n

I digressed. Lisbon itself turned out be smaller in the population size than I\nthought it would be. Hosting just 500 thousand inhabitants - it is only 30%\nbigger than the city where I reside, Las Palmas. Yet Las Palmas feels like a\nprovince whilst Lisbon, or should I say Lisboa, is definitely a capital. The\ndensity of beautiful buildings in the city shows that it has a wealthy history;\nit also has a metro system. I guess 500,000 figure is deceptive, as their\nmetropolitan population is much much bigger than Las Palmas. Whilst I enjoy the\nbeautiful buildings that make up the city center, due to the hot climate at the\nmoment my sight-seeing abilities are limited. What I absolutely love about the\ncity is the abundance of various parks and squares for people to hide from the\nscorching sun. Here, in the shade, I spend a lot of time these days reading my\ne-reader and thinking about what to write.<\/p>\n\n

\"belem\"<\/p>\n\n

I am staying here at a very modest hotel, especially given its price. There\u2019s\nno swimming pool, my room is tiny, nor do I agree with their liberal use\nof the word gym. Sure, a room with a few kettlebells is a gym. The city centre\nis full of hotels. So what brings all the people then? For me it\u2019s a\ncombination of factors. Lisbon has become popular with remote workers,\nespecially crypto-bros because of the taxation. It\u2019s a lively city with a good\nclimate and prices still lower than other European capitals. I see the appeal.\nSo far I found myself much more fascinated with the history of the place and\nthe affinity with Brazil. As I keep exploring the city more I see them tourists\nidly wondering around downtown, drinking cocktails straight from the pineapple.\nDoesn\u2019t feel that different to other popular destinations. But as a motorized\nrickshaw pass by I hear the girls cheerfully singing<\/p>\n

\n

Nossa, nossa
\nAssim voc\u00ea me mata
\nAi, se eu te pego<\/p>\n<\/blockquote>\n\n

a very catchy Brazilian song I picked up from hanging out with Latin musicians.<\/p>\n\n

\"pineapple<\/p>\n\n

I keep finding it incredible that all the way back in 1500\u2019s Portuguese sailors\ncolonised Brazil. Later they colonised other places in Africa and Asia. No\nwonder it took them a long time to swallow their pride and give up their\nterritories. They lost thousands of people too in 1961-1974 still trying to hold\nonto their last colonies. Ultimately they suffered a defeat. Now that the\nnation is cured of its pride and they have become a modern modest democracy\nthey\u2019ve discovered the problem of declining population. Rich westerners have\nbetter things to do than to have kids. Appears current solution to the problem\nis to import people from the ex colonies. And so there are Angolian, Mozambican\nand, of course, Brazilian restaurants here. So far I\u2019ve only tried one\nBrazilian dish (moqueca de peixe), and it was a very rich explosion of exotic,\nspicy flavours. After having the incredible dish and being exposed to the\ncatchy music I must admit it is tempting to pick up some basic Portuguese\nand travel through Brazil as my next journey.<\/p>\n\n

\"tower\"<\/p>\n\n

And so where am I, in the end? I am in a modern Western capital with a distinct\nculture and a flavour of colonial past. Welcome to Lisboa, welcome to not-Spain!<\/p>\n\n

\"staircase\"<\/p>\n"},{"title":"Computing rolling median in Go","link":"https:\/\/afiodorov.github.io\/2020\/02\/04\/rolling-median\/","pubDate":"Tue, 04 Feb 2020 00:00:00 +0000","author":"Tom A. Fiodorov","description":"

Back in the day when I was teaching myself how to program<\/del> land a programming\njob, I\u2019d spend quite some time learning basic data structures such as queues,\nstacks, heaps, linked lists, etc. To my disappointment though, now that I am\nemployed in tech I almost never get a chance to implement these anymore.\nReasons are: I can often use a library or I simply don\u2019t need these ingredients\nas they are too \u201clow-level\u201d.<\/p>\n\n

Recently though I have had a chance to brush up on my Computer Science 101\nknowledge, and weeks go by but I am still excited. Thus I bring the\nexcitement to you, my dear reader!<\/p>\n\n

The story begins in me having to make the following computation in go:<\/p>\n\n

ser<\/span>.<\/span>rolling<\/span>(<\/span>period<\/span>=<\/span>30<\/span>).<\/span>median<\/span>()<\/span> # this is python's Pandas library, N=30<\/span><\/code><\/pre><\/figure>\n\n
\n\n
Without further ado, let\u2019s begin our rolling median implementation. Start with\nan underlying data structure that holds our timestamped numbers:<\/p>\n\n
package<\/span> main<\/span>\n\nimport<\/span> (<\/span>\n\t\"math\"<\/span>\n\t\"sort\"<\/span>\n\t\"time\"<\/span>\n)<\/span>\n\n\/\/ Data holds Value & Time when value was observed<\/span>\ntype<\/span> Data<\/span> struct<\/span> {<\/span>\n\tValue<\/span> float64<\/span>\n\tTime<\/span>  time<\/span>.<\/span>Time<\/span>\n}<\/span><\/code><\/pre><\/figure>\n\n
\/\/ SortedFloatSlice assumes elements are sorted<\/span>\ntype<\/span> SortedFloatSlice<\/span> []<\/span>float64<\/span>\n\n\/\/ Insert into slice maintaing the sort order<\/span>\nfunc<\/span> (<\/span>f<\/span> SortedFloatSlice<\/span>)<\/span> Insert<\/span>(<\/span>value<\/span> float64<\/span>)<\/span> SortedFloatSlice<\/span> {<\/span>\n\ti<\/span> :=<\/span> sort<\/span>.<\/span>SearchFloat64s<\/span>(<\/span>f<\/span>,<\/span> value<\/span>)<\/span>\n\tn<\/span> :=<\/span> append<\/span>(<\/span>f<\/span>,<\/span> 0<\/span>)<\/span>\n\tcopy<\/span>(<\/span>n<\/span>[<\/span>i<\/span>+<\/span>1<\/span>:<\/span>],<\/span> n<\/span>[<\/span>i<\/span>:<\/span>])<\/span>\n\tn<\/span>[<\/span>i<\/span>]<\/span> =<\/span> value<\/span>\n\treturn<\/span> n<\/span>\n}<\/span>\n\n\/\/ Delete from slice maintaing the sort order<\/span>\nfunc<\/span> (<\/span>f<\/span> SortedFloatSlice<\/span>)<\/span> Delete<\/span>(<\/span>value<\/span> float64<\/span>)<\/span> SortedFloatSlice<\/span> {<\/span>\n\ti<\/span> :=<\/span> sort<\/span>.<\/span>SearchFloat64s<\/span>(<\/span>f<\/span>,<\/span> value<\/span>)<\/span>\n\tn<\/span> :=<\/span> append<\/span>(<\/span>f<\/span>[<\/span>:<\/span>i<\/span>],<\/span> f<\/span>[<\/span>i<\/span>+<\/span>1<\/span>:<\/span>]<\/span>...<\/span>)<\/span>\n\treturn<\/span> n<\/span>\n}<\/span>\n\n\/\/ Median of the slice<\/span>\nfunc<\/span> (<\/span>f<\/span> SortedFloatSlice<\/span>)<\/span> Median<\/span>()<\/span> float64<\/span> {<\/span>\n\tif<\/span> len<\/span>(<\/span>f<\/span>)<\/span>%<\/span>2<\/span> ==<\/span> 1<\/span> {<\/span>\n\t\treturn<\/span> f<\/span>[<\/span>len<\/span>(<\/span>f<\/span>)<\/span>\/<\/span>2<\/span>]<\/span>\n\t}<\/span>\n\treturn<\/span> (<\/span>f<\/span>[<\/span>len<\/span>(<\/span>f<\/span>)<\/span>\/<\/span>2<\/span>]<\/span> +<\/span> f<\/span>[<\/span>len<\/span>(<\/span>f<\/span>)<\/span>\/<\/span>2<\/span>-<\/span>1<\/span>])<\/span> \/<\/span> 2<\/span>\n}<\/span><\/code><\/pre><\/figure>\n\n
We have all the components to make the computation possible. Just like pandas\nwe pre-fill the result with NaN<\/code>\u2019s where computation is impossible. We\ninitialise the sliding window at the start of the series, and keep advancing\nthe window one element at a time, whilst keeping the window sorted<\/p>\n\n
func<\/span> movingMedian<\/span>(<\/span>series<\/span> []<\/span>Data<\/span>,<\/span> period<\/span> int<\/span>)<\/span> []<\/span>Data<\/span> {<\/span>\n\tres<\/span> :=<\/span> make<\/span>([]<\/span>Data<\/span>,<\/span> len<\/span>(<\/span>series<\/span>))<\/span>\n\n\twindow<\/span> :=<\/span> make<\/span>(<\/span>SortedFloatSlice<\/span>,<\/span> 0<\/span>,<\/span> period<\/span>-<\/span>1<\/span>)<\/span>\n\tfor<\/span> _<\/span>,<\/span> v<\/span> :=<\/span> range<\/span> series<\/span>[<\/span>:<\/span>(<\/span>period<\/span> -<\/span> 1<\/span>)]<\/span> {<\/span>\n\t\twindow<\/span> =<\/span> append<\/span>(<\/span>window<\/span>,<\/span> v<\/span>.<\/span>Value<\/span>)<\/span>\n\t}<\/span>\n\tsort<\/span>.<\/span>Float64s<\/span>(<\/span>window<\/span>)<\/span>\n\n\tmedian<\/span> :=<\/span> float64<\/span>(<\/span>0<\/span>)<\/span>\n\tfor<\/span> i<\/span>,<\/span> v<\/span> :=<\/span> range<\/span> series<\/span> {<\/span>\n\t\tif<\/span> i<\/span> <<\/span> period<\/span>-<\/span>1<\/span> {<\/span>\n\t\t\tmedian<\/span> =<\/span> math<\/span>.<\/span>NaN<\/span>()<\/span>\n\t\t}<\/span> else<\/span> {<\/span>\n\t\t\tif<\/span> i<\/span>-<\/span>period<\/span> >=<\/span> 0<\/span> {<\/span>\n\t\t\t\twindow<\/span> =<\/span> window<\/span>.<\/span>Delete<\/span>(<\/span>series<\/span>[<\/span>i<\/span>-<\/span>period<\/span>]<\/span>.<\/span>Value<\/span>)<\/span>\n\t\t\t}<\/span>\n\t\t\twindow<\/span> =<\/span> window<\/span>.<\/span>Insert<\/span>(<\/span>v<\/span>.<\/span>Value<\/span>)<\/span>\n\t\t\tmedian<\/span> =<\/span> window<\/span>.<\/span>Median<\/span>()<\/span>\n\t\t}<\/span>\n\n\t\tres<\/span>[<\/span>i<\/span>]<\/span> =<\/span> Data<\/span>{<\/span>\n\t\t\tTime<\/span>:<\/span>  v<\/span>.<\/span>Time<\/span>,<\/span>\n\t\t\tValue<\/span>:<\/span> median<\/span>,<\/span>\n\t\t}<\/span>\n\t}<\/span>\n\n\treturn<\/span> res<\/span>\n}<\/span><\/code><\/pre><\/figure>\n\n
\n\n
\n\n
This concludes the code & the blog post. Hurray!<\/p>\n\n
TL;DR I used a sorted array to make the computation.<\/p>\n\n
TL;DR;TL;DR I am (still) a massive nerd.<\/p>\n\n"},{"title":"Getting a musical note from audio","link":"https:\/\/afiodorov.github.io\/2019\/12\/06\/note\/","pubDate":"Fri, 06 Dec 2019 00:00:00 +0000","author":"Tom A. Fiodorov","description":"
Lately I have been learning music<\/s> playing around with computers & maths\nunder the pretence of learning music. I am particularly interested in breaking\ndown melodies into individual notes. However since actually training my\nmusical ear is hard work, I am going to use basic signal processing in order to\nachieve my goal.<\/p>\n\n
\n    
Let's start with this 1 second music audio:<\/figcaption>\n    
CODE<\/summary>
\n\n
import<\/span> librosa<\/span>\nimport<\/span> pandas<\/span> as<\/span> pd<\/span>\nimport<\/span> plotly.express<\/span> as<\/span> px<\/span>\n\nnote_filepath<\/span> =<\/span> \"\/home\/tom\/afiodorov.github.io\/assets\/note.wav\"<\/span>\ny<\/span>,<\/span> sample_rate<\/span> =<\/span> librosa<\/span>.<\/span>load<\/span>(<\/span>note_filepath<\/span>,<\/span> sr<\/span>=<\/span>None<\/span>)<\/span>\ndf<\/span> =<\/span> pd<\/span>.<\/span>DataFrame<\/span>({<\/span>'time (s)'<\/span>:<\/span> np<\/span>.<\/span>arange<\/span>(<\/span>len<\/span>(<\/span>y<\/span>))<\/span> \/<\/span> sample_rate<\/span>,<\/span> 'wave'<\/span>:<\/span> y<\/span>})<\/span>\nfig<\/span> =<\/span> px<\/span>.<\/span>line<\/span>(<\/span>df<\/span>,<\/span> x<\/span>=<\/span>\"time (s)\"<\/span>,<\/span> y<\/span>=<\/span>\"wave\"<\/span>,<\/span> title<\/span>=<\/span>'Raw Waveform'<\/span>)<\/span>\nfig<\/span>.<\/span>show<\/span>()<\/span><\/code><\/pre><\/figure>\n\n<\/p><\/details>\n\n
<\/div>\n\n
Turns out digital audio is represented by a series of real numbers, i.e. it is\na one-dimensional array. When we plot our series, we get extremely wiggly\nline. Such plot is referred to as (raw) waveform. If you zoom in enough in the\nabove chart, you will see that it\u2019s just a line. I found it surprising how easy\nit is to represent music. The greatest, most complex compositions you have ever\nlistened to are just a bunch of real numbers. Let\u2019s print first 10 numbers of the\nabove file just for illustrative purpose:<\/p>\n\n
\n\n
y<\/span>[:<\/span>10<\/span>]<\/span>\n><\/span> array<\/span>([<\/span>-<\/span>0.03649902<\/span>,<\/span> -<\/span>0.04107666<\/span>,<\/span> -<\/span>0.05935669<\/span>,<\/span> -<\/span>0.09130859<\/span>,<\/span> -<\/span>0.1232605<\/span> ,<\/span>\n><\/span>        -<\/span>0.13696289<\/span>,<\/span> -<\/span>0.15524292<\/span>,<\/span> -<\/span>0.17349243<\/span>,<\/span> -<\/span>0.18261719<\/span>,<\/span> -<\/span>0.20089722<\/span>],<\/span>\n><\/span>              dtype<\/span>=<\/span>float32<\/span>)]<\/span><\/code><\/pre><\/figure>\n\n<\/p>\n\n
Do these 10 numbers sound like a note to you :)? Just kidding. Even if you\ndiscovered a way to enjoy music by reading a long list of numbers, these 10\nwould actually be inaudible: they represent a miniscule part of the 1 second\ninterval above. In the plot above 44 100 points are used to represent 1 second\nworth of audio. This is the same as saying that the sampling rate of our\nrecording is 44 100 Hz. To put it in elementary terms, we record the audio wave\nvalue 44 100 times per second at equally spaced intervals. Why 44 100? This\nnumber is linked to human hearing. People can hear sounds from 20 to 20 000 Hz,\nwhere the higher the number of Hz, the higher the sound sounds to you. It turns\nout in order to record audio in such range, we need to sample twice as often,\nthus the sampling rate of 44 1001<\/a><\/sup> has been used often in digital records.<\/p>\n\n
\n\n
Basic musical theory: we have 7 notes: do, re, mi, fa, sol, la, si, do,\nalternatively known as C, D, E, F, G, A, B, C. The latter do is an octave above\nthe first do. What\u2019s an octave from a physical point of view? Turns out if a\nnote has frequency twice as high as another note than those two notes are\nthe same but one octave apart. Often subscript is used to denote octaves,\ne.g. C\u2082 and C\u2083 are one octave apart whereas F\u2084 and F\u2087 are 3 octaves apart. At\nsome point in the 20th century musicians all around the world agreed that A\u2084\n(la) is 440 Hz. Now let\u2019s look at notes on a piano:<\/p>\n\n
\"piano\"<\/p>\n\n
There are actually 12 keys between the two C\u2019s when we count the black keys.\nThese 12 keys are equally separated musically. Curiously there\u2019s no black key\nbetween mi (E) & fa (F). The distance between F and G is a tone and the\ndistance between E & F is a semitone. What do I mean by equally separated? The\nratio of frequencies matter when we compare notes, as opposed to their absolute\nfrequency. Thus for adjacent notes to have the same ratio I can apply a factor of\n\\( \\sqrt[12]{2} \\approx 1.06 \\) every time I go up by a semitone. This way\nafter I apply it 12 times, I double the frequency and thus complete the octave.\nUsing this knowledge it\u2019s a trivial exercise in computer science to generate\nfrequencies of all notes. I presented to you a cropped table of some of them\nfrom octave 0 to 6:<\/p>\n\n\n  \n    \n      \n    \n      \n      \n      \n      \n      \n      \n      
octave<\/th>\n      F#<\/th>\n      G<\/th>\n      G#<\/th>\n      A<\/th>\n      A#<\/th>\n      B<\/th>\n    <\/tr>\n  <\/thead>\n  
0<\/td>\n      23.1247<\/td>\n      24.4997<\/td>\n      25.9565<\/td>\n      27.5<\/td>\n      29.1352<\/td>\n      30.8677<\/td>\n    <\/tr>\n    
1<\/td>\n      46.2493<\/td>\n      48.9994<\/td>\n      51.9131<\/td>\n      55<\/td>\n      58.2705<\/td>\n      61.7354<\/td>\n    <\/tr>\n    
2<\/td>\n      92.4986<\/td>\n      97.9989<\/td>\n      103.826<\/td>\n      110<\/td>\n      116.541<\/td>\n      123.471<\/td>\n    <\/tr>\n    
3<\/td>\n      184.997<\/td>\n      195.998<\/td>\n      207.652<\/td>\n      220<\/td>\n      233.082<\/td>\n      246.942<\/td>\n    <\/tr>\n    
4<\/td>\n      369.994<\/td>\n      391.995<\/td>\n      415.305<\/td>\n      440<\/td>\n      466.164<\/td>\n      493.883<\/td>\n    <\/tr>\n    
5<\/td>\n      739.989<\/td>\n      783.991<\/td>\n      830.609<\/td>\n      880<\/td>\n      932.328<\/td>\n      987.767<\/td>\n    <\/tr>\n    
6<\/td>\n      1479.98<\/td>\n      1567.98<\/td>\n      1661.22<\/td>\n      1760<\/td>\n      1864.66<\/td>\n      1975.53<\/td>\n    <\/tr>\n  <\/tbody>\n<\/table>\n\n
rest of the notes<\/summary>
\n\n\n\n
  octave<\/th>        C<\/th>       C#<\/th>        D<\/th>       D#<\/th>        E<\/th>        F<\/th><\/tr>\n<\/thead>\n
       0<\/td>  16.3516<\/td>  17.3239<\/td>  18.354 <\/td>  19.4454<\/td>  20.6017<\/td>  21.8268<\/td><\/tr>\n
       1<\/td>  32.7032<\/td>  34.6478<\/td>  36.7081<\/td>  38.8909<\/td>  41.2034<\/td>  43.6535<\/td><\/tr>\n
       2<\/td>  65.4064<\/td>  69.2957<\/td>  73.4162<\/td>  77.7817<\/td>  82.4069<\/td>  87.3071<\/td><\/tr>\n
       3<\/td> 130.813 <\/td> 138.591 <\/td> 146.832 <\/td> 155.563 <\/td> 164.814 <\/td> 174.614 <\/td><\/tr>\n
       4<\/td> 261.626 <\/td> 277.183 <\/td> 293.665 <\/td> 311.127 <\/td> 329.628 <\/td> 349.228 <\/td><\/tr>\n
       5<\/td> 523.251 <\/td> 554.365 <\/td> 587.33  <\/td> 622.254 <\/td> 659.255 <\/td> 698.456 <\/td><\/tr>\n
       6<\/td>1046.5   <\/td>1108.73  <\/td>1174.66  <\/td>1244.51  <\/td>1318.51  <\/td>1396.91  <\/td><\/tr>\n<\/tbody>\n<\/table>\n<\/p><\/details>\n\n
CODE<\/summary>
\n\n
def<\/span> num_semitones_between<\/span>(<\/span>note1<\/span>:<\/span> str<\/span>,<\/span> note2<\/span>:<\/span> str<\/span>)<\/span> -><\/span> int<\/span>:<\/span>\n    notes<\/span> =<\/span> [<\/span>'C'<\/span>,<\/span> 'C#'<\/span>,<\/span> 'D'<\/span>,<\/span> 'D#'<\/span>,<\/span> 'E'<\/span>,<\/span> 'F'<\/span>,<\/span> 'F#'<\/span>,<\/span> 'G'<\/span>,<\/span> 'G#'<\/span>,<\/span> 'A'<\/span>,<\/span> 'A#'<\/span>,<\/span> 'B'<\/span>]<\/span>\n    return<\/span> notes<\/span>.<\/span>index<\/span>(<\/span>note1<\/span>)<\/span> -<\/span> notes<\/span>.<\/span>index<\/span>(<\/span>note2<\/span>)<\/span>\n\ndef<\/span> note_frequency<\/span>(<\/span>note<\/span>:<\/span> str<\/span>,<\/span> octave<\/span>:<\/span> int<\/span> =<\/span> 4<\/span>)<\/span> -><\/span> float<\/span>:<\/span>\n    a_4<\/span> =<\/span> 440<\/span> #  standard pitch\n<\/span>    a_frequency<\/span> =<\/span> a_4<\/span> *<\/span> 2<\/span> **<\/span> (<\/span>octave<\/span> -<\/span> 4<\/span>)<\/span>\n    adjustment<\/span> =<\/span> num_semitones_between<\/span>(<\/span>note<\/span>,<\/span> \"A\"<\/span>)<\/span>\n    return<\/span> a_frequency<\/span> *<\/span> 2<\/span> **<\/span> (<\/span>adjustment<\/span> \/<\/span> 12<\/span>)<\/span>\n\ndef<\/span> gen_notes_table<\/span>()<\/span> -><\/span> pd<\/span>.<\/span>DataFrame<\/span>:<\/span>\n    l<\/span> =<\/span> []<\/span>\n    notes<\/span> =<\/span> [<\/span>'C'<\/span>,<\/span> 'C#'<\/span>,<\/span> 'D'<\/span>,<\/span> 'D#'<\/span>,<\/span> 'E'<\/span>,<\/span> 'F'<\/span>,<\/span> 'F#'<\/span>,<\/span> 'G'<\/span>,<\/span> 'G#'<\/span>,<\/span> 'A'<\/span>,<\/span> 'A#'<\/span>,<\/span> 'B'<\/span>]<\/span>\n    for<\/span> octave<\/span> in<\/span> range<\/span>(<\/span>7<\/span>):<\/span>\n        l<\/span>.<\/span>append<\/span>([<\/span>note_frequency<\/span>(<\/span>note<\/span>,<\/span> octave<\/span>=<\/span>octave<\/span>)<\/span> for<\/span> note<\/span> in<\/span> notes<\/span>])<\/span>\n    df<\/span> =<\/span> pd<\/span>.<\/span>DataFrame<\/span>(<\/span>l<\/span>,<\/span> columns<\/span>=<\/span>notes<\/span>)<\/span>\n    df<\/span>.<\/span>index<\/span>.<\/span>name<\/span> =<\/span> 'octave'<\/span>\n    return<\/span> df<\/span>\n\nimport<\/span> tabulate<\/span>\ntabulate<\/span>.<\/span>tabulate<\/span>(<\/span>gen_notes_table<\/span>(),<\/span> tablefmt<\/span>=<\/span>\"github\"<\/span>,<\/span> headers<\/span>=<\/span>\"keys\"<\/span>)<\/span><\/code><\/pre><\/figure>\n\n<\/p><\/details>\n\n
\n\n
Note: in the above table the ratio between A# and A is roughly 1.06 (466 \/\n440), just like advertised.<\/p>\n\n
OK, but how do we get the goddamn note from the audio snippet? We need some\nsort of transformation of the waveform into frequencies. Fourier transform<\/a>, an\ninteresting piece of mathematics discovered by Joseph Fourier in 1822, allows\nus to \u201cpluck out\u201d specific frequencies from the sound and see how much each of\nthem contribute. Graphically it means that we are going to take the waveform\nshown above, and try to approximate it by adding functions cosine and sine\nuntil the result looks almost exactly like our waveform. A process illustrated\nbelow for a simple waveform that is of quadratic shape:<\/p>\n\n
\"fourier\"<\/p>\n\n
The more cosine & sine functions we add, the more precise the approximation becomes.\nWhat we are interested in is which exactly cosine & sine contribute the most to\nthe recorded waveform, i.e. which notes are present in the audio? Let\u2019s perform\nthe transform and plot it:<\/p>\n\n
CODE<\/summary>
\n\n
import<\/span> numpy<\/span> as<\/span> np<\/span>\n\nft<\/span> =<\/span> np<\/span>.<\/span>fft<\/span>.<\/span>rfft<\/span>(<\/span>y<\/span>)<\/span>\ndf<\/span> =<\/span> pd<\/span>.<\/span>DataFrame<\/span>(<\/span>np<\/span>.<\/span>absolute<\/span>(<\/span>ft<\/span>[<\/span>1<\/span>:]),<\/span> index<\/span>=<\/span>range<\/span>(<\/span>1<\/span>,<\/span> len<\/span>(<\/span>ft<\/span>)))<\/span>\ndf<\/span>.<\/span>index<\/span>.<\/span>name<\/span> =<\/span> \"Frequency (Hz)\"<\/span>\ndf<\/span>.<\/span>columns<\/span> =<\/span> [<\/span>'Amplitude'<\/span>]<\/span>\ndf<\/span> =<\/span> df<\/span>.<\/span>reset_index<\/span>()<\/span>\nfig<\/span> =<\/span> px<\/span>.<\/span>line<\/span>(<\/span>df<\/span>[:<\/span>2000<\/span>],<\/span> x<\/span>=<\/span>\"Frequency (Hz)\"<\/span>,<\/span> y<\/span>=<\/span>\"Amplitude\"<\/span>,<\/span> title<\/span>=<\/span>'Fourier transform'<\/span>)<\/span>\nfig<\/span>.<\/span>show<\/span>()<\/span><\/code><\/pre><\/figure>\n\n<\/p><\/details>\n\n
<\/div>\n\n
From the above graph we see that the frequency that is the most present is 370\nHz, meaning the note recorded is F#\u2084 (fi). Interestingly enough F#\u2085 (740 Hz)\nand F#\u2086 (\u2248 1486 Hz) are also present. These are so-called overtones that make\nthe sound sound more pleasant. Whew, so much maths to identify the note, but we\nfinally got here! Finally, I am including two more audio samples, for\neducational purposes.<\/p>\n\n
\n    
This is how the snippet would sound without the overtones:<\/figcaption>\n    
Complexity analysis: we have to do insert\/delete from a sorted array at each\nstep, which is O(window). Since we have N steps the resulting complexity is\nO(N*window). Can we do better? Turns out the canonical algorithm for computing\na rolling median is to keep window elements in a skip list<\/a> instead.\nSkip list allows one to insert, delete and random access sorted elements in\nlogN times, thus the complexity of the canonical algorithm is lower:\nO(N*log(window)). However for windows of small sizes, I feel like the advantage\nof skip lists is not worth the cost of much more complex implementation.<\/p>\n\n
When computing rolling median, we will need to remember last N (where N is the\nsize of the window) entries in a sorted order. The fact that elements are\nsorted allows us to quickly find the median by picking the element in the\nmiddle. Go provides a handy sort<\/code> library that allows one to use binary search<\/a>\nto insert & delete elements from a slice that\u2019s already sorted:<\/p>\n\n
i.e. computing rolling\/moving median<\/a> of a series of timestamped numbers.\nOne of the reasons I like coding in Go sometimes is because it forces you to\nimplement things like that from scratch, rather than rely on someone else\u2019s\ncode. It\u2019s problems like these that got me enthusiastic about entering this\nprofession (as opposed to having to peruse Django<\/a> documentation: a read that\ndoesn\u2019t have a great ending).<\/p>\n\n