{"title":"Amanjeev Sethi","subtitle":"Personal website and blog of Amanjeev Sethi","link":[{"@attributes":{"rel":"self","type":"application\/atom+xml","href":"https:\/\/amanjeev.com\/atom.xml"}},{"@attributes":{"rel":"alternate","type":"text\/html","href":"https:\/\/amanjeev.com"}}],"generator":"Zola","updated":"2024-02-09T00:00:00+00:00","id":"https:\/\/amanjeev.com\/atom.xml","entry":[{"title":"Banality of Grief (Poem)","published":"2024-02-09T00:00:00+00:00","updated":"2024-02-09T00:00:00+00:00","author":{"name":"\n            \n              Unknown\n            \n          "},"link":{"@attributes":{"rel":"alternate","type":"text\/html","href":"https:\/\/amanjeev.com\/blog\/poem-banality-of-grief\/"}},"id":"https:\/\/amanjeev.com\/blog\/poem-banality-of-grief\/","content":"<h2 id=\"banality-of-grief\">Banality of Grief<\/h2>\n<p>Hold the opposing truths, gently<br \/>\nin your gaze<br \/>\nyou may sound obvious<br \/>\nbut it is real<br \/>\nfor you<br \/>\nYou may feel small<br \/>\nbut isn\u2019t it weighing<br \/>\non you?<br \/>\nYou must speak<br \/>\ncry even<br \/>\nThe banality of your Grief<br \/>\nthat you think, is in your mind<br \/>\nYou may weep<br \/>\neven shout<br \/>\nbut you must say<br \/>\nthe obvious out loud<\/p>\n"},{"title":"Some notes on OpenWrt","published":"2023-12-16T00:00:00+00:00","updated":"2023-12-16T00:00:00+00:00","author":{"name":"\n            \n              Unknown\n            \n          "},"link":{"@attributes":{"rel":"alternate","type":"text\/html","href":"https:\/\/amanjeev.com\/blog\/notes-on-openwrt\/"}},"id":"https:\/\/amanjeev.com\/blog\/notes-on-openwrt\/","content":"<h2 id=\"what-is-openwrt\">What is <a href=\"https:\/\/openwrt.org\/\">OpenWrt<\/a>?<\/h2>\n<p>I pronounce it as \"Open Wart\" but I digress and the notes haven't even begun. It is a Linux(-based), open source operating system for embedded devices but it's most use if found in your routers. If you want your router to have an open source and free operating system, this is a decent bet.<\/p>\n<p>To download firmware for your device\/architecture go to <a href=\"https:\/\/firmware-selector.openwrt.org\/\">OpenWrt Firmware download page and try to find your device<\/a>.<\/p>\n<h3 id=\"architectures-and-devices\">Architectures and devices<\/h3>\n<p>OpenWrt is open to adding new devices. Check <a href=\"https:\/\/openwrt.org\/docs\/guide-developer\/adding_new_device#adding_a_new_device\">how to add new device page<\/a> for details. If you want to go that route, maybe check if your device is already supported by going to <a href=\"https:\/\/openwrt.org\/toh\/start\">\"Table of Hardware\" support page<\/a>. This is also available as an archive to download and grep locally.<\/p>\n<h3 id=\"packages\">Packages<\/h3>\n<p>OpenWrt also has a <a href=\"https:\/\/openwrt.org\/packages\/start\">package ecosystem<\/a>. So, you can package your own application for the OS. In fact, the UI (Web based) called Luci is a package itself that you can install once you have the OS up and running. The packages are divided into <a href=\"https:\/\/openwrt.org\/packages\/index_owrt21_2\/start\">many categories<\/a> and creating a new package means you should choose one of those categories and likely sub-categories.<\/p>\n<p>You can install packages using the <code>opkg<\/code> installation tool from the commandline. You will need to SSH into the router\/device after installing the OS.<\/p>\n<h2 id=\"rust-and-openwrt\">Rust and OpenWrt<\/h2>\n<p>This is new! Starting <a href=\"https:\/\/www.phoronix.com\/news\/OpenWrt-23.05-Released\">OpenWrt version 23.05 you have native Rust package support<\/a>. This can make life easier if you want to create a simple Rust package for OpenWrt.<\/p>\n<p>For me, my application was far more complex than what the OpenWrt Rust support toolchain can handle right now because we bundle some web artifacts etc. So, in my case it was easier to find the architecture and compile my binary for that using <a href=\"https:\/\/github.com\/NixOS\/nix\">Nix<\/a>.<\/p>\n<p>You can find a few things by running these commands for your router -<\/p>\n<h3 id=\"uname-a\"><code>uname -a<\/code><\/h3>\n<p>Shows you system information in brief. For example,<\/p>\n<pre data-lang=\"shell\" style=\"background-color:#eff1f5;color:#4f5b66;\" class=\"language-shell \"><code class=\"language-shell\" data-lang=\"shell\"><span>root@OpenWrt:~# uname -a\n<\/span><span>Linux OpenWrt 5.14.171 # 21:05:12 2023 armv7l GNU\/Linux\n<\/span><\/code><\/pre>\n<h3 id=\"lscpu\"><code>lscpu<\/code><\/h3>\n<p>This will show you architecture in detail. For example,<\/p>\n<pre data-lang=\"shell\" style=\"background-color:#eff1f5;color:#4f5b66;\" class=\"language-shell \"><code class=\"language-shell\" data-lang=\"shell\"><span>root@OpenWrt:~# lscpu\n<\/span><span>Architecture:           armv7l\n<\/span><span>  Byte Order:           Little Endian\n<\/span><span>CPU(s):                 2\n<\/span><span>  On-line CPU(s) list:  0,1\n<\/span><span>Vendor ID:              ARM\n<\/span><span>  Model name:           Cortex-A9\n<\/span><span>    Model:              1\n<\/span><span>    Thread(s) per core: 1\n<\/span><span>    Core(s) per socket: 2\n<\/span><span>    Socket(s):          1\n<\/span><\/code><\/pre>\n<p>The important part is that you can also get the Model name (Cortex-A9) which maybe needed to choose the right architecture.<\/p>\n<h3 id=\"ldd-version\"><code>ldd --version<\/code><\/h3>\n<p>This will tell you which base library your system is using. For example, the following uses <code>musl<\/code>,<\/p>\n<pre data-lang=\"shell\" style=\"background-color:#eff1f5;color:#4f5b66;\" class=\"language-shell \"><code class=\"language-shell\" data-lang=\"shell\"><span>root@OpenWrt:~# ldd --version\n<\/span><span>musl libc (armhf)\n<\/span><span>Version 1.2.3\n<\/span><span>Dynamic Program Loader\n<\/span><span>Usage: ldd [options] [--] pathname\n<\/span><\/code><\/pre>\n<h2 id=\"create-a-package-for-openwrt\">Create a package for OpenWrt<\/h2>\n<p>I should begin by saying that in this exercise, you need a lot of GNU Make knowledge. The hell of Make will be enough for you to say no to creating OpenWrt package. However, let's look at a few things I learned during this \"simple\" exercise.<\/p>\n<h3 id=\"easy-way-out-is-to-build-binary-yourself\">Easy way out is to build binary yourself<\/h3>\n<p>If you know the architectures you want to support and can publish the package yourself, you can avoid the entire drama and just host things yourself. Honestly, it was far easier to do that especially if you have a complex build process. Which brings the next point.<\/p>\n<p>Quick before the next point, the <code>.ipk<\/code> file is just a <code>tar.gz<\/code>. So, you can unpack some existing package and see what kind of files are inside. It kinda feels like debian based structure but I could be wrong.<\/p>\n<h3 id=\"the-packaging-process-is-a-nightmare\">The packaging process is a nightmare<\/h3>\n<p>You can try and follow their <a href=\"https:\/\/openwrt.org\/docs\/guide-developer\/packages\">tutorial for packaging on their Wiki<\/a>.<\/p>\n<p>If you want to package it using their Make stuff, you will end up having more architectural support but it is a nightmare and not even like the Halloween kind where you get the candy.<\/p>\n<p>No really, look at this makefile -<\/p>\n<pre data-lang=\"make\" style=\"background-color:#eff1f5;color:#4f5b66;\" class=\"language-make \"><code class=\"language-make\" data-lang=\"make\"><span style=\"color:#b48ead;\">include $(<\/span><span style=\"color:#bf616a;\">TOPDIR<\/span><span style=\"color:#b48ead;\">)<\/span><span style=\"color:#a3be8c;\">\/rules.mk\n<\/span><span> \n<\/span><span style=\"color:#bf616a;\">PKG_NAME<\/span><span>:=<\/span><span style=\"color:#a3be8c;\">bridge\n<\/span><span style=\"color:#bf616a;\">PKG_VERSION<\/span><span>:=<\/span><span style=\"color:#a3be8c;\">1.0.6\n<\/span><span style=\"color:#bf616a;\">PKG_RELEASE<\/span><span>:=<\/span><span style=\"color:#a3be8c;\">1\n<\/span><span> \n<\/span><span style=\"color:#bf616a;\">PKG_BUILD_DIR<\/span><span>:=<\/span><span style=\"color:#b48ead;\">$(<\/span><span style=\"color:#bf616a;\">BUILD_DIR<\/span><span style=\"color:#b48ead;\">)<\/span><span style=\"color:#a3be8c;\">\/bridge-utils-<\/span><span style=\"color:#b48ead;\">$(<\/span><span style=\"color:#bf616a;\">PKG_VERSION<\/span><span style=\"color:#b48ead;\">)\n<\/span><span style=\"color:#bf616a;\">PKG_SOURCE<\/span><span>:=<\/span><span style=\"color:#a3be8c;\">bridge-utils-<\/span><span style=\"color:#b48ead;\">$(<\/span><span style=\"color:#bf616a;\">PKG_VERSION<\/span><span style=\"color:#b48ead;\">)<\/span><span style=\"color:#a3be8c;\">.tar.gz\n<\/span><span style=\"color:#bf616a;\">PKG_SOURCE_URL<\/span><span>:=<\/span><span style=\"color:#a3be8c;\">@SF\/bridge\n<\/span><span style=\"color:#bf616a;\">PKG_HASH<\/span><span>:=<\/span><span style=\"color:#a3be8c;\">9b7dc52656f5cbec846a7ba3299f73bd\n<\/span><span> \n<\/span><span style=\"color:#b48ead;\">include $(<\/span><span style=\"color:#bf616a;\">INCLUDE_DIR<\/span><span style=\"color:#b48ead;\">)<\/span><span style=\"color:#a3be8c;\">\/package.mk\n<\/span><span> \n<\/span><span style=\"color:#b48ead;\">define<\/span><span style=\"color:#bf616a;\"> Package\/bridge\n<\/span><span style=\"color:#a3be8c;\">  SECTION:=base\n<\/span><span style=\"color:#a3be8c;\">  CATEGORY:=Network\n<\/span><span style=\"color:#a3be8c;\">  TITLE:=Ethernet bridging configuration utility\n<\/span><span style=\"color:#a3be8c;\">  #DESCRIPTION:=This variable is obsolete. use the Package\/name\/description define instead!\n<\/span><span style=\"color:#a3be8c;\">  URL:=http:\/\/bridge.sourceforge.net\/\n<\/span><span style=\"color:#b48ead;\">endef\n<\/span><span> \n<\/span><span style=\"color:#b48ead;\">define<\/span><span style=\"color:#bf616a;\"> Package\/bridge\/description\n<\/span><span style=\"color:#a3be8c;\"> Ethernet bridging configuration utility\n<\/span><span style=\"color:#a3be8c;\"> Manage ethernet bridging; a way to connect networks together to\n<\/span><span style=\"color:#a3be8c;\"> form a larger network.\n<\/span><span style=\"color:#b48ead;\">endef\n<\/span><span> \n<\/span><span style=\"color:#b48ead;\">define<\/span><span style=\"color:#bf616a;\"> Build\/Configure\n<\/span><span style=\"color:#a3be8c;\">  $(call Build\/Configure\/Default,--with-linux-headers=$(LINUX_DIR))\n<\/span><span style=\"color:#b48ead;\">endef\n<\/span><span> \n<\/span><span style=\"color:#b48ead;\">define<\/span><span style=\"color:#bf616a;\"> Package\/bridge\/install\n<\/span><span style=\"color:#a3be8c;\">        $(INSTALL_DIR) $(1)\/usr\/sbin\n<\/span><span style=\"color:#a3be8c;\">        $(INSTALL_BIN) $(PKG_BUILD_DIR)\/brctl\/brctl $(1)\/usr\/sbin\/\n<\/span><span style=\"color:#b48ead;\">endef\n<\/span><span> \n<\/span><span style=\"color:#b48ead;\">$(<\/span><span style=\"color:#96b5b4;\">eval <\/span><span style=\"color:#b48ead;\">$(<\/span><span style=\"color:#d08770;\">call <\/span><span style=\"color:#bf616a;\">BuildPackage<\/span><span>,bridge<\/span><span style=\"color:#b48ead;\">))\n<\/span><\/code><\/pre>\n<p>This reminds me of the <a href=\"https:\/\/github.com\/ncbi\/ncbi-cxx-toolkit-public\">CMake hell of NCBI C++ Toolkit<\/a>. You can see there is a configuration file to configure CMake first.<\/p>\n"},{"title":"(In)secure C++ Training with Patricia Aas","published":"2023-11-23T00:00:00+00:00","updated":"2023-11-23T00:00:00+00:00","author":{"name":"\n            \n              Unknown\n            \n          "},"link":{"@attributes":{"rel":"alternate","type":"text\/html","href":"https:\/\/amanjeev.com\/blog\/insecure-cpp-training-with-patricia\/"}},"id":"https:\/\/amanjeev.com\/blog\/insecure-cpp-training-with-patricia\/","content":"<p>A few weeks ago I had the privilege of attending the security training (using C++) by <a href=\"https:\/\/patricia.no\">Patricia Aas<\/a>. This was also the only training she ever did in the North American timezone and that was such a fortunate thing for me. This was such a fun experience and I want to give you a bit of detail on what you can expect from this training.<\/p>\n<h2 id=\"basic-structure\">Basic structure<\/h2>\n<p>This is an intense course, but it is also a lot of fun. As a trainer myself, I am just in sheer awe of Patricia. This was 4 (four) days of 8 (eight) hours each day. You do get an hour of break in the middle but I can tell you this is still a lot of work for the trainer. Apart from that, there is a healthy variety of theory (including history) with practical exercises.<\/p>\n<h2 id=\"highlights-for-me\">Highlights (for me)<\/h2>\n<h3 id=\"material\">Material<\/h3>\n<p>I have always wanted to get hands-on security courses from the vantage of reverse-engineering, fuzzing, shellcode, and learn about the tooling around it. All of that was covered.<\/p>\n<h3 id=\"slides\">Slides<\/h3>\n<p>I give extra mention for slides because Patricia\u2019s slide-deck is so beautiful. You can get some idea of what to expect by <a href=\"https:\/\/patricia.no\">visiting her website<\/a>.<\/p>\n<h3 id=\"exercises\">Exercises<\/h3>\n<p>There are a plenty of exercises and hands-on work. You even get to do your own fuzzing, you learn about <del>heartbless<\/del> heartbleed as an exercises! I am going to setup my own fuzzing lab at home.<\/p>\n<h2 id=\"is-it-for-you\">Is it for you?<\/h2>\n<p>In my group, there were seasoned security and reverse-engineering folks. There were also total beginners like me. Seldom is a course where people from both ends of the spectrum of knowledge can get so much out of. I can assure you that no matter where you are, this course will be fun and educational for you.<\/p>\n"},{"title":"Publications","published":"2023-09-13T00:00:00+00:00","updated":"2023-09-13T00:00:00+00:00","author":{"name":"\n            \n              Unknown\n            \n          "},"link":{"@attributes":{"rel":"alternate","type":"text\/html","href":"https:\/\/amanjeev.com\/publications\/"}},"id":"https:\/\/amanjeev.com\/publications\/","content":"<p>These are some of my writings published elsewhere. They are posted here in reverse-chronological order.<\/p>\n<h2 id=\"articles\">Articles<\/h2>\n<ul>\n<li><a href=\"https:\/\/ferrous-systems.com\/blog\/cheatsheet-for-confusing-rust-terms\/\">A cheatsheet for some potentially confusing terms in Rust<\/a> - March 09, 2023<\/li>\n<li><a href=\"https:\/\/ferrous-systems.com\/blog\/binding-with-bindgen\/\">The <code>bindgen<\/code> project update<\/a> - March 06, 2023<\/li>\n<li><a href=\"https:\/\/www.distributedgenomics.ca\/posts\/candigv2-aai\/\">The CanDIGv2 Authentication and Authorization Stack<\/a> - April 13, 2021<\/li>\n<li>CanDIG\u2019s input on CINECA blog post \"<a href=\"https:\/\/www.cineca-project.eu\/blog-all\/integration-of-new-cohort-infrastructures-to-the-elixir-aai\">Integration of new cohort infrastructures to the ELIXIR AAI<\/a>\" - January 07, 2020<\/li>\n<li><a href=\"https:\/\/www.distributedgenomics.ca\/posts\/gentle-intro-to-federated-distributed-authz\/\">A Gentle Introduction to Federated and Distributed Authorization<\/a> - March 04, 2020<\/li>\n<li><a href=\"https:\/\/blog.rust-lang.org\/inside-rust\/2019\/10\/28\/rustc-learning-working-group-introduction.html\">The Rustc Dev Guide Working Group - An Introduction<\/a> - October 28, 2019<\/li>\n<\/ul>\n<h2 id=\"research-papers\">Research papers<\/h2>\n<ul>\n<li><a href=\"https:\/\/doi.org\/10.1016\/j.xgen.2021.100033\">CanDIG: Federated network across Canada for multi-omic and health data discovery and analysis<\/a> - 10 November, 2021 (co-author)<\/li>\n<li><a href=\"https:\/\/doi.org\/10.1101\/2021.03.30.434101\">CanDIG: Secure Federated Genomic Queries and Analyses Across Jurisdictions<\/a> -  March 31, 2021 (co-author)<\/li>\n<li><a href=\"https:\/\/doi.org\/10.1093\/nar\/gkv1222\">ClinVar: public archive of interpretations of clinically relevant variants<\/a> - November 17, 2015 (contributor)<\/li>\n<\/ul>\n"},{"title":"How to fail at STM32L433RC board with Embassy Rust embedded framework","published":"2022-12-16T00:00:00+00:00","updated":"2022-12-16T00:00:00+00:00","author":{"name":"\n            \n              Unknown\n            \n          "},"link":{"@attributes":{"rel":"alternate","type":"text\/html","href":"https:\/\/amanjeev.com\/blog\/stm32-embassy-rust-love\/"}},"id":"https:\/\/amanjeev.com\/blog\/stm32-embassy-rust-love\/","content":"<p>This blog post is about failures that ate two days of my time with STM32L433RC board with Rust.  I gave up many times and still not entirely back from the land of <em>given ups<\/em>.<\/p>\n<p>The story starts when I got <a href=\"https:\/\/www.st.com\/en\/microcontrollers-microprocessors\/stm32l433rc.html\">this beautiful board<\/a> to try out some Rust.  I am still learning embedded systems so even though I know some Rust, I struggle with the hardware part.  Datasheets for the hardware I use are really cool to read but they are hard for me to translate to Rust many times.<\/p>\n<p><img src=\"https:\/\/amanjeev.com\/blog\/stm32-embassy-rust-love\/.\/STM32L433RC.jpeg\" alt=\"STM32L433RC board with two LEDs turned on\" \/><\/p>\n<p>Before we go further -<\/p>\n<ul>\n<li><a href=\"https:\/\/github.com\/amanjeev\/stm32l433love\">the poopy code for this post is here<\/a><\/li>\n<li><a href=\"https:\/\/www.st.com\/en\/microcontrollers-microprocessors\/stm32l433rc.html\">the board is STM32L433RC<\/a><\/li>\n<\/ul>\n<h1 id=\"list-of-my-failure-modes\">List of my failure modes<\/h1>\n<h2 id=\"not-knowing-where-to-begin-with-rust-embedded\">Not knowing where to begin with Rust embedded<\/h2>\n<p>This is a classic!  That is, when I am excited about tinkering with hardware and no clue where to begin.  There are a ton of frameworks!  Friends' and experts' suggestion point to the latest Rust frameworks and libraries and sometimes it gets daunting.  I was pointed to <a href=\"https:\/\/embassy.dev\/\">Embassy<\/a>.  The cool part about this framework is that it is ground up support for Rust's <code>async\/await<\/code> in embedded world.  This <code>async\/await<\/code> style of programming allows for better resource allocation when there are IO blocking and wait times.<\/p>\n<p>Anyway, <a href=\"https:\/\/embassy.dev\/book\/dev\/index.html#_what_is_async\">Embassy docs explain about what is <code>async\/await<\/code><\/a> here.<\/p>\n<h2 id=\"dropping-the-framework-midway-and-jumping-to-another\">Dropping the framework midway and jumping to another<\/h2>\n<p>I made this another classic mistake.  I went down rabbit holes and lost track of how many <em>projects<\/em> I had created.<\/p>\n<h2 id=\"not-looking-at-the-examples-in-the-framework\">Not looking at the examples in the framework<\/h2>\n<p>Yes, I am told the best of the best also first head to the example directory to see how things are.  The only downside is that as a beginner, I do not have any prior knowledge to build upon.  Most embedded programmers can look at a new framework but they have a reference from prior experience with other frameworks etc.  I did not even remember to look at examples in at first.<\/p>\n<p>Anyway, <a href=\"https:\/\/github.com\/embassy-rs\/embassy\/tree\/master\/examples\/stm32l4\/src\/bin\">Embassy has some nice example programs for my board here<\/a>.<\/p>\n<h2 id=\"not-knowing-what-to-do-with-the-board\">Not knowing what to do with the board<\/h2>\n<p>I have no idea what I want to with the hardware but I like blinking lights.  That's where I start and perhaps most others too.  I am told that blinking lights are the <code>println!(\"Hello, world!\")<\/code> of the embedded world.  So, I did that.<\/p>\n<p>Anyway, I have now started to see that some of these boards have some sensors on them.  For example, I see that there is a temperature sensor on this board I am working with.<\/p>\n<h2 id=\"unable-to-figure-out-the-algorithms\">Unable to figure out the \"algorithms\"<\/h2>\n<p>Ok, I may have used the incorrect word here but by algorithms I mean correct steps to do a certain task.  For example, my task of reading the temperature sensor value continuously and print on the console.  Some of the examples of the things I had no clue about are -<\/p>\n<h3 id=\"how-does-pushing-a-program-i-am-writing-on-my-pc-to-the-board-work\">How does \"pushing\" a program I am writing on my PC, to the board work?<\/h3>\n<p>This is called <em>Flashing<\/em>.  The amazing thing with Rust embedded work is that you do not have to worry too much about it because <code>cargo<\/code> takes care of most of it.  Often you just need <a href=\"https:\/\/github.com\/amanjeev\/stm32l433love\/blob\/main\/.cargo\/config.toml\">a file <code>.cargo\/config.toml<\/code> in your project root<\/a> with some content like this -<\/p>\n<pre data-lang=\"toml\" style=\"background-color:#eff1f5;color:#4f5b66;\" class=\"language-toml \"><code class=\"language-toml\" data-lang=\"toml\"><span>[target.&#39;<\/span><span style=\"color:#a3be8c;\">cfg(all(target_arch = &quot;arm&quot;, target_os = &quot;none&quot;))<\/span><span>&#39;]\n<\/span><span style=\"color:#a7adba;\"># replace the chip below with your chip \n<\/span><span style=\"color:#a7adba;\">#  as listed in `probe-run --list-chips`\n<\/span><span style=\"color:#bf616a;\">runner <\/span><span>= &quot;<\/span><span style=\"color:#a3be8c;\">probe-run --chip STM32L433RCTxP<\/span><span>&quot;\n<\/span><span>\n<\/span><span>[build]\n<\/span><span style=\"color:#bf616a;\">target <\/span><span>= &quot;<\/span><span style=\"color:#a3be8c;\">thumbv7em-none-eabi<\/span><span>&quot;\n<\/span><span>\n<\/span><span>[env]\n<\/span><span style=\"color:#bf616a;\">DEFMT_LOG <\/span><span>= &quot;<\/span><span style=\"color:#a3be8c;\">trace<\/span><span>&quot;\n<\/span><\/code><\/pre>\n<p>Here the <code>target<\/code> is the architecture.  Using <code>rustup<\/code> works like <code>rustup target add thumbv7em-none-eabi<\/code> to add this target on your PC.<\/p>\n<h3 id=\"how-to-make-sure-we-tell-rust-the-right-environment\">How to make sure we tell Rust the right environment?<\/h3>\n<p>This is trickier because for STM32 boards at least you need <a href=\"https:\/\/github.com\/amanjeev\/stm32l433love\/blob\/main\/memory.x\">a special file called <code>memory.x<\/code> and you may have to do some magic<\/a> to put it in the right place during the build phase of your project.  I failed at this miserably but datasheet of your board can tell you the values you need to specify.<\/p>\n<pre style=\"background-color:#eff1f5;color:#4f5b66;\"><code><span>MEMORY\n<\/span><span>{\n<\/span><span>  \/* NOTE 1 K = 1 KiBi = 1024 bytes *\/\n<\/span><span>  \/* These values correspond to the STM32L433 *\/\n<\/span><span>  FLASH : ORIGIN = 0x08000000, LENGTH = 256K\n<\/span><span>  RAM : ORIGIN = 0x20000000, LENGTH = 64K\n<\/span><span>}\n<\/span><\/code><\/pre>\n<p>This is to tell Rust the memory regions on your board.  <a href=\"https:\/\/docs.rust-embedded.org\/book\/start\/hardware.html\">Rust embedded book has a page to clarify more on this memory.x stuff<\/a>.<\/p>\n<h3 id=\"what-are-the-peripherals-and-how-do-we-access-them\">What are the peripherals and how do we access them?<\/h3>\n<p>To know your peripherals you need to look at the datasheet.  How to access them via Rust depends on your framework but with Embassy it looks like this -<\/p>\n<pre data-lang=\"rust\" style=\"background-color:#eff1f5;color:#4f5b66;\" class=\"language-rust \"><code class=\"language-rust\" data-lang=\"rust\"><span style=\"color:#b48ead;\">let<\/span><span> p = embassy_stm32::init(Default::default());\n<\/span><\/code><\/pre>\n<h2 id=\"not-knowing-what-the-value-of-sensor-mean\">Not knowing what the value of sensor mean?<\/h2>\n<p>In my case the temperature sensor just spits out a number that Embassy framework's API serves as <code>u16<\/code> (if I understand any of this, that is the voltage). Usually, you have to pass it through some formula to get the actual temperature.  Datasheet for this board has the following -<\/p>\n<p><code>Temperature (in \u00b0C) = (TS_CAL2_TEMP \u2013 TS_CAL1_TEMP) \/ (TS_CAL2 \u2013 TS_CAL1) \u00d7 (TS_DATA \u2013 TS_CAL1) + 30<\/code><\/p>\n<p>Where:<\/p>\n<ul>\n<li><code>TS_CAL2<\/code> is the temperature sensor calibration value acquired at <code>TS_CAL2_TEMP<\/code>.<\/li>\n<li><code>TS_CAL1<\/code> is the temperature sensor calibration value acquired at <code>TS_CAL1_TEMP<\/code>.<\/li>\n<li><code>TS_DATA<\/code> is the actual temperature sensor output value converted by ADC. Refer to the device datasheet for more information about <code>TS_CAL1<\/code> and <code>TS_CAL2<\/code> calibration points.<\/li>\n<\/ul>\n<p>I do get some <em>cool<\/em> output LOL!<\/p>\n<pre data-lang=\"bash\" style=\"background-color:#eff1f5;color:#4f5b66;\" class=\"language-bash \"><code class=\"language-bash\" data-lang=\"bash\"><span style=\"color:#bf616a;\">cargo<\/span><span> run<\/span><span style=\"color:#bf616a;\"> --bin<\/span><span> temp\n<\/span><span>    <\/span><span style=\"color:#bf616a;\">Finished<\/span><span> dev <\/span><span style=\"color:#b48ead;\">[<\/span><span>unoptimized + debuginfo<\/span><span style=\"color:#b48ead;\">]<\/span><span> target(s) <\/span><span style=\"color:#bf616a;\">in<\/span><span> 0.07s\n<\/span><span>     <\/span><span style=\"color:#bf616a;\">Running <\/span><span>`<\/span><span style=\"color:#bf616a;\">probe-run --chip<\/span><span> STM32L433RCTxP target\/thumbv7em-none-eabi\/debug\/temp`\n<\/span><span>(<\/span><span style=\"color:#bf616a;\">HOST<\/span><span>) INFO  flashing program (60 pages \/ 60.00 KiB)\n<\/span><span>(<\/span><span style=\"color:#bf616a;\">HOST<\/span><span>) INFO  success!\n<\/span><span style=\"color:#bf616a;\">\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\n<\/span><span style=\"color:#bf616a;\">0.001861<\/span><span> internal is 64\n<\/span><span style=\"color:#bf616a;\">\u2514\u2500<\/span><span> temp::____embassy_main_task::{async_fn#0} @ src\/bin\/temp.rs:33\n<\/span><span style=\"color:#bf616a;\">0.005584<\/span><span> Temperature is: 57.070312\n<\/span><span style=\"color:#bf616a;\">\u2514\u2500<\/span><span> temp::____embassy_main_task::{async_fn#0} @ src\/bin\/temp.rs:41\n<\/span><span style=\"color:#bf616a;\">0.010131<\/span><span> Temperature is: 51.26953\n<\/span><span style=\"color:#bf616a;\">\u2514\u2500<\/span><span> temp::____embassy_main_task::{async_fn#0} @ src\/bin\/temp.rs:41\n<\/span><span style=\"color:#bf616a;\">0.014678<\/span><span> Temperature is: 46.757812\n<\/span><span style=\"color:#bf616a;\">\u2514\u2500<\/span><span> temp::____embassy_main_task::{async_fn#0} @ src\/bin\/temp.rs:41\n<\/span><span style=\"color:#bf616a;\">0.019226<\/span><span> Temperature is: 44.179688\n<\/span><span style=\"color:#bf616a;\">\u2514\u2500<\/span><span> temp::____embassy_main_task::{async_fn#0} @ src\/bin\/temp.rs:41\n<\/span><span style=\"color:#bf616a;\">...\n<\/span><\/code><\/pre>\n<h2 id=\"not-putting-the-correct-board-feature-in-cargo-toml\">Not putting the correct board \"feature\" in <code>Cargo.toml<\/code><\/h2>\n<p>Yes, when you include <code>embassy-stm32<\/code> in your <code>Cargo.toml<\/code> dependencies, you need to specify the correct board <em>ID<\/em>. In my case it was <code>stm32l433rc<\/code>.  <a href=\"https:\/\/github.com\/amanjeev\/stm32l433love\/blob\/main\/Cargo.toml#L13\">See my <code>Cargo.toml<\/code> for details<\/a>.<\/p>\n<h1 id=\"conclusion\">Conclusion<\/h1>\n<p>I have failed but this was so much fun!  I will continue playing.<\/p>\n"},{"title":"Rust in Science and ever-changing requirements","published":"2020-09-12T00:00:00+00:00","updated":"2023-11-23T00:00:00+00:00","author":{"name":"\n            \n              Unknown\n            \n          "},"link":{"@attributes":{"rel":"alternate","type":"text\/html","href":"https:\/\/amanjeev.com\/blog\/rust-in-science-and-ever-changing-requirements\/"}},"id":"https:\/\/amanjeev.com\/blog\/rust-in-science-and-ever-changing-requirements\/","content":"<p>I have heard many times over that for a given proof-of-concept\nif you have fast, changing requirements, then you are better off\nwith a <a href=\"https:\/\/en.wikipedia.org\/wiki\/Dynamic_programming_language\">Dynamic programming language<\/a>\nlike Python. Python gives the illusion of faster development\nbecause you do not have to think about the <em>rigidity<\/em> of the <a href=\"https:\/\/en.wikipedia.org\/wiki\/Type_system\">Type system<\/a>\nas much. Hence, it makes these dynamic languages good for prototyping and creating\nproofs-of-concept.<\/p>\n<p>However, in this essay, I am trying to dump some thoughts about\n<a href=\"https:\/\/www.rust-lang.org\/\">Rust<\/a> usage in scientific computation, its benefits,\nand generic chatter in the community. I think using Rust has an advantage in an\never-mutating environment like research and I think even for prototyping, Rust\ncan be much more beneficial than a language like Python.<\/p>\n<h2 id=\"introduction\">Introduction<\/h2>\n<p>I am very much in a love affair with Rust. This is not going to come to you as\na surprise at all if you <a href=\"https:\/\/twitter.com\/amanjeev\">follow me on Twitter<\/a>.\nPart of the reason is that I work as a software writer and general yak shaver\nfor science-(genomics)-adjacent work and I appreciate what Rust has to offer\nin terms of the ecosystem, safety, and speed. I'd argue that a lot of work that\nis usually written in C\/C++, Python, Perl, etc. could be replaced by Rust.<\/p>\n<p>Perhaps I do not need to spend a lot of effort to convince you that\nRust can replace some C\/C++ codebase more easily because Rust is a\n<a href=\"https:\/\/en.wikipedia.org\/wiki\/System_programming_language\">Systems programming language<\/a>.\nBut for proofs-of-concept, you ask, how can Rust be better than, say Python? Isn't it\npainful to change the code, which you do in early prototyping when you have to use a\nStatic language[1]?<\/p>\n<h2 id=\"the-case-of-python\">The case of Python<\/h2>\n<p>When I started in the sciences as a programmer, reading Perl was always an issue and the choice\nthat I had to make was clear to me \u2014 I chose Python. Perl was slowly being displaced by Python\nanyway. So, I will give you some anecdotal comparisons from my experiences.<\/p>\n<p>You might think that I am comparing apples to oranges and you are absolutely correct. This is\nan opinion blog post! I am choosing Python also because it is touted as the language\nwhich makes life easier to work with given ever-changing requirements.<\/p>\n<h2 id=\"ever-changing-requirements\">Ever-changing requirements<\/h2>\n<p>One issue that I always hear is the idea of \u201crefactoring\u201d and\/or \u201cchanging course\u201d.\nComputation in sciences, they say, is a lot of trial and error. I find this to be true! I have worked in\nnovel projects where even the stakeholders (scientists) were learning while we were building the\nprojects. There is a lot of backtracking and honestly a lot of your \"software engineering\" breaks\ndown.<\/p>\n<p>For this article, I am going to focus on some attributes of languages that instill\nconfidence in making the changes so that you can iterate. I believe the following attributes are necessary\nto work with constantly changing requirements \u2014<\/p>\n<ol>\n<li>Readability<\/li>\n<li>Testability<\/li>\n<li>Feedback<\/li>\n<li>Toolchain<\/li>\n<\/ol>\n<h3 id=\"readability\">Readability<\/h3>\n<h4 id=\"syntax\">Syntax<\/h4>\n<p>The clear advantage of Python is its syntactic readability if you are considerate of that while writing\n(but it does not always pan out!). This means that it is easier to keep the code in your head to make the model, while you\nare making changes. Rust can be hard to read because it does not fit the imperative, object-oriented style\nmental model we have with languages like Python. Rust\u2019s ownership model and typing definitely have some\nlearning curve.<\/p>\n<p>I have taught Git and Python to my colleagues and in many ways, they \u2014 and I \u2014 have had to\nbuild new mental models for these tools as well, especially if they are already familiar with C++, Perl, and SVN,\nCVS. Learning new mental models is what we do all the time. Still, I agree that this requires a lot of\neffort on the programmer\u2019s part. This is especially true for an experienced programmer.<\/p>\n<p>As <a href=\"https:\/\/twitter.com\/ekuber\">Esteban<\/a> says in his <a href=\"https:\/\/youtu.be\/Z6X7Ada0ugE\">RustConf 2020 talk \u2014 Bending the\nCurve: A Personal Tutor at Your Fingertips<\/a> \u2014<\/p>\n<blockquote>\n<p>Rust has a curse, it has many but this one is critical \u2014 inefficient code is generally visible.\nExperienced developers hate to notice that their code is inefficient.<\/p>\n<\/blockquote>\n<h4 id=\"jumping-around-the-codebase\">Jumping around the codebase<\/h4>\n<p>Rust is statically typed, empowers your editor or\n<a href=\"https:\/\/en.wikipedia.org\/wiki\/Integrated_development_environment\">Integrated Development Environment (IDE)<\/a>\nto link the symbols in your code for easy access. A function defined somewhere in a <code>struct<\/code> can\nbe easily found from where it is called. This makes looking up code for dependencies much easier\nand faster.<\/p>\n<p>The only tool in Python that brings us any closer to this style of working is\n<a href=\"https:\/\/www.jetbrains.com\/pycharm\/\">JetBrains PyCharm<\/a>. And even this IDE for Python\nfails to look up <em>symbols<\/em> if you mess up your\n<a href=\"https:\/\/docs.python.org\/3\/tutorial\/venv.html\"><code>virtualenv<\/code><\/a> or fail to register it\nwith your PyCharm project. You can use <a href=\"https:\/\/docs.python.org\/3\/library\/typing.html\"><code>typing<\/code><\/a>\nto annotate your code with the types but as you can see that there is a warning at the top of\n<a href=\"https:\/\/docs.python.org\/3\/library\/typing.html\">the <code>typing<\/code> documentation<\/a> -<\/p>\n<blockquote>\n<p>Note: The Python runtime does not enforce function and variable type annotations.\nThey can be used by third-party tools such as type checkers, IDEs, linters, etc.<\/p>\n<\/blockquote>\n<p>With Rust, you can survive with just the compiler because these types are a part of the\nlanguage. Add <a href=\"https:\/\/github.com\/rust-lang\/rls\">Rust Langauge Server (RLS)<\/a> to that and\nyou can comfortably navigate your codebase.<\/p>\n<h4 id=\"docs\">Docs!<\/h4>\n<p>Since Rust is statically typed, you can see what is the expected type of each function\nargument or what is the type of the value that the function returns. Its type system\nbecomes part of the documentation! Whereas in Python you have to use external tools and\ndepend on function annotations to generate documentation,\n<a href=\"https:\/\/doc.rust-lang.org\/rustdoc\/what-is-rustdoc.html\">Rust brings the documentation to you via <code>rustdoc<\/code><\/a>.<\/p>\n<p>For example, please compare<\/p>\n<pre data-lang=\"python\" style=\"background-color:#eff1f5;color:#4f5b66;\" class=\"language-python \"><code class=\"language-python\" data-lang=\"python\"><span style=\"color:#b48ead;\">def <\/span><span style=\"color:#8fa1b3;\">chillin<\/span><span>(<\/span><span style=\"color:#bf616a;\">name<\/span><span>, <\/span><span style=\"color:#bf616a;\">place<\/span><span>):\n<\/span><span>    <\/span><span style=\"color:#a7adba;\">&quot;&quot;&quot;Informs what&#39;s the user&#39;s chillin&#39; number\n<\/span><span style=\"color:#a7adba;\">\n<\/span><span style=\"color:#a7adba;\">    @param name: string, name of the user\n<\/span><span style=\"color:#a7adba;\">    @param palce: string, where the user is chillin&#39;\n<\/span><span style=\"color:#a7adba;\">    @return: int, information\n<\/span><span style=\"color:#a7adba;\">    &quot;&quot;&quot;\n<\/span><span>    result = <\/span><span style=\"color:#d08770;\">0\n<\/span><span>    <\/span><span style=\"color:#b48ead;\">if <\/span><span>place == &quot;<\/span><span style=\"color:#a3be8c;\">toronto<\/span><span>&quot; {\n<\/span><span>        result = <\/span><span style=\"color:#d08770;\">100\n<\/span><span>    }\n<\/span><span>    <\/span><span style=\"color:#b48ead;\">return <\/span><span>result\n<\/span><\/code><\/pre>\n<p>with<\/p>\n<pre data-lang=\"rust\" style=\"background-color:#eff1f5;color:#4f5b66;\" class=\"language-rust \"><code class=\"language-rust\" data-lang=\"rust\"><span style=\"color:#a7adba;\">\/\/\/ Create an alias to integer type\n<\/span><span style=\"color:#a7adba;\">\/\/\/ Now it is much clearer what the return type \n<\/span><span style=\"color:#a7adba;\">\/\/\/  of the function stands for\n<\/span><span style=\"color:#b48ead;\">type <\/span><span>ChillinNum = <\/span><span style=\"color:#b48ead;\">u32<\/span><span>;\n<\/span><span>\n<\/span><span style=\"color:#a7adba;\">\/\/\/ Informs what&#39;s the user&#39;s chillin&#39; number\n<\/span><span style=\"color:#a7adba;\">\/\/\/ * `name`: name of the user\n<\/span><span style=\"color:#a7adba;\">\/\/\/ * `place`: where the user is chillin&#39;\n<\/span><span style=\"color:#b48ead;\">fn <\/span><span style=\"color:#8fa1b3;\">chillin<\/span><span>(<\/span><span style=\"color:#bf616a;\">name<\/span><span>: String, <\/span><span style=\"color:#bf616a;\">place<\/span><span>: &amp;<\/span><span style=\"color:#b48ead;\">str<\/span><span>) -&gt; ChillinNum {\n<\/span><span>    <\/span><span style=\"color:#b48ead;\">match<\/span><span> place {\n<\/span><span>        &quot;<\/span><span style=\"color:#a3be8c;\">toronto<\/span><span>&quot; =&gt; <\/span><span style=\"color:#b48ead;\">return <\/span><span style=\"color:#d08770;\">100<\/span><span>,\n<\/span><span>        _ =&gt; <\/span><span style=\"color:#b48ead;\">return <\/span><span style=\"color:#d08770;\">0<\/span><span>,\n<\/span><span>    }\n<\/span><span>}\n<\/span><\/code><\/pre>\n<p>It is a tiny example and while Rust's syntax is denser, it also provides clarity on what\nthe elements stand for. Here, <code>ChillinNum<\/code> is the type of the return value. We could have just\nused <code>u32<\/code> but using <code>ChillinNum<\/code> is clearer. If you run <code>cargo doc<\/code> in your codebase,\nRust compiler will take all these\ntriple-slash <code>\/\/\/<\/code> comments and generate nice documentation for you. You can see the\n<a href=\"https:\/\/docs.rs\/eyre\/0.6.0\/eyre\/\">documentation of the crate <code>eyre<\/code><\/a> and the\n<a href=\"https:\/\/github.com\/yaahc\/eyre\/blob\/master\/src\/lib.rs\">source of the crate <code>eyre<\/code> that generates it<\/a>.<\/p>\n<h3 id=\"testability\">Testability<\/h3>\n<p>Python has a ton of testing frameworks and Rust is slowly catching up. A clear advantage in favor\nof Python. However, the things you are testing also dictate how much confidence you have in your\ncodebase. With the type system in Rust, you do not have to test for certain cases that in Python\nyou\u2019d have to. This means there are certain test cases that you do not need to write if you\u2019re\nbuilding your project in Rust. To me, this makes the iteration faster with more confidence in\nthe code I am writing.<\/p>\n<p>As <a href=\"https:\/\/twitter.com\/ekuber\">Esteban<\/a> puts it -<\/p>\n<blockquote>\n<p>The reduced need for tests is because of using patterns that leverage the type system\nto completely eliminate the representability of an invalid state.<\/p>\n<\/blockquote>\n<p>Take Rust's <code>match<\/code> pattern, as an example. The following code will fail to compile because\nwe failed to provide a catch-all -<\/p>\n<pre data-lang=\"rust\" style=\"background-color:#eff1f5;color:#4f5b66;\" class=\"language-rust \"><code class=\"language-rust\" data-lang=\"rust\"><span style=\"color:#a7adba;\">\/\/\/ Create an alias to integer type\n<\/span><span style=\"color:#a7adba;\">\/\/\/ Now it is much clearer what the return type \n<\/span><span style=\"color:#a7adba;\">\/\/\/  of the function stands for\n<\/span><span style=\"color:#b48ead;\">type <\/span><span>ChillinNum = <\/span><span style=\"color:#b48ead;\">u32<\/span><span>;\n<\/span><span>\n<\/span><span style=\"color:#a7adba;\">\/\/\/ Informs what&#39;s the user&#39;s chillin&#39; number\n<\/span><span style=\"color:#a7adba;\">\/\/\/ * `name`: name of the user\n<\/span><span style=\"color:#a7adba;\">\/\/\/ * `place`: where the user is chillin&#39;\n<\/span><span style=\"color:#b48ead;\">fn <\/span><span style=\"color:#8fa1b3;\">chillin<\/span><span>(<\/span><span style=\"color:#bf616a;\">name<\/span><span>: String, <\/span><span style=\"color:#bf616a;\">place<\/span><span>: &amp;<\/span><span style=\"color:#b48ead;\">str<\/span><span>) -&gt; ChillinNum {\n<\/span><span>    <\/span><span style=\"color:#b48ead;\">match<\/span><span> place {\n<\/span><span>        &quot;<\/span><span style=\"color:#a3be8c;\">toronto<\/span><span>&quot; =&gt; <\/span><span style=\"color:#b48ead;\">return <\/span><span style=\"color:#d08770;\">100<\/span><span>,\n<\/span><span>    }\n<\/span><span>}\n<\/span><\/code><\/pre>\n<p>Previously, we had provided the default\/catch-all case <code>_ =&gt; return 0,<\/code>. Similarly, the Rust compiler\nwill complain if you are not covering all the variants of a Rust <code>enum<\/code> in your <code>match<\/code> pattern.\nThis helps the programmer in considering all the paths and cases. The compiler will show an error like this -<\/p>\n<pre style=\"background-color:#eff1f5;color:#4f5b66;\"><code><span>error[E0004]: non-exhaustive patterns: `&amp;_` not covered\n<\/span><span>  --&gt; src\/main.rs:12:11\n<\/span><span>   |\n<\/span><span>12 |     match place {\n<\/span><span>   |           ^^^^^ pattern `&amp;_` not covered\n<\/span><span>   |\n<\/span><span>   = help: ensure that all possible cases are being handled, possibly by adding wildcards or more match arms\n<\/span><span>   = note: the matched value is of type `&amp;str`\n<\/span><\/code><\/pre>\n<p>I do not enjoy writing tests and eating my veggies but if I could eat fewer veggies to achieve the\nsame level of confidence in my changes, I\u2019d always choose that.<\/p>\n<p>This also brings me to the point of designing your project. Just because your project is in\nalpha\/beta\/whateva, does not mean that you are not allowed to think about your data structures,\nyour types before the implementation. A little bit of thought in the structure of your codebase\ngoes a long way, especially when you have to change it. Once again, listen to <a href=\"https:\/\/twitter.com\/ekuber\">Esteban<\/a> -<\/p>\n<blockquote>\n<p>Thinking about the API surface first and then changing the internal logic and in-memory\nrepresentation to make it faster\/more efficient is much easier in type-safe languages than in\ndynamic languages, and more so in Rust simply because more things are represented in the type\nsystem than is customary in other languages. This makes \"iteratively fixing the compiler errors\"\na valid refactoring strategy.<\/p>\n<\/blockquote>\n<p>These points, to me, sound like Rust has more advantages than Python if we count\nTesting as a property of confidence in the changes we are making.<\/p>\n<h3 id=\"feedback\">Feedback<\/h3>\n<p>Even though Rust has its <em>new<\/em> paradigm issues[2] when it comes to learning the language, I still\nfeel that Rust compiler emits some of the most helpful error messages and help messages. The safety\nfeatures are built right into the compiler and the team has done a fantastic job so far in making\nmessages ergonomic. For example, in Rust, the array out of bounds error looks like this -<\/p>\n<pre data-lang=\"rust\" style=\"background-color:#eff1f5;color:#4f5b66;\" class=\"language-rust \"><code class=\"language-rust\" data-lang=\"rust\"><span style=\"color:#b48ead;\">let<\/span><span> x: [<\/span><span style=\"color:#b48ead;\">u8<\/span><span>; <\/span><span style=\"color:#d08770;\">3<\/span><span>] = [<\/span><span style=\"color:#d08770;\">10<\/span><span>; <\/span><span style=\"color:#d08770;\">3<\/span><span>];    \n<\/span><span>println!(&quot;<\/span><span style=\"color:#d08770;\">{:?}<\/span><span>&quot;, x[<\/span><span style=\"color:#d08770;\">4<\/span><span>]);\n<\/span><\/code><\/pre>\n<pre data-lang=\"rust\" style=\"background-color:#eff1f5;color:#4f5b66;\" class=\"language-rust \"><code class=\"language-rust\" data-lang=\"rust\"><span>error: this operation will panic at runtime\n<\/span><span>  --&gt; src\/main.rs:<\/span><span style=\"color:#d08770;\">23<\/span><span>:<\/span><span style=\"color:#d08770;\">22\n<\/span><span>   |\n<\/span><span style=\"color:#d08770;\">23 <\/span><span>|     println!(&quot;<\/span><span style=\"color:#d08770;\">{:?}<\/span><span>&quot;, x[<\/span><span style=\"color:#d08770;\">4<\/span><span>]);\n<\/span><span>   |                      ^^^^^ index out of <\/span><span style=\"color:#bf616a;\">bounds<\/span><span>: the len is 3 but the index is 4\n<\/span><span>   |\n<\/span><span>   = note: `#[<\/span><span style=\"color:#bf616a;\">deny<\/span><span>(unconditional_panic)]` on by default\n<\/span><\/code><\/pre>\n<p>For Python, this looks like this -<\/p>\n<pre data-lang=\"python\" style=\"background-color:#eff1f5;color:#4f5b66;\" class=\"language-python \"><code class=\"language-python\" data-lang=\"python\"><span>x = [<\/span><span style=\"color:#d08770;\">10<\/span><span>,<\/span><span style=\"color:#d08770;\">10<\/span><span>,<\/span><span style=\"color:#d08770;\">10<\/span><span>]\n<\/span><span style=\"color:#96b5b4;\">print<\/span><span>(x[<\/span><span style=\"color:#d08770;\">4<\/span><span>])\n<\/span><\/code><\/pre>\n<pre data-lang=\"python\" style=\"background-color:#eff1f5;color:#4f5b66;\" class=\"language-python \"><code class=\"language-python\" data-lang=\"python\"><span>    File &quot;<\/span><span style=\"color:#a3be8c;\">app.py<\/span><span>&quot;, line <\/span><span style=\"color:#d08770;\">3<\/span><span>, in &lt;module&gt;\n<\/span><span>        <\/span><span style=\"color:#96b5b4;\">print<\/span><span>(x[<\/span><span style=\"color:#d08770;\">4<\/span><span>])\n<\/span><span>IndexError: list index out of <\/span><span style=\"color:#96b5b4;\">range\n<\/span><\/code><\/pre>\n<p>We've already seen an error message above for missing catch-all but let me show that again -<\/p>\n<pre style=\"background-color:#eff1f5;color:#4f5b66;\"><code><span>error[E0004]: non-exhaustive patterns: `&amp;_` not covered\n<\/span><span>  --&gt; src\/main.rs:12:11\n<\/span><span>   |\n<\/span><span>12 |     match place {\n<\/span><span>   |           ^^^^^ pattern `&amp;_` not covered\n<\/span><span>   |\n<\/span><span>   = help: ensure that all possible cases are being handled, possibly by adding wildcards or more match arms\n<\/span><span>   = note: the matched value is of type `&amp;str`\n<\/span><\/code><\/pre>\n<p>Side note: It is amazing that languages can and do learn from each other and copy parts that seem\nhelpful for the users. One recent exciting case is <a href=\"https:\/\/devblogs.microsoft.com\/cppblog\/new-safety-rules-in-c-core-check\/\">New safety rules in C++ Core\nCheck<\/a>. I am very happy to have other\nlanguages learn, as Rust has from what came before.<\/p>\n<p>I hope you can appreciate the effort being put into the compiler and its error messages. Once\nagain, <a href=\"https:\/\/twitter.com\/ekuber\">Esteban<\/a>\u2019s talk is a fantastic watch,\n<a href=\"https:\/\/www.youtube.com\/embed\/Z6X7Ada0ugE\">where he talks about the Rust compiler as a tutor<\/a>.<\/p>\n<blockquote>\n<p>When we are emitting diagnostic errors, it is the perfect place and moment to teach\npeople [that] they have made a mistake and we can explain to them why they made it.<\/p>\n<\/blockquote>\n<p>I will embed the talk here for you \u2014<\/p>\n<div>\n    <iframe\n        src=\"https:\/\/www.youtube.com\/embed\/Z6X7Ada0ugE\"\n        webkitallowfullscreen\n        mozallowfullscreen\n        allowfullscreen>\n    <\/iframe>\n<\/div>\n<p>Python, on the other hand, had me stuck for a whole day because it complained\nsomewhere I was scripting <code>NoneType<\/code>. If you care, this is the pet peeve that started this blog post.<\/p>\n<h3 id=\"toolchain\">Toolchain<\/h3>\n<p>In the Python world, the packaging and environment setup is still something that makes me sad. The\nclosest I have come to create some sanity in my life is to use <code>direnv<\/code> and <code>shell.nix<\/code> in my\nrepositories. This sucks because this is a solution that is at the OS (NixOS) level[3].\nIndeed there are tools available for auto-loading environments, and I am thankful for that.\nHowever, Python toolset feels like a moving target.\nYears ago, it was just <code>setup.py<\/code> and today it is <code>setup.cfg<\/code>, <code>pyproject.toml<\/code> as well but\nthe latter two do not support all the features so you end up having <code>setup.py<\/code> in your codebase.\nHere, <a href=\"https:\/\/snarky.ca\/what-the-heck-is-pyproject-toml\/\">let Tall, Snarky Canadian explain it to you<\/a>.<\/p>\n<p>On the Rust side though, the ease of use of <code>cargo<\/code> and <code>rustup<\/code> has not been beaten for me,\nso far. The Rust ecosystem is young and has the advantage of learning from other platforms.<\/p>\n<p>Something else that made me happy about Rust is the promise of backward compatibility. Most of us\nhave been burnt with Python 2 to 3 move although I am in awe of the Python community that undertook\nsuch a massive effort and made it happen.<\/p>\n<p>The biggest disadvantage in Python that I see is that if I move a module out of a package, it is\nmuch harder for my editor to figure out the details of my refactoring. PyCharm has made my life\nmuch easier but there are cases where it fails because it can do only so much with a dynamic language.\nThis is obvious because Python is a dynamic language. My good friend, <a href=\"https:\/\/twitter.com\/ekuber\">Esteban<\/a>\nonce again nails it -<\/p>\n<blockquote>\n<p>The kind of dynamism that Python has, where almost everything\nyou write is valid code, can be a behavior you did not want, makes it harder to write a Python\ninterpreter that helps you get back on track. Rust leverages its rigid syntax to simultaneously\nkeep the language simpler than it otherwise would be and to make it easier to figure out what\nthe user was actually intending to do.<\/p>\n<\/blockquote>\n<p>While this does allow us to write code without thinking too much about types, it does not give\nus a lot of confidence when we are changing the code. This means that one can likely write much\nfaster in Python, but to change the code for new requirements is much harder because it is likely\nto be error-prone.<\/p>\n<h2 id=\"not-all-is-ready-yet\">Not all is ready, yet<\/h2>\n<p>I do not think that a piece should be entirely replaced, just because it can be.\nSome tools being used are more mature than Rust is at the moment. However, there\nare clear benefits to be had. I am happy to report that I am <a href=\"https:\/\/benjamin.computer\/posts\/2019-07-31-rust-research.html\">not the only one who\nfeels<\/a> <a href=\"https:\/\/blog.luizirber.org\/2018\/08\/23\/sourmash-rust\/\">this way in (some of)\nthe<\/a>\n<a href=\"https:\/\/rust-bio.github.io\/\">community<\/a>.<\/p>\n<p>There is some way to go with the data structures and libraries etc. but I think it is a\nchicken-and-egg problem and as the adoption increases there will be more libraries available. <a href=\"https:\/\/doc.rust-lang.org\/nomicon\/ffi.html\">The\nbest part is that you can do piecemeal work by delegating parts of your codebase to Rust using\nFFI<\/a>.\n<a href=\"https:\/\/blog.luizirber.org\/2020\/01\/10\/sourmash-pr\/\">Let Luiz explain it to you how he works in his research with Rust and Python!<\/a><\/p>\n<h2 id=\"static-for-dynamicity\">Static for dynamicity<\/h2>\n<p>This is where I would conclude that if you are starting a journey and are sure that the things will\nchange many times over, you may be better off with a language that gives you<\/p>\n<ul>\n<li>a solid structure to build upon with<\/li>\n<li>tools to warn you when parts of that structures are moved<\/li>\n<li>by giving your advice on \u201cwhy\u201d, \u201cwhere\u201d and \u201chow\u201d to do that<\/li>\n<\/ul>\n<p>I also conclude that you could leverage Rust to replace parts of your project that\nrequire more safety and speed.<\/p>\n<p>I enjoy writing Python because of its syntax but I keep missing the Rust compiler's help every day.<\/p>\n<p>Once again, this article would not have been possible without the help and guidance of the following\nfriends -<\/p>\n<ul>\n<li><a href=\"https:\/\/hoverbear.org\/\">Ana Hobden<\/a><\/li>\n<li><a href=\"https:\/\/twitter.com\/ekuber\">Esteban K\u00fcber<\/a><\/li>\n<li>Shveta Kumar<\/li>\n<\/ul>\n<p>[1] I am sorry but I am taking a generic leeway to combine Dynamic languages with Dynamically\ntyped languages and Static languages with Statically typed languages<\/p>\n<p>[2] They are rather non-mainstream because Rust <em>borrows<\/em> almost everything from other languages\nbut you get the idea<\/p>\n<p>[3] If you want to know how to manage your work environment for a project using NixOS and <code>direnv<\/code>, you can read\nmy small blog post about\n<a href=\"https:\/\/amanjeev.com\/blog\/nix-shell-manage-project-dependencies\">Environment management with nix-shell<\/a>.<\/p>\n"},{"title":"A year in CanDIG project","published":"2020-07-06T00:00:00+00:00","updated":"2020-07-06T00:00:00+00:00","author":{"name":"\n            \n              Unknown\n            \n          "},"link":{"@attributes":{"rel":"alternate","type":"text\/html","href":"https:\/\/amanjeev.com\/blog\/one-year-at-candig\/"}},"id":"https:\/\/amanjeev.com\/blog\/one-year-at-candig\/","content":"<p>Last month, I completed a year in my current project -\n<a href=\"https:\/\/www.distributedgenomics.ca\/\">CanDIG<\/a>. This project is about building\na platform to connect \"data lakes\" within Canadian institutions in a federated\nmanner. I was hired to mostly do <a href=\"https:\/\/www.distributedgenomics.ca\/posts\/gentle-intro-to-federated-distributed-authz\/index.html\">federated authorization, which you can read\nabout in this blog post on CanDIG's blog<\/a>.<\/p>\n<p>Those who do not know, I have spent the past few years\nin the Genomics area but mostly on the side of software infrastructure. Some of\nthe Biology and Genomics knowledge that I have is courtesy\n<a href=\"https:\/\/en.wikipedia.org\/wiki\/Donna_R._Maglott\">Donna Maglott<\/a> at NCBI.<\/p>\n<p>However, I left United States for Canada, once I found a project that I\nwas excited about, and of course, never again have to worry about H1B Visa\nor the immigration system in US. Having worked a year in the team, I have learned\na few things in general that I thought are worth sharing.<\/p>\n<h2 id=\"value-matching-with-the-team-and-project\">Value matching with the team and project<\/h2>\n<p>Even though I have mostly worked solo due to the nature of AAI work,\nit is a tremendous help to work in a team that matches your values.\nCollaboration with other colleagues on projects that overlap with my goals makes me happy.\nFor example, on the \"Dynamic Consents\" project with an esteemed colleague that allows for\npatients to give their consent, and change the consent (either granularity or revocation)\nif they want while taking into account aspects like the accessibility of remote communities\nin Canada. I love the fact that our team is taking into account various social issues\nthat hinder marginalized communities' participation, representation, and ownership of their data.<\/p>\n<h2 id=\"love-hate-relationship-with-software-development-in-science\">Love-Hate relationship with software development in science<\/h2>\n<p>At the time of this writing, there has been a lot of improvement in\nscientific computing and software development. There are\norganizations, like <a href=\"https:\/\/researchsoftware.org\/\">RSE<\/a>, that are helping move\nthe state of the scientific software to a better place. However, there is a lot\nthat remains to be desired. This is also good news for people like me because\nthat allows us to voice the concerns and keep our job.<\/p>\n<p>I also believe that most of the time, as a scientific software developer\nyou can work on a variety of projects and tasks which keep your tech-life\nless monotonous. I understand, that for some people, shaving Python yaks is\nalso monotonous but as long as I get leeway to learn and grow, I am happy.<\/p>\n<p>The part that I am most concerned about (still) is licensing. A lot of professors\nin a lot of places are not happy with Copyleft licenses and I am not sure why. I\nhave written a <a href=\"https:\/\/amanjeev.com\/blog\/scientific-software-license\">blog post about this question on Copyleft licenses in scientific\nsoftware<\/a>. I still do not\nhave a good answer as to why we should avoid Copyleft licenses when they serve the Science\nmore than other licenses, representing the true spirit of Science.<\/p>\n<h2 id=\"don-t-roll-your-own-crypto\">Don't roll your own crypto<\/h2>\n<p>A lot of work within the group was related to AAI and there are a bunch of things\nthat you can do by yourself. However, there are likely tools available that you can leverage\nwithout having to write those on your own. The point is that the quote \"don't roll\nyour own crypto\" is not just for the crypto algorithms or PKI\netc. It can be for small things like releasing or signing <a href=\"https:\/\/en.wikipedia.org\/wiki\/JSON_Web_Token\">JWT<\/a>s.\nYes, you read that right - JWTs! Please don't judge, we need JWTs because we use OIDC (more later).\nIt is super easy to slip into \"build your own\" for smaller things. So far, we've avoided\nthat by using Keycloak, Tyk, and libraries for differential privacy, etc. and hopefully will\ncontinue on that trajectory by using products like <a href=\"https:\/\/www.vaultproject.io\/\">Hashicorp Vault<\/a>.<\/p>\n<h2 id=\"federated-distributed-and-other-overloaded-terms\">Federated, Distributed and other overloaded terms<\/h2>\n<p>I never thought I'd even look at something like <a href=\"https:\/\/openid.net\/connect\/\">OIDC<\/a> in my career\nbut here I am. I have now achieved two things in the year -<\/p>\n<ol>\n<li>Perform cross-atlantic (Europe+Canada) connection of resources (e.g. genomic services)\nwhich allow for both sides to log into each other's instances and be authorized at different\nlevels just by using OIDC. This could be called federated authorization.<\/li>\n<li>CanDIG's first draft of authorization, which is a distributed infrastructure that connects multiple sites within Canada\nto share genomic data. This authorization can also be called federated authorization.<\/li>\n<\/ol>\n<p>The European infrastructure is much more centralized so the challenges in tying them with\nour infrastructure are different than what we are building within CanDIG and Canada. What we\nare doing within CanDIG is much more difficult because we certainly do not have the leeway\nthat a centralized system has, where an alias of the external user can be created to facilitate\ncommunication. The terms are <em>definitely<\/em> overloaded!<\/p>\n<p>But as you can see, the issue is that we need to use a protocol that -<\/p>\n<ul>\n<li>supports complex authorization and permission schemes (don't roll your eyes)<\/li>\n<li>can be extended with custom permissions and claims<\/li>\n<li>is open<\/li>\n<li>supported in a wide variety of cases and products<\/li>\n<li>is documented (enough)<\/li>\n<li>can be agreed upon by scientists from across the globe<\/li>\n<\/ul>\n<p>Much of this work is being done in <a href=\"https:\/\/www.cineca-project.eu\/\">CINECA project<\/a>\nand a lot of input to and from <a href=\"https:\/\/www.ga4gh.org\/\">GA4GH<\/a>. There are very few\nprotocols that come close to what exists in the form of OIDC. Let me be clear, I hate\nit. However, I do not have much say and I do not have any suggestions either. If you are\nsaying \"Build your own\" then please see the previous point. It is ridiculously hard to\ncome to consensus in any such scientific gathering and then there is the question of $$$.<\/p>\n<p>And I <em>have<\/em> looked into https:\/\/oauth.xyz\/ and no it is not ready yet and I do not\neven know if it is going to be any better, yet. Complexity breeds issues but some of\nthe problems at-hand require complex solutions. Sure, OIDC is complicated as well.<\/p>\n<p>I appreciate you read so far what seems like a brain dump of last year. Here's to another\nyear with a focus on systems, Rust, privacy, and federation!<\/p>\n"},{"title":"My process for bibliography and notes","published":"2020-06-24T00:00:00+00:00","updated":"2020-06-24T00:00:00+00:00","author":{"name":"\n            \n              Unknown\n            \n          "},"link":{"@attributes":{"rel":"alternate","type":"text\/html","href":"https:\/\/amanjeev.com\/blog\/bibliography-and-notes\/"}},"id":"https:\/\/amanjeev.com\/blog\/bibliography-and-notes\/","content":"<p>For those who are starting new in research, you might want to keep running notes\nand citations for the relavant research papers, journals, books etc. that you\ncome across. This helps maintain clarity of what is important, what has been\nread already and the notes that help you revisit the material if and when you\nneed to.<\/p>\n<p>After a decade and half worth of haitus, I am starting to delve into research\nonce again. This means reading papers more than I am used to. During my tenure\nas software engineer in various organizations, most papers I had read were\nlimited and I could afford to read them as many times as I wanted.<\/p>\n<p>Things have changed now and I need tools and new habits to help me navigate\nthrough a ton of literature. I cannot afford to spend countless hours on one\nparagraph in one paper. Above all, I cannot afford to think that all papers\nand books are created equal and have the same value for my research and\neventual PoC that I have to deliver.<\/p>\n<p>Here are a few things I am trying to achieve<\/p>\n<ol>\n<li>Forming new habits\n<ul>\n<li>ability to skim papers<\/li>\n<li>jotting down quick notes for the papers<\/li>\n<li>peruse references if interesting and get those references<\/li>\n<li>revist the papers for deeper read if needed<\/li>\n<\/ul>\n<\/li>\n<li>A way to manage a \"to-read\" list of papers<\/li>\n<li>A way to manage \"already-read\" list of papers<\/li>\n<li>Tag papers that need revisiting for deeper read<\/li>\n<\/ol>\n<p>As you can tell, I do not want a mere list of citations that I have read for\nmy research and projects. I want a traceable way to know what I read, as well\nas why it was important. Here is an approach I am starting with -<\/p>\n<h2 id=\"start-with-a-book-on-the-research-if-available-skim-it-first\">Start with a book on the research, if available. Skim it first.<\/h2>\n<p>A book allows easy introduction to the topic without having to figure out\nwhich papers to read first on that topic. If a decent book is not available\nfor the field\/topic, there might be a way to start with early review papers\nor first papers.<\/p>\n<p>Note: At the time of writing, I am not aware of anything else to alleviate\nsome starting pains.<\/p>\n<h3 id=\"why-skim\">Why skim<\/h3>\n<p>It is critical that I skim the book to break the habit of deep reading in first\npass. If I am spending too much time going deep and trying to retain everything\nI am losing out on getting a broad idea of the topic at hand. I need to be have\nbroad knowledge as soon as possible so I can then drill down into specifics.<\/p>\n<h2 id=\"keep-short-notes\">Keep short notes<\/h2>\n<p>Skimming does not mean that you avoid taking notes. You aren't reading a novel\nafter all (although I have been known to keep notes on novels). Keeping such\nnotes allows you to come back to them to decide whether something was worthwhile\nand requires deeper reading. I try to keep notes of the following things:<\/p>\n<ul>\n<li>Chapter names and general idea<\/li>\n<li>List of topics each chapter covers<\/li>\n<li>Mark\/tag a chapter important or to revisit<\/li>\n<li>List of reference from each chapter that you want to fetch<\/li>\n<\/ul>\n<p>These notes enhance your bibliography.<\/p>\n<h2 id=\"next\">Next<\/h2>\n<p>If you are comfortable at the end, maybe it is worthwhile to revisit the\nchapters you've marked and fetching the references. I personally fetch all\nthe references at the end of each chapter, if I can. At this time, mark all\nthe papers and chapters yet again if they need revisiting. Tag them further\nto make sure you can search for them later when you need them.<\/p>\n<p>I find keeping at most three tags just enough to have entropy under control,\nat the same time not losing my sould in tagging everything.<\/p>\n<blockquote>\n<p>Remember, the goal is to manage the items and related notes to quickly make a\ndecision on important aspects of the topic at hand.<\/p>\n<\/blockquote>\n<p>Another cycle of skimming, taking notes, and moving to next starts!<\/p>\n<p>I hope this will help me build an intuition by helping with new habits\nof swimming through the giant corpus of any topic that I want to study.<\/p>\n<h2 id=\"tools\">Tools<\/h2>\n<h3 id=\"save-and-synchronize-papers\">Save and synchronize papers<\/h3>\n<p>I started using <a href=\"https:\/\/www.zotero.org\/\">Zotero<\/a> and it has been working fine so far. Some of the\nkey features that I find useful -<\/p>\n<ul>\n<li>Easy, one-click fetching and saving of papers (browser plugins available)<\/li>\n<li>Ability to tag them as well as create \"collections\" to manage files<\/li>\n<li>Ability to add notes to the papers<\/li>\n<li>Syncrhonizing your papers list, notes and everything under the Sun<\/li>\n<li>Works on Linux!<\/li>\n<li>Works with third-party apps on mobile devices and tablets<\/li>\n<li>Advanced search that you can save<\/li>\n<li>Ability to generate ciatations and reports from each item<\/li>\n<li>You can export your data<\/li>\n<li>Free for upto 300MB of storage for synchronization<\/li>\n<\/ul>\n<p>On my iPad, where I do most of the reading, I use <a href=\"https:\/\/www.papershipapp.com\/\">PaperShip<\/a> with <a href=\"https:\/\/www.zotero.org\/\">Zotero<\/a>\naccount.<\/p>\n<h3 id=\"search-and-discover-papers\">Search and discover papers<\/h3>\n<p>A few places I go to, to search papers and to discover new papers.<\/p>\n<ul>\n<li><a href=\"http:\/\/arxiv.org\/\">arXiv<\/a> - a preprint to find stuff sooner<\/li>\n<li><a href=\"https:\/\/github.com\/karpathy\/arxiv-sanity-preserver\">arXiv Sanity<\/a> - a self hosted private instance for my areas of interest<\/li>\n<li><a href=\"https:\/\/www.semanticscholar.org\/\">Semantic Scholar<\/a> - Also allows you to create feeds for papers of interest<\/li>\n<\/ul>\n<h3 id=\"notes-and-tags\">Notes and tags<\/h3>\n<p>I add quick notes in Zotero as well as use local Emacs+Markdown and push the\nnotes to GitHub (or <a href=\"https:\/\/sourcehut.org\/\">SourceHut<\/a> if you hate GitHub because of ICE).<\/p>\n<p>I save papers in particular collections in Zotero and tag the papers frequently.\nFor example, I tag \"done-reading\" when I am done reading and I have a saved\nsearch that quickly lets me see all papers I have already read. Another saved\nsearch allows me to quickly see all \"to read\" papers.<\/p>\n<p>I have been using \"Cite\" feature of Zotero and Semantic Scholar to create the\ntitle of each item in the bibliography and underneath that I jot down notes and\nother relevant details like links.<\/p>\n<h2 id=\"feedback\">Feedback<\/h2>\n<p>I would love to hear what has worked for you. Is there something I am missing.\nReach out to me via email or twitter.<\/p>\n<p>Many thanks to <a href=\"https:\/\/www.dursi.ca\/\">Jonathan Dursi<\/a> for constantly helping me improve this process.<\/p>\n"},{"title":"Why on Earth are copyleft software licenses bad for scientific software?","published":"2020-03-21T00:00:00+00:00","updated":"2020-03-21T00:00:00+00:00","author":{"name":"\n            \n              Unknown\n            \n          "},"link":{"@attributes":{"rel":"alternate","type":"text\/html","href":"https:\/\/amanjeev.com\/blog\/copyleft-licenses-scientific-software\/"}},"id":"https:\/\/amanjeev.com\/blog\/copyleft-licenses-scientific-software\/","content":"<p>My quest for this blog started with the discussion about the choice of license for the software under Open Differential Privacy (OpenDP) Initiative[17]. I wanted to understand why <a href=\"https:\/\/en.wikipedia.org\/wiki\/Copyleft\">copyleft licenses<\/a> are a taboo in Scientific circles. OpenDP choosing <a href=\"https:\/\/mit-license.org\/\">MIT license<\/a> prompted me to ask why wasn't something like <a href=\"https:\/\/www.gnu.org\/licenses\/agpl-3.0.en.html\">AGPL<\/a> used. I got this as an answer:<\/p>\n<p>https:\/\/twitter.com\/cbavitz\/status\/1260662650230505472?s=20<\/p>\n<p>This answer prompted a feeling I have been harboring for a while \u2014 no one explains what exactly is wrong with the copyleft licenses and why they restrict adoption and participation.<\/p>\n<p>I and many countless others have been involved in some kind of scientific field as software programmers. We, the research software engineers, now even have our own organizations[18]! In a lot of ways, the scientific programming community is a bit behind the commercial software world. This is certainly true for best-practices, tools, etc. but it is also true for software licensing. In this post, you will get a glimpse of the most prominent (or prominently searchable) software licensing discussion in the scientific community.<\/p>\n<h2 id=\"disclosure\">Disclosure<\/h2>\n<p>For me, being in sciences gives the benefit of learning something new and help support scientific discovery along the way. I believe that copyleft licenses are truly aligned with the academic and scientific pursuit. I understand that commercial companies and folks who work there are vocal about not using copyleft licenses but I believe that for scientific knowledge to thrive, we need to use more copyleft licenses.<\/p>\n<p>Also, this blog post will focus on non-military software. The kind of software that is not under an embargo or security curtains.<\/p>\n<h2 id=\"types-of-scientific-software\">Types of scientific software<\/h2>\n<p>My experience mostly comes from genomics and adjacent areas. It is important to note that in this area, the privacy of individuals is critical and so is their data.<\/p>\n<h3 id=\"software-accompanying-research-paper-s\">Software accompanying research paper(s)<\/h3>\n<p>This is the most common piece of software associated with scientific work and is generally published somewhere. This type of software can be a library, framework or even a hosted service with its source code hosted somewhere else. Most of the time, these are repositories sitting on GitHub that have not been touched since the paper was published. Some are even with open pull requests and issues with no response from the owner of the repository. But this rant is for another day.<\/p>\n<p>You may think of this as a basic proof-of-concept that is not to be used in any production level work.<\/p>\n<h3 id=\"software-accompanied-by-research-paper-s\">Software accompanied by research paper(s)<\/h3>\n<p>These are the tools and frameworks that help run scientific research projects. Software published by NCBI or NASA that facilitates researchers in their respective fields to run computations or simulations could be seen as this type of software. Or you can think of tools like NumPy, SciPy, etc. My current work project \u2014 CanDIG[19] \u2014 is of a similar kind.<\/p>\n<h2 id=\"commentary-on-licensing-in-the-scientific-community\">Commentary on licensing in the scientific community<\/h2>\n<p>The most common papers and articles that I could find are from <a href=\"https:\/\/en.wikipedia.org\/wiki\/Victoria_Stodden\">Prof. Stodden<\/a>. Prof. Stodden has published more about software licensing than anyone else in academia that I could personally find. Some quotes for a slide of a talk by Prof. Stodden[1]:<\/p>\n<blockquote>\n<p>The scientific ethos precludes directing another scientist\u2019s creative\ncontribution.<\/p>\n<\/blockquote>\n<blockquote>\n<p>Copyleft licenses make demands on downstream code, namely that they\nuse the upstream license on the entire library of new code. Two codes\nunder two different copyleft licenses, therefore, cannot be mixed, as\ncode cannot carry two licenses.<\/p>\n<\/blockquote>\n<blockquote>\n<p>Science creates knowledge, our best estimate of the truth - a public\ngood. People are free to use and build on public goods however they see fit.<\/p>\n<\/blockquote>\n<blockquote>\n<p>Copyleft creates a barrier the transmission of knowledge and scientific\nprogress, that is not compensated by other benefits.<\/p>\n<\/blockquote>\n<blockquote>\n<p>Copyleft inhibits or prevents collaboration with industrial partners.<\/p>\n<\/blockquote>\n<p>Don't scientific ethos also encourage \"communism\" and \"disinterestedness\"[20]? I fail to understand here that how is forcing companies to disclose the source is against the scientific ethos. Commercial companies have been using and continue to use software with copyleft licenses. Following is another statement by Prof. Stodden that makes my belief in copyleft licenses being more important in scientific software, actually stronger.<\/p>\n<blockquote>\n<p>Society\u2019s right to knowledge is an unconditional right.<\/p>\n<\/blockquote>\n<p>The knowledge, in part, lives in the software we write to facilitate the studies. A couple of questions before we move forward:<\/p>\n<ol>\n<li>How is it progress if someone takes your work, builds upon it, but does not give back? That is the argument being made by the commercial side.<\/li>\n<li>How is that I am reading the same sentence and coming to an opposite conclusion? :)<\/li>\n<\/ol>\n<p>It has been hard to get answers apart from just statements like \"copyleft licenses are not permissive...\". So, I searched for more...<\/p>\n<p>A 2012 paper in PLOS by Moris et. al.[5] is a good read where they suggest what you should choose and why. A blog post in Astronomy Computing Today[4] with the same title, referring to this paper, summarizes this nicely:<\/p>\n<blockquote>\n<p>If you want\u2026<\/p>\n<\/blockquote>\n<blockquote>\n<p>\u2026the widest possible distribution and adoption, fewest restrictions on\nusers, open and transparent source code, peer review, community\ncontributions to the codebase, and easy incorporation of your code by\nothers\u2026 then a permissive FOSS license such as the BSD\/MIT, Apache, or ECL\nlicenses may work well<\/p>\n<\/blockquote>\n<blockquote>\n<p>\u2026to assure the benefits and openness of FOSS in all future derivatives of\nyour work, open and transparent source code, peer review, community\ncontributions to the codebase, and the potential incorporation of your\ncode into other copyleft- licensed works\u2026 then you should consider a\ncopyleft FOSS license like the GPL, LGPL, or MPL.<\/p>\n<\/blockquote>\n<blockquote>\n<p>the ability to separately pursue proprietary models while leveraging the\nwide distribution, adoption, community contributions, and other benefits\nof open source software\u2026 then a hybrid or multi-license scheme may be\nap-propriate.<\/p>\n<\/blockquote>\n<blockquote>\n<p>\u2026protect the confidentiality of your source code, reserve maximum control\nover the distribution and use of your software, and derive licensing\nrevenue\u2026 then you should consider a proprietary license.\u201d<\/p>\n<\/blockquote>\n<p>Now we have a few pointers on when to choose which license. According to the comments above, a copyleft license makes sure that future derivatives are open as well. As I noted above, in my experience a vast majority of repositories are never touched again, after the publication of the research paper (this is purely anecdotal). There is proof that wide adoption can still happen, even with copyleft licenses, if the published software has value.<\/p>\n<p>A blog post from Jake VanderPlas on \"The Whys and Hows of Licensing Scientific Code\"[13] is also a good starting point for scientists to figure out what they want to do. The summary in the post is:<\/p>\n<blockquote>\n<ol>\n<li>Always license your code.  Unlicensed code is closed code, so any\nopen license is better than none (but see #2).<\/li>\n<li>Always use a GPL-compatible license. GPL-compatible licenses ensure\nbroad compatibility for your code, and include GPL, new BSD, MIT,\nand others (but see #3).<\/li>\n<li>Always use a permissive, BSD-style license. A permissive license\nsuch as new BSD or MIT is preferable to a copyleft license such as\nGPL or LGPL.<\/li>\n<\/ol>\n<\/blockquote>\n<p>This aligns with Prof. Stodden's views. One of the amazing things about the post[13] is the discussion in its comments which was incredibly insightful. I will reproduce some of those comments here:<\/p>\n<p>Jeremy Sanders writes:<\/p>\n<blockquote>\n<p>If I\u2019m publicly releasing code, I\u2019m not writing it so that it can be\ntaken by a for-profit company and essentially close-sourced. The GPL\nis philosophically much closer to academic freedom, where any modified\nversions distributed, also have the source distributed.<\/p>\n<\/blockquote>\n<p>To which, Jake responds:<\/p>\n<blockquote>\n<p>If I\u2019m publicly releasing code, I hope it\u2019s useful enough that a library\nlike scipy, scikit-learn, or matplotlib will want to incorporate it. In that\ncase, GPL is a barrier to its usefulness. Remember that in academia, it\u2019s not\nthe text of a license that protects your ideas, but the norms of the academic\ncommunity.<\/p>\n<\/blockquote>\n<p>Jeremy again:<\/p>\n<blockquote>\n<p>Jake: that\u2019s all very well for code that is purely academic and not\ninteresting to commercial users. If you code is at all commercially\nviable, the GPL is a much better prospect. If your code is useful,\npeople will use it regardless of the choice of free license. People\neven use black boxes like sm and galfit.<\/p>\n<\/blockquote>\n<p>Jake believes that projects like Matplotlib, Numpy, scipy, pandas, scikit-learn, IPython are successful because it is \"much more than a simple coincidence\". I think this is alluding to the fact that they use permissive licenses. This is where it gets even more interesting because Matthew Turk who moved his yt library to BSD and wrote about it in \"Relicensing yt from GPLv3 to BSD\"[14], chimes in:<\/p>\n<blockquote>\n<p>Hi Jeremy, I agree. I was really quite torn about re-licensing yt, as I am\npersonally an enormous fan of the GPL and the copyleft. I am, in fact, still\nrather torn about this for the reasons you wrote here \u2014 I am committed to\nensuring that the code remains free, and that future versions of it are\ninspectable at the source level. BSD licensing does not enforce that freedom,\nbut we as a community do \u2014 yt itself, even if a nefarious corporation takes\nit and improves it and sells it, will be free software, respecting all four\nfreedoms.\n...<\/p>\n<\/blockquote>\n<blockquote>\n<p>I guess my main point is, I think you\u2019re right, and I think the issue is a\nsubtle one, but I <em>also<\/em> think that in the end the choice that we made as\nthe yt community was the right one for us at the time. But that doesn\u2019t keep\nme from being a little sad that the GPL is viewed as such a pariah, as in\nessence I firmly believe that the de facto standards of FLOSS in science\nwill be GPL-like, with an active curation and preservation of the four\nfreedoms, even if BSD in actual license.<\/p>\n<\/blockquote>\n<p>This comment is critical to understand that there are scientists who do believe that a GPL-like license is aligned with the scientific endeavor.<\/p>\n<p>Moving along, if you are scrolling through comments there is a gem by Jeremy Sanders:<\/p>\n<blockquote>\n<p>...as Linux shows that it\u2019s certainly possible for commercial companies to\nthrive in a GPL ecosystem. I\u2019ve had commercial companies contributing towards\nmy GPL code with no licensing issues. If a company is scared off by the GPL,\nit\u2019s likely they need better lawyers. The code being GPL protects their\ncontributions from being swallowed up by competitors in rival closed source\napps.<\/p>\n<\/blockquote>\n<p>and a detailed comment to prior \"bundling\" comments by Mohammad Akhlaghi:<\/p>\n<blockquote>\n<p>This is a follow up to Jake\u2019s comment at March 14, 2014 at 10:09 am.\nSorry for continuing this discussion after about 2 years. I just wanted to\nknow exactly what part of GNU LGPL restricted the bundling?\n...\nSo if I have understood GNU LGPL correctly, you can easily bundle an GNU LGPL\nlibrary into a BSD-licenced program without much effort. But then again, I am\nnot a lawyer or have too much experience in the different software licenses\nyet, so I would be grateful if you could let me know if I have incorrectly\nunderstood the GNU LGPL.<\/p>\n<\/blockquote>\n<p>And finally, Don Barry's comment's first paragraph:<\/p>\n<blockquote>\n<p>I must disagree strongly with your endorsement of permissive licenses and\nyour arguments for them. What is nowhere here discussed is the enormous\namount of corporate lobbying which has influenced the issue. Google, with\nthe exception of work with the Linux kernel, a \u201ctoo big to avoid\u201d project,\nhas actively sought to expunge the GPL from other efforts of theirs: Android\ncontains only the kernel as GPL software. The fragmented Android ecosystem\nis one of many examples where this choice is hostile to the end user.<\/p>\n<\/blockquote>\n<p>So far, the most common theme among scientists (and many programmers) that I see is the leaning towards MIT or BSD type of licenses. Licenses that allow users to freely modify and distribute, without any strings attached to where the source came from or giving back. This is true even if it means that some of them still love GPL and its derivatives and are sad about not being able to use copyleft licenses. It'd be worth looking into how much of the sentiment against copyleft licenses is due to the industry pressure.<\/p>\n<h2 id=\"chatter-outside-the-scientific-community\">Chatter outside the scientific community<\/h2>\n<p>There is no dearth of commercial programmers hating on copyleft licenses. But some voices clarify why we should stick to a license that helps gain the modifications back. Look at what MongoDB writes in their post about licensing and their choice of AGPL[7]:<\/p>\n<blockquote>\n<p>To say this another way: if you modify the core database source code,\nthe goal is that you have to contribute those modifications back to\nthe community.<\/p>\n<\/blockquote>\n<p>This is the second piece of software (other than GNU\/Linux kernel) that is \"famous\" and uses a copyleft license. I still fail to see why copyleft licenses are a stigma and hence the title of this post. It is working fine for some commercially successful software.<\/p>\n<p>Moving on, David Wheeler has an essay[6] that GPL compatibility is important and how to achieve it. It also notes which licenses to avoid and software projects that consider compatibility with GPL important. I think it is worth a read even though it is an old post.<\/p>\n<p>In <a href=\"https:\/\/web.archive.org\/web\/20090831045815\/http:\/\/zedshaw.com\/blog\/2009-07-13.html\">an old blog post (now unavailable, wayback link[21])<\/a> Zed Shaw, talked about why he is using (A\/L)GPL. However, in his Reddit comment[3] he says he was wrong and gives the reason(s):<\/p>\n<blockquote>\n<p>This article is old but basically everyone is right and I was wrong: I didn't\nmake any money on Lamson and nobody contributed to it, no matter what license\nI used, and no matter where I hosted it. Pretty much, Lamson proved to me that\nall the reasons you're given for open source success (license, github, BDFL,\netc.) are a load of horse shit. It's marketing dollars, propaganda, and random\nchance just like everything else.\nSo now my stuff is BSD because I write books at\nhttp:\/\/learncodethehardway.org\/ and there's not really an alternative license\nthat works for code in books. That's all. No major revelations or drama, I\njust moved on to something better.<\/p>\n<\/blockquote>\n<p>So, if Z. Shaw is to be believed that the commercial success of a project is mostly marketing, money, and chance, then to me, it makes sense to stick to copyleft license if sharing and openness is something you even remotely care about. It seems to me that copyleft licenses are beneficial, in general, to the scientific openness. I am not sure if the importance of \"permissivity\" of the license mostly a cry from those who find it cumbersome to give back while raking profits or there is more to it. I would love to know. However, in science, we cannot let this dictate the ethos of sharing and openness.<\/p>\n<h2 id=\"open-and-accessible-is-the-final-word\">Open and accessible is the final word<\/h2>\n<p>In recent years, the term \"Open Science\"[16] has become popular in our community. One of the ways Wikipedia defines it as:<\/p>\n<blockquote>\n<p>Open science is transparent and accessible knowledge that is shared and developed through collaborative networks.<\/p>\n<\/blockquote>\n<p>It is a noble effort and I am glad that there is awareness within the community to make sure that the information stays open and accessible. However, the question remains that why shouldn't science be \"open\" by default? Almost every researcher I know understands the power certain paid journals have since many research publications are behind paywalls and convoluted logins. Many agree that these places hinder science in the end. You should be able to use previous research and build upon it and this new work can inform the previous work that can be improved even further. If this model works for other parts of scientific work, then why is scientific software any different?<\/p>\n<p>Many thanks to the following people who reviewed this post:<\/p>\n<ul>\n<li><a href=\"https:\/\/www.dursi.ca\/\">Jonathan Dursi<\/a><\/li>\n<li>Shveta Kumar<\/li>\n<li><a href=\"https:\/\/hoverbear.org\/\">Ana Hobden<\/a><\/li>\n<\/ul>\n<h2 id=\"references\">References<\/h2>\n<ol>\n<li>\n<p>Prof. Stodden, V  \"Why Copyleft Isn\u2019t Right for Scientific Code\" https:\/\/web.stanford.edu\/~vcs\/talks\/VictoriaStoddenIPSC2010.pdf<\/p>\n<\/li>\n<li>\n<p>VanderPlas, J  \"The Whys and Hows of Licensing Scientific Code\" https:\/\/www.astrobetter.com\/blog\/2014\/03\/10\/the-whys-and-hows-of-licensing-scientific-code\/<\/p>\n<\/li>\n<li>\n<p>Reddit Comment by Zed Shaw https:\/\/www.reddit.com\/r\/programming\/comments\/llw0s\/zed_shaw_on_gpl\/c2tt5e3\/<\/p>\n<\/li>\n<li>\n<p>astrocompute. \u201cA Quick Guide To Software Licensing for the Scientist Programmer.\u201d Astronomy Computing Today (blog), February 25, 2014. https:\/\/astrocompute.wordpress.com\/2014\/02\/25\/a-quick-guide-to-software-licensing-for-the-scientist-programmer\/.<\/p>\n<\/li>\n<li>\n<p>Morin, Andrew, Jennifer Urban, and Piotr Sliz. \u201cA Quick Guide to Software Licensing for the Scientist-Programmer.\u201d PLOS Computational Biology 8, no. 7 (July 26, 2012): e1002598. https:\/\/doi.org\/10.1371\/journal.pcbi.1002598.<\/p>\n<\/li>\n<li>\n<p>\u201cMake Your Open Source Software GPL-Compatible. Or Else.\u201d Accessed April 20, 2020. https:\/\/dwheeler.com\/essays\/gpl-compatible.html.<\/p>\n<\/li>\n<li>\n<p>MongoDB. \u201cThe AGPL | MongoDB Blog.\u201d Accessed May 10, 2020. https:\/\/www.mongodb.com\/blog\/post\/the-agpl.<\/p>\n<\/li>\n<li>\n<p>\u201cConditions on Distributing Ghostscript in a Commercial Context.\u201d Accessed May 10, 2020. https:\/\/www.ghostscript.com\/doc\/current\/Commprod.htm.<\/p>\n<\/li>\n<li>\n<p>\u201cLicenses &amp; Standards | Open Source Initiative.\u201d Accessed April 20, 2020. https:\/\/opensource.org\/licenses.<\/p>\n<\/li>\n<li>\n<p>astrocompute. \u201cLicensing Your Code: GPL, BSD and Edvard Munch\u2019s \u2018The Scream.\u2019\u201d Astronomy Computing Today (blog), January 17, 2014. https:\/\/astrocompute.wordpress.com\/2014\/01\/17\/licensing-your-code-gpl-bsd-and-edvard-munchs-the-scream\/.<\/p>\n<\/li>\n<li>\n<p>\u201cNeuroimaging in Python \u2014 NIPY Documentation.\u201d Accessed April 20, 2020. http:\/\/nipy.sourceforge.net\/nipy\/stable\/faq\/johns_bsd_pitch.html.<\/p>\n<\/li>\n<li>\n<p>\u201cThe GPL and License Compatibility.\u201d Accessed April 20, 2020. https:\/\/producingoss.com\/en\/license-compatibility.html.<\/p>\n<\/li>\n<li>\n<p>\u201cThe Whys and Hows of Licensing Scientific Code.\u201d Accessed April 20, 2020. https:\/\/www.astrobetter.com\/blog\/2014\/03\/10\/the-whys-and-hows-of-licensing-scientific-code\/.<\/p>\n<\/li>\n<li>\n<p>The yt Project Blog. \u201cThe Yt Project Blog\u202f\u00bb Post: Relicensing Yt from GPLv3 to BSD,\u201d September 12, 2013. http:\/\/blog.yt-project.org\/post\/Relicensing\/.<\/p>\n<\/li>\n<li>\n<p>\u201cWhy I Don\u2019t Use the GPL | Linux Journal.\u201d Accessed April 20, 2020. https:\/\/www.linuxjournal.com\/article\/5935.<\/p>\n<\/li>\n<li>\n<p>\u201cOpen Science.\u201d In Wikipedia, May 12, 2020. https:\/\/en.wikipedia.org\/w\/index.php?title=Open_science&amp;oldid=956239136.<\/p>\n<\/li>\n<li>\n<p>\u201cOpenDP.\u201d Accessed May 20, 2020. https:\/\/privacytools.seas.harvard.edu\/opendp.<\/p>\n<\/li>\n<li>\n<p>\u201cSociety of Research Software Engineering.\u201d Accessed May 20, 2020. https:\/\/society-rse.org\/.<\/p>\n<\/li>\n<li>\n<p>\u201cCanDIG.\u201d Accessed May 20, 2020. https:\/\/www.distributedgenomics.ca\/.<\/p>\n<\/li>\n<li>\n<p>\u201cMertonian Norms.\u201d In Wikipedia, May 9, 2020. https:\/\/en.wikipedia.org\/w\/index.php?title=Mertonian_norms&amp;oldid=955747690.<\/p>\n<\/li>\n<li>\n<p>\"Why I (A\/L)GPL\" Zed Shaw, From Wayback Machine, July 13, 2009. https:\/\/web.archive.org\/web\/20090831045815\/http:\/\/zedshaw.com\/blog\/2009-07-13.html<\/p>\n<\/li>\n<\/ol>\n"},{"title":"NixOS and other tools","published":"2020-01-25T00:00:00+00:00","updated":"2020-01-25T00:00:00+00:00","author":{"name":"\n            \n              Unknown\n            \n          "},"link":{"@attributes":{"rel":"alternate","type":"text\/html","href":"https:\/\/amanjeev.com\/blog\/nixos\/"}},"id":"https:\/\/amanjeev.com\/blog\/nixos\/","content":"<h3 id=\"note-this-blog-post-is-old-and-many-things-here-have-been-changed-and-moved-on-i-will-write-another-post-to-update\">Note: This blog post is old and many things here have been changed and moved on. I will write another post to update.<\/h3>\n<p>Before I begin, you must know that my inspiration to move to NixOS was <a href=\"https:\/\/spacekookie.de\/\">Spacekookie<\/a>. I had no idea about NixOS\nbefore I learned about it from Spacekookie. You can read the <a href=\"https:\/\/spacekookie.de\/blog\/home-manager-or-how-not-to-yakhave\/\">NixOS\/Home Manager post<\/a>.<\/p>\n<p>You can see my <a href=\"https:\/\/github.com\/amanjeev\/somenix\">NixOS and Home Manager configs<\/a>. This is a big deal!<\/p>\n<h2 id=\"history\">History<\/h2>\n<p>Years ago, I moved to the Apple ecosystem for stability and things like the Time Machine backup system. That is still where my personal computing lives.\nBut I was getting more and more worried about having personal data on the same machine where I download and run code. I was hoping to\nseparate my development setup from my personal computing setup. I could have gone the route of QubesOS but back then I was not ready\nto shave more yaks than what I was already helping groom. So, I decided to buy a laptop and have been running Ubuntu on it, up until\na few months ago, when I removed Ubuntu and moved to NixOS.<\/p>\n<h2 id=\"why-nixos\">Why <a href=\"https:\/\/nixos.org\/\">NixOS<\/a><\/h2>\n<p>For years I have looked at way too many tools to have a decent backup and restore strategy and nothing stood out. I can manage\nto backup my home directory and restore it but I needed something that helped reliably reinstall all the applications, dotfiles\nand other tools that I need day-to-day. Then I can just reinstall in case of a computer crash and fetch the home contents from a backup.\nThis is where <a href=\"https:\/\/nixos.org\/\">NixOS<\/a> helps. In short, it is a Linux distribution that is based on <a href=\"https:\/\/nixos.org\/nix\/\">Nix<\/a>, a purely\nfunctional package manager (it is also a functional programming language). This allows your installations, upgrades, etc. to be atomic, which\nmakes them much more reliable. That is, every time you install or upgrade, NixOS creates a new <em>snapshot<\/em>. You can clean older <em>snapshots<\/em>\nwhen you feel like it or boot into an older one if you wanted. There is also <a href=\"https:\/\/github.com\/rycee\/home-manager\">home-manager<\/a> which is userspace Nix. I highly recommend\nyou look into that as well.<\/p>\n<p>For those interested here is my <a href=\"https:\/\/github.com\/amanjeev\/somenix\">Nixos and Home Manager configs<\/a>. I can give a quick run down.<\/p>\n<h3 id=\"a-build-shell-script\"><a href=\"https:\/\/github.com\/amanjeev\/somenix\/blob\/master\/build.sh#L14-L15\">A build shell script<\/a><\/h3>\n<p>This is needed because I want to be able to override certain packages and have the config live in my home directory rather than the default\n<code>\/etc<\/code>.<\/p>\n<h3 id=\"root-s\"><a href=\"https:\/\/github.com\/amanjeev\/somenix\/tree\/master\/nixos\/roots\">Root(s)<\/a><\/h3>\n<p>This is where I store the root configs for various machines. A root config is your <code>configuration.nix<\/code> and <code>hardware-configuration.nix<\/code>\nthat is created the first time you bootstrap while installing NixOS. You can use <code>configuration.nix<\/code> for everything but it is better\nto have some semblance of order. Programmers, eh!<\/p>\n<p>I have three development machines and at the time of this writing, two of those are on NixOS. Yes, I love wolves and yes I came up with the name Torontula.<\/p>\n<h3 id=\"common-stuff\"><a href=\"https:\/\/github.com\/amanjeev\/somenix\/blob\/master\/nixos\/common.nix\">Common stuff<\/a><\/h3>\n<p>This is mostly common stuff for all machines without having to make duplicates. This could go into a <code>configuration.nix<\/code> but I chose to move it\ninto its module.<\/p>\n<ul>\n<li>This file <a href=\"https:\/\/github.com\/amanjeev\/somenix\/blob\/master\/nixos\/common.nix#L4-L8\">defines the <code>unstable<\/code> branch<\/a> of Nix because certain packages\nare only available in that. At the time of this writing, I have Zulip which I needed for Rust's official chat. Note that you can define <code>ref<\/code> and\n<code>revision<\/code> and fetch only that. Talk about thinking of pinning from the ground up.<\/li>\n<li>It installs Virtualbox and related services. Yes, I need this on all machines and it takes horribly long to recompile.<\/li>\n<li>When was the last time you <a href=\"https:\/\/github.com\/amanjeev\/somenix\/blob\/master\/nixos\/common.nix#L45\">defined a font<\/a> in the config? Or <a href=\"https:\/\/github.com\/amanjeev\/somenix\/blob\/master\/nixos\/common.nix#L20-L27\">created a user in the config<\/a> with some Puppet\/Chef\/etc.? ;)<\/li>\n<li>Oh, look at the <a href=\"https:\/\/github.com\/amanjeev\/somenix\/blob\/master\/nixos\/common.nix#L69-L72\">desktop managers here<\/a>.<\/li>\n<\/ul>\n<h3 id=\"packages\"><a href=\"https:\/\/github.com\/amanjeev\/somenix\/blob\/master\/nixos\/pkgs\/default.nix\">Packages<\/a><\/h3>\n<ul>\n<li>Ahem! <a href=\"https:\/\/github.com\/amanjeev\/somenix\/blob\/master\/nixos\/pkgs\/default.nix#L6-L10\">Manager your config files<\/a>. The confs live in a\nseparate directory.<\/li>\n<li><a href=\"https:\/\/github.com\/amanjeev\/somenix\/blob\/master\/nixos\/pkgs\/default.nix#L12-L92\">This entire section is the list of packages I want to be installed<\/a>.\nI add and remove most of them here and run the rebuild command.<\/li>\n<li>When was the last time you <a href=\"https:\/\/github.com\/amanjeev\/somenix\/blob\/master\/nixos\/pkgs\/default.nix#L94-L122\">managed your git config and settings<\/a> like this?<\/li>\n<li>Do you use emacs (or even vim)? Here is the <a href=\"https:\/\/github.com\/amanjeev\/somenix\/blob\/master\/nixos\/pkgs\/default.nix#L126-L164\">list of all the emacs packages that I use<\/a>.\nSome of these are directly from <a href=\"https:\/\/spacekookie.de\/\">Spacekookie<\/a>!<\/li>\n<\/ul>\n<h3 id=\"bonus\">Bonus<\/h3>\n<ul>\n<li><a href=\"https:\/\/github.com\/amanjeev\/somenix\/blob\/master\/nixos\/roots\/torontula\/configuration.nix#L31\">Change your Linux Kernel in one line<\/a>.<\/li>\n<li>I moved to <a href=\"https:\/\/fishshell.com\/\">fish shell<\/a>; again thanks to <a href=\"https:\/\/spacekookie.de\/\">Spacekookie<\/a> from whom I\n<a href=\"https:\/\/github.com\/amanjeev\/somenix\/tree\/master\/nixos\/pkgs\/confs\/fish\">stole this entire set of functions<\/a>.<\/li>\n<li>I moved to <a href=\"https:\/\/github.com\/alacritty\/alacritty\">Alacritty<\/a> as my main terminal emulator.\nHere is <a href=\"https:\/\/github.com\/amanjeev\/somenix\/blob\/master\/nixos\/pkgs\/confs\/alacritty.yml\">Alacritty's config YAML<\/a>\nand <a href=\"https:\/\/github.com\/amanjeev\/somenix\/blob\/master\/nixos\/pkgs\/default.nix#L10\">here is how you \"install\" Alacritty's config<\/a>\nIt is truly faster than any other terminal emulator I have used.<\/li>\n<li>If you like QubesOS, you should check out <a href=\"https:\/\/spectrum-os.org\/\">SpectrumOS<\/a>. It is like QubesOS but based on Nix. I am excited about this and I wish I had time\nand knowledge to contribute to this project.<\/li>\n<\/ul>\n<p>If you want to get help, folks on the NixOS IRC channel are very helpful!<\/p>\n"},{"title":"man7.org Linux training and Google cloud virtual machines","published":"2019-03-03T00:00:00+00:00","updated":"2019-03-03T00:00:00+00:00","author":{"name":"\n            \n              Unknown\n            \n          "},"link":{"@attributes":{"rel":"alternate","type":"text\/html","href":"https:\/\/amanjeev.com\/blog\/man7-training-and-virual-machines\/"}},"id":"https:\/\/amanjeev.com\/blog\/man7-training-and-virual-machines\/","content":"<p>I had the privilege to attend a week-long training by <a href=\"http:\/\/man7.org\/\">Michael Kerrisk<\/a> on \"System Programming for Liunux Containers\". If you are interested in Linux deep-dive, I cannot recommend anyone more than Michael. I have been waiting for his training for about four years and it finally happened! So, keep your hopes high, you may someday get that too.<\/p>\n<p><a href=\"http:\/\/man7.org\/mtk\/index.html\">Michael is the maintainer of the Linux man-pages<\/a> and is extremely knowledgeable. His style of teaching is balanced\nwith enough theory for you to get started and decent collection of exercises along the way. In his class, you will be working as pair-programmers. This helps speed up the work you have to do and encourages discussion with your peers. He is receptive to comments and suggestions.<\/p>\n<p>You should contact him!<\/p>\n<p>For his course, you need a Linux machine and it is better if it is your laptop but you could work with a virtual machine on your laptop. To make sure that we have spare VMs, I created a bunch on Google cloud platform. <a href=\"https:\/\/github.com\/amanjeev\/man7-training\">You can use this Terraform repo to do the same<\/a>. For his courses, you may have to access the GRUB prompt to edit Linux kernel args. The output for Terraform code after it generates the VMs, will show the command to do that as well.<\/p>\n<p>In short, <a href=\"https:\/\/cloud.google.com\/compute\/docs\/instances\/interacting-with-serial-console\">Google cloud platform allows you to connect to your VM's GRUB prompt via a seial console<\/a> (that you have to enable prior to VM creation).<\/p>\n<p>Happy hacking!<\/p>\n"},{"title":"How to document Rust's macro invocation","published":"2019-02-21T00:00:00+00:00","updated":"2019-02-21T00:00:00+00:00","author":{"name":"\n            \n              Unknown\n            \n          "},"link":{"@attributes":{"rel":"alternate","type":"text\/html","href":"https:\/\/amanjeev.com\/blog\/rust-document-macro-invocations\/"}},"id":"https:\/\/amanjeev.com\/blog\/rust-document-macro-invocations\/","content":"<p>Since, I have spent way too much time on this, I decided it is worth noting down.<\/p>\n<h3 id=\"the-problem\">The problem<\/h3>\n<p>You can easily document your Rust items like functions by putting three slashes \/\/\/.<\/p>\n<pre data-lang=\"rust\" style=\"background-color:#eff1f5;color:#4f5b66;\" class=\"language-rust \"><code class=\"language-rust\" data-lang=\"rust\"><span style=\"color:#a7adba;\">\/\/\/ My amazing function\n<\/span><span style=\"color:#a7adba;\">\/\/\/ # Example\n<\/span><span style=\"color:#a7adba;\">\/\/\/ amazing();\n<\/span><span style=\"color:#b48ead;\">fn <\/span><span style=\"color:#8fa1b3;\">amazing<\/span><span>() -&gt; () {\n<\/span><span>    println!(&quot;<\/span><span style=\"color:#a3be8c;\">Rust is amazing!<\/span><span>&quot;);\n<\/span><span>}\n<\/span><\/code><\/pre>\n<p>You can also document your macro like this -<\/p>\n<pre data-lang=\"rust\" style=\"background-color:#eff1f5;color:#4f5b66;\" class=\"language-rust \"><code class=\"language-rust\" data-lang=\"rust\"><span style=\"color:#a7adba;\">\/\/\/ My amazing macro&#39;s own doc\n<\/span><span style=\"color:#96b5b4;\">macro_rules! <\/span><span>amazing {\n<\/span><span>    (<\/span><span style=\"color:#bf616a;\">$func<\/span><span>:<\/span><span style=\"color:#b48ead;\">ident<\/span><span>) =&gt; (\n<\/span><span>        <\/span><span style=\"color:#b48ead;\">fn <\/span><span style=\"color:#bf616a;\">$func<\/span><span>() -&gt; () {\n<\/span><span>            println!(&quot;<\/span><span style=\"color:#a3be8c;\">Rust is amazing! Rust has powerful features!<\/span><span>&quot;);\n<\/span><span>        };   \n<\/span><span>    )\n<\/span><span>}\n<\/span><\/code><\/pre>\n<p>However, if you want to document each separate invocation of your amazing! macro, it is not that straightfoward. You might ask why should we be doing that anyway? Because macros can generate a lot of boilerplate code for you and each of those invocations might be slightly different which must be conveyed to your user in docs.rs. For example, this will not work<\/p>\n<pre data-lang=\"rust\" style=\"background-color:#eff1f5;color:#4f5b66;\" class=\"language-rust \"><code class=\"language-rust\" data-lang=\"rust\"><span style=\"color:#a7adba;\">\/\/\/ this is a macro invocation but\n<\/span><span style=\"color:#a7adba;\">\/\/\/ these doc strings will not show\n<\/span><span>amazing!(amazing_function_1);\n<\/span><\/code><\/pre>\n<p>Phew!<\/p>\n<h3 id=\"solution\">Solution<\/h3>\n<h4 id=\"slightly-modify-your-macro\">Slightly modify your macro<\/h4>\n<pre data-lang=\"rust\" style=\"background-color:#eff1f5;color:#4f5b66;\" class=\"language-rust \"><code class=\"language-rust\" data-lang=\"rust\"><span style=\"color:#a7adba;\">\/\/\/ My amazing macro&#39;s own doc \n<\/span><span style=\"color:#96b5b4;\">macro_rules! <\/span><span>amazing {\n<\/span><span>    (\n<\/span><span>        $(#[<\/span><span style=\"color:#bf616a;\">$meta<\/span><span>:<\/span><span style=\"color:#b48ead;\">meta<\/span><span>])*\n<\/span><span>        <\/span><span style=\"color:#bf616a;\">$func<\/span><span>:<\/span><span style=\"color:#b48ead;\">ident\n<\/span><span>    ) =&gt; (\n<\/span><span>        $(#[<\/span><span style=\"color:#bf616a;\">$meta<\/span><span>])*\n<\/span><span>        <\/span><span style=\"color:#b48ead;\">fn <\/span><span style=\"color:#bf616a;\">$func<\/span><span>() -&gt; () {\n<\/span><span>            println!(&quot;<\/span><span style=\"color:#a3be8c;\">Rust is amazing! Rust has powerful features!<\/span><span>&quot;);\n<\/span><span>        }\n<\/span><span>    )\n<\/span><span>}\n<\/span><\/code><\/pre>\n<p>Let me explain briefly what those two new lines mean.<\/p>\n<h5 id=\"capture-meta\">Capture meta<\/h5>\n<p><code class=\"inline\">$(#[$meta:meta])*<\/code> tells the macro to match 0 or more instances of meta attributes. Rust doc comment <code>\/\/\/ a comment<\/code> is just another way of saying <code>#[doc = \" a comment\"]<\/code> This means that if we provide a comment in our invocation, this will now be captured.<\/p>\n<h5 id=\"output-meta\">Output meta<\/h5>\n<p><code class=\"inline\">$(#[meta])*<\/code> just spits out all the meta attributes right before the function.<\/p>\n<h4 id=\"add-comment-in-your-invocation\">Add comment in your invocation<\/h4>\n<pre data-lang=\"rust\" style=\"background-color:#eff1f5;color:#4f5b66;\" class=\"language-rust \"><code class=\"language-rust\" data-lang=\"rust\"><span>amazing!(\n<\/span><span>    <\/span><span style=\"color:#a7adba;\">\/\/\/ this macro invocation will have a comment\n<\/span><span>    <\/span><span style=\"color:#a7adba;\">\/\/\/ # Examples\n<\/span><span>    <\/span><span style=\"color:#a7adba;\">\/\/\/ amazing_function_2();\n<\/span><span>    amazing_function_2\n<\/span><span>);\n<\/span><span>\n<\/span><span>amazing!(\n<\/span><span>    <\/span><span style=\"color:#a7adba;\">\/\/\/ this macro invocation will have another, its own, comment\n<\/span><span>    <\/span><span style=\"color:#a7adba;\">\/\/\/ # Examples\n<\/span><span>    <\/span><span style=\"color:#a7adba;\">\/\/\/ amazing_function_3();\n<\/span><span>    amazing_function_3\n<\/span><span>);\n<\/span><\/code><\/pre>\n<p>The comments in the docs will now show up for both functions that were created via invoking macro. :)<\/p>\n<p>Many thanks to kind folks in Rust community discord server.<\/p>\n"},{"title":"AWS load balancers and instance health checks with terraform","published":"2018-10-07T00:00:00+00:00","updated":"2018-10-07T00:00:00+00:00","author":{"name":"\n            \n              Unknown\n            \n          "},"link":{"@attributes":{"rel":"alternate","type":"text\/html","href":"https:\/\/amanjeev.com\/blog\/aws-load-balancers-and-health-checks-with-terraform\/"}},"id":"https:\/\/amanjeev.com\/blog\/aws-load-balancers-and-health-checks-with-terraform\/","content":"<p><a href=\"https:\/\/docs.aws.amazon.com\/autoscaling\/ec2\/userguide\/AutoScalingGroup.html\">Auto Scaling Group (ASG)<\/a> is an AWS feature that allows you to manage the size of a cluster (group) of similar instances. You can create an ASG with a minimum number and maximum number of the instances of a particular image. In other words, a <em>group<\/em> of instances that <em>scale<\/em> <em>auto<\/em>matically.<\/p>\n<p>In its simplest forms, it relies on the <em>Health checks<\/em> to determine if any of the instances is unhealthy. In a more advanced setup, if configured, it can scale the number of instances up or down depending on the usage of the instances.<\/p>\n<h3 id=\"health-check-types\">Health check types<\/h3>\n<p>From what I understand, there are three major types of health checks that AWS provides (not counting custom health checks).<\/p>\n<h4 id=\"aws-ec2-status-checks\">AWS EC2 Status Checks<\/h4>\n<p>For the most basic ASG, the health checks are simply based on the EC2 instance's vitals like system power, networking issues, memory exhaustion etc. You can read more about <a href=\"https:\/\/docs.aws.amazon.com\/AWSEC2\/latest\/UserGuide\/monitoring-system-instance-status-check.html\">Status checks for your instances<\/a>.<\/p>\n<p>These are default checks and are readily available to use. However, there are a couple of issues that you may see with these checks:<\/p>\n<ul>\n<li>They can only tell you about the instance or system health and not the application level health. This means that we have to rely on metrics that may not be telling the true state of your application health. For example, the CPU consumption may be low and network maybe fine but the application itself crashed.<\/li>\n<li>We cannot truly gauge the load on our instances and hence, scaling up or down in the number of instances may not be feasible with these checks alone.<\/li>\n<\/ul>\n<p>For this reason, AWS offers a <a href=\"https:\/\/docs.aws.amazon.com\/autoscaling\/ec2\/userguide\/autoscaling-load-balancer.html\">feature of adding a Load Balancer<\/a> in front of your instances. Amazon calls it <a href=\"https:\/\/docs.aws.amazon.com\/elasticloadbalancing\/latest\/userguide\/what-is-load-balancing.html\">Elastic Load Balancer<\/a>.<\/p>\n<h4 id=\"elastic-load-balancing-health-checks-classic-load-balancer\">Elastic Load Balancing Health Checks - Classic Load Balancer<\/h4>\n<p><a href=\"https:\/\/docs.aws.amazon.com\/elasticloadbalancing\/latest\/classic\/introduction.html\">Classic Load Balancer<\/a> is meant mostly for EC2-Classic network. New customers do not get this option (EC2-Classic) to launch instances anymore but it is worth writing about the limitations.<\/p>\n<p>The idea is simple enough \u2014 you define the CLB, you define health check and the Load Balancer does not route traffic to an unhealthy instance in the group. This is already an improvement over the EC2 Status Checks because these LB health checks let you define more granular checks and lets you rely on a \"200 OK\" response status of your application.<\/p>\n<p>The downside of CLBs is that you have only one health check per LB. If you want to have more than one health check, then you have to create new LBs and point them to the backend instances. This can grow cumbersome real fast but there are other ways to do more checks without maintaining multiple LBs. One of them is the Application Load Balancer.<\/p>\n<h4 id=\"elastic-load-balancing-health-checks-application-load-balancer\">Elastic Load Balancing Health Checks - Application Load Balancer<\/h4>\n<p><a href=\"https:\/\/docs.aws.amazon.com\/elasticloadbalancing\/latest\/application\/introduction.html\">Application Load Balancer<\/a> is a strict Layer-7 Load Balancing. It is much better than the Classic Load Balancer in many ways -<\/p>\n<ul>\n<li>Host-based routing via HTTP Host header along with Path-based routing.<\/li>\n<li>HTTP\/2 support.<\/li>\n<li>Wider range of error codes (200-499).<\/li>\n<\/ul>\n<p>The major point to remember is that you create <em>target groups<\/em>, which each have one health check. Then you can configure a listener for ALB and provide rules to the listener that tell it to route to a particular target group.<\/p>\n<h5 id=\"overview-of-steps-to-create-an-alb\">Overview of steps to create an ALB<\/h5>\n<p>Two major resources that you need to pay attention to are Listeners and Targets.<\/p>\n<h6 id=\"target-groups-and-health-checks\">Target Groups and health checks<\/h6>\n<p><a href=\"https:\/\/docs.aws.amazon.com\/elasticloadbalancing\/latest\/application\/load-balancer-target-groups.html\">Setup Target Groups<\/a> and configure health checks for each group.<\/p>\n<pre data-lang=\"terraform\" style=\"background-color:#eff1f5;color:#4f5b66;\" class=\"language-terraform \"><code class=\"language-terraform\" data-lang=\"terraform\"><span># An example of target group\n<\/span><span>resource &quot;aws_alb_target_group&quot; &quot;target-group-1&quot; {\n<\/span><span>  name = &quot;target-group-1&quot;\n<\/span><span>  port = 80\n<\/span><span>  protocol = &quot;HTTP&quot;\n<\/span><span>\n<\/span><span>  lifecycle { create_before_destroy=true }\n<\/span><span>\n<\/span><span>  health_check {\n<\/span><span>    path = &quot;\/api\/1\/resolve\/default?path=\/service\/my-service&quot;\n<\/span><span>    port = 2001\n<\/span><span>    healthy_threshold = 6\n<\/span><span>    unhealthy_threshold = 2\n<\/span><span>    timeout = 2\n<\/span><span>    interval = 5\n<\/span><span>    matcher = &quot;200&quot;  # has to be HTTP 200 or fails\n<\/span><span>  }\n<\/span><span>}\n<\/span><\/code><\/pre>\n<h6 id=\"listener-and-listener-rules\">Listener and Listener rules<\/h6>\n<p><a href=\"https:\/\/docs.aws.amazon.com\/elasticloadbalancing\/latest\/application\/load-balancer-listeners.html\">Setup a Listener<\/a> with <a href=\"https:\/\/docs.aws.amazon.com\/elasticloadbalancing\/latest\/application\/load-balancer-listeners.html#listener-rules\">Listener Rules<\/a> that allow you to forward the requests to appropriate <em>targets<\/em> in one or more <a href=\"https:\/\/docs.aws.amazon.com\/elasticloadbalancing\/latest\/application\/load-balancer-target-groups.html\"><em>target groups<\/em><\/a>.<\/p>\n<pre data-lang=\"terraform\" style=\"background-color:#eff1f5;color:#4f5b66;\" class=\"language-terraform \"><code class=\"language-terraform\" data-lang=\"terraform\"><span># An example of a Listener\n<\/span><span>resource &quot;aws_alb_listener&quot; &quot;my-alb-listener&quot; {\n<\/span><span>  default_action {\n<\/span><span>    target_group_arn = &quot;${aws_alb_target_group.target-group-1.arn}&quot;\n<\/span><span>    type = &quot;forward&quot;\n<\/span><span>  }\n<\/span><span>  load_balancer_arn = &quot;${aws_alb.my-app-alb.arn}&quot;\n<\/span><span>  port = 80\n<\/span><span>  protocol = &quot;HTTP&quot;\n<\/span><span>}\n<\/span><span>\n<\/span><span># An example of a Listener rule\n<\/span><span>resource &quot;aws_alb_listener_rule&quot; &quot;rule-1&quot; {\n<\/span><span>  action {\n<\/span><span>    target_group_arn = &quot;${aws_alb_target_group.target-group-1.arn}&quot;\n<\/span><span>    type = &quot;forward&quot;\n<\/span><span>  }\n<\/span><span>\n<\/span><span>  condition { field=&quot;path-pattern&quot; values=[&quot;\/api\/1\/resolve\/default&quot;] }\n<\/span><span>\n<\/span><span>  listener_arn = &quot;${aws_alb_listener.my-alb-listener.id}&quot;\n<\/span><span>  priority = 100\n<\/span><span>}\n<\/span><\/code><\/pre>\n<h3 id=\"conclusion\">Conclusion<\/h3>\n<p>As you can see above, each target group has a check and the ALB listener rules decide which group to send the request to based on the rules like PATH, Host header etc. What makes it more convenient is that you always have a default rule that is a catch-all. Also, you can have multiple conditions in a rule like<\/p>\n<pre data-lang=\"terraform\" style=\"background-color:#eff1f5;color:#4f5b66;\" class=\"language-terraform \"><code class=\"language-terraform\" data-lang=\"terraform\"><span>resource &quot;aws_alb_listener_rule&quot; &quot;multi-condition-rule&quot; {\n<\/span><span>  &quot;action&quot; {\n<\/span><span>    target_group_arn = &quot;${aws_alb_target_group.my-specific-target-group.arn}&quot;\n<\/span><span>    type = &quot;forward&quot;\n<\/span><span>  }\n<\/span><span>\n<\/span><span>  condition { field=&quot;path-pattern&quot;  values=[&quot;\/api\/1\/resolve\/default&quot;]               }\n<\/span><span>  condition { field=&quot;host-header&quot;   values=[&quot;example.org&quot;]  }\n<\/span><span>  \n<\/span><span>  listener_arn = &quot;${aws_alb_listener.my-listener.id}&quot;\n<\/span><span>  priority = 108\n<\/span><span>}\n<\/span><\/code><\/pre>\n<p>Finally, here is a sort of semi-complete Terraform code to get you some idea. I have also provided the code as <a href=\"https:\/\/gist.github.com\/amanjeev\/7ad52fed4a1ee2d38ebf08023d943e53\">Github Gist: AWS Auto Scaling Group with Application Load Balancer using Terraform<\/a>.<\/p>\n<pre data-lang=\"terraform\" style=\"background-color:#eff1f5;color:#4f5b66;\" class=\"language-terraform \"><code class=\"language-terraform\" data-lang=\"terraform\"><span># Create a basic ALB \n<\/span><span>resource &quot;aws_alb&quot; &quot;my-app-alb&quot; {\n<\/span><span>  name = &quot;my-app-alb&quot;\n<\/span><span>}\n<\/span><span>\n<\/span><span># Create target groups with one health check per group\n<\/span><span>resource &quot;aws_alb_target_group&quot; &quot;target-group-1&quot; {\n<\/span><span>  name = &quot;target-group-1&quot;\n<\/span><span>  port = 80\n<\/span><span>  protocol = &quot;HTTP&quot;\n<\/span><span>\n<\/span><span>  lifecycle { create_before_destroy=true }\n<\/span><span>\n<\/span><span>  health_check {\n<\/span><span>    path = &quot;\/api\/1\/resolve\/default?path=\/service\/my-service&quot;\n<\/span><span>    port = 2001\n<\/span><span>    healthy_threshold = 6\n<\/span><span>    unhealthy_threshold = 2\n<\/span><span>    timeout = 2\n<\/span><span>    interval = 5\n<\/span><span>    matcher = &quot;200&quot;\n<\/span><span>  }\n<\/span><span>}\n<\/span><span>\n<\/span><span>resource &quot;aws_alb_target_group&quot; &quot;target-group-2&quot; {\n<\/span><span>  name = &quot;target-group-2&quot;\n<\/span><span>  port = 80\n<\/span><span>  protocol = &quot;HTTP&quot;\n<\/span><span>\n<\/span><span>  lifecycle { create_before_destroy=true }\n<\/span><span>\n<\/span><span>  health_check {\n<\/span><span>    path = &quot;\/api\/2\/resolve\/default?path=\/service\/my-service&quot;\n<\/span><span>    port = 2010\n<\/span><span>    healthy_threshold = 6\n<\/span><span>    unhealthy_threshold = 2\n<\/span><span>    timeout = 2\n<\/span><span>    interval = 5\n<\/span><span>    matcher = &quot;200&quot;\n<\/span><span>  }\n<\/span><span>}\n<\/span><span>\n<\/span><span># Create a Listener \n<\/span><span>resource &quot;aws_alb_listener&quot; &quot;my-alb-listener&quot; {\n<\/span><span>  default_action {\n<\/span><span>    target_group_arn = &quot;${aws_alb_target_group.target-group-1.arn}&quot;\n<\/span><span>    type = &quot;forward&quot;\n<\/span><span>  }\n<\/span><span>  load_balancer_arn = &quot;${aws_alb.my-app-alb.arn}&quot;\n<\/span><span>  port = 80\n<\/span><span>  protocol = &quot;HTTP&quot;\n<\/span><span>}\n<\/span><span>\n<\/span><span># Create Listener Rules\n<\/span><span>resource &quot;aws_alb_listener_rule&quot; &quot;rule-1&quot; {\n<\/span><span>  action {\n<\/span><span>    target_group_arn = &quot;${aws_alb_target_group.target-group-1.arn}&quot;\n<\/span><span>    type = &quot;forward&quot;\n<\/span><span>  }\n<\/span><span>\n<\/span><span>  condition { field=&quot;path-pattern&quot; values=[&quot;\/api\/1\/resolve\/default&quot;] }\n<\/span><span>\n<\/span><span>  listener_arn = &quot;${aws_alb_listener.my-alb-listener.id}&quot;\n<\/span><span>  priority = 100\n<\/span><span>}\n<\/span><span>\n<\/span><span>resource &quot;aws_alb_listener_rule&quot; &quot;rule-2&quot; {\n<\/span><span>  action {\n<\/span><span>    target_group_arn = &quot;${aws_alb_target_group.target-group-2.arn}&quot;\n<\/span><span>    type = &quot;forward&quot;\n<\/span><span>  }\n<\/span><span>\n<\/span><span>  condition { field=&quot;path-pattern&quot; values=[&quot;\/api\/2\/resolve\/default&quot;] }\n<\/span><span>\n<\/span><span>  listener_arn = &quot;${aws_alb_listener.my-alb-listener.id}&quot;\n<\/span><span>  priority = 101\n<\/span><span>}\n<\/span><span>\n<\/span><span># Create an ASG that ties all of this together\n<\/span><span>resource &quot;aws_autoscaling_group&quot; &quot;my-alb-asg&quot; {\n<\/span><span>  name = &quot;my-alb-asg&quot;\n<\/span><span>  min_size = &quot;3&quot;\n<\/span><span>  max_size = &quot;6&quot;\n<\/span><span>  launch_configuration = &quot;${aws_launch_configuration.my-app-alb.name}&quot;\n<\/span><span>  termination_policies = [\n<\/span><span>    &quot;OldestInstance&quot;,\n<\/span><span>    &quot;OldestLaunchConfiguration&quot;,\n<\/span><span>  ]\n<\/span><span>  \n<\/span><span>  health_check_type = &quot;ELB&quot;\n<\/span><span>\n<\/span><span>  depends_on = [\n<\/span><span>    &quot;aws_alb.my-app-alb&quot;,\n<\/span><span>  ]\n<\/span><span>\n<\/span><span>  target_group_arns = [\n<\/span><span>    &quot;${aws_alb_target_group.target-group-1.arn}&quot;,\n<\/span><span>    &quot;${aws_alb_target_group.target-group-2.arn}&quot;,\n<\/span><span>  ]\n<\/span><span>\n<\/span><span>  lifecycle {\n<\/span><span>    create_before_destroy = true\n<\/span><span>  }\n<\/span><span>}\n<\/span><\/code><\/pre>\n<h3 id=\"further-todo\">Further ToDo<\/h3>\n<ul>\n<li>Learn more about Network Load Balancer.<\/li>\n<li>Comparison to GCP and Azure.<\/li>\n<\/ul>\n"},{"title":"RustConf 2018 Portland OR","published":"2018-09-03T00:00:00+00:00","updated":"2018-09-03T00:00:00+00:00","author":{"name":"\n            \n              Unknown\n            \n          "},"link":{"@attributes":{"rel":"alternate","type":"text\/html","href":"https:\/\/amanjeev.com\/blog\/rustconf-2018-portland-oregon\/"}},"id":"https:\/\/amanjeev.com\/blog\/rustconf-2018-portland-oregon\/","content":"<p>RustConf has been the first ever conference for a programming language that I was eager to attend. Since, it\u2019s a new-ish language, the conference was not as big as, say, PyCon or KubeCon would be. This made it easy to talk to people you generally only know via Twitter. Also, the duration of conference was one day (two if you include the training day). During this conference I met three kinds of rustaceans \u2014<\/p>\n<ol>\n<li>Core team.<\/li>\n<li>Using Rust at work.<\/li>\n<li>Trying to push my boss to use Rust at work.<\/li>\n<\/ol>\n<p>Out of the three categories, most folks I met were in #3, including myself. I met a bunch of enthusiasts who just showed up and I like that.<\/p>\n<br>\n<blockquote class=\"twitter-tweet\"><p lang=\"en\" dir=\"ltr\">\u201cHello everyone! Welcome to <a href=\"https:\/\/twitter.com\/rustconf?ref_src=twsrc%5Etfw\">@rustconf<\/a> everyone! How many of you is this your first time at RustConf? [almost the whole room raises their hands] Holy s***!\u201d \u2014<a href=\"https:\/\/twitter.com\/arshia__?ref_src=twsrc%5Etfw\">@arshia__<\/a> \ud83d\ude06<\/p>&mdash; Chris Krycho (@chriskrycho) <a href=\"https:\/\/twitter.com\/chriskrycho\/status\/1030497830731886592?ref_src=twsrc%5Etfw\">August 17, 2018<\/a><\/blockquote> <script async src=\"https:\/\/platform.twitter.com\/widgets.js\" charset=\"utf-8\"><\/script>\n<br>\nI have never been a part of community that I feel was working for this level of inclusivity. The community and the language, both support the philosophy of \u201cHack without fear\u201d. The level of commitment from core team and the community organizers resulted in a very diverse conference, compared to other conferences I have been to. I hope this will continue to get better and the community will be supportive.\n<h3 id=\"training\">training<\/h3>\n<p>I am a big proponent of good training days in conferences. These are some of the best days for learning via hands-on.<\/p>\n<h4 id=\"traits-and-threads-by-aaron-turon\">traits and threads by Aaron Turon<\/h4>\n<ul>\n<li><a href=\"https:\/\/twitter.com\/aaron_turon\">Aaron Turon on Twitter<\/a><\/li>\n<li><a href=\"http:\/\/www.rust-tutorials.com\/exercises\/\">Tutorial with code<\/a><\/li>\n<li><a href=\"https:\/\/doc.rust-lang.org\/book\/second-edition\/ch16-01-threads.html\">Traits - Rust book<\/a><\/li>\n<\/ul>\n<p>Aaron is a mindful tutor. This workshop was beginner-friendly. Even though the description says that \u201ctutorial assumes basic understanding of ownership and borrowing\u201d, Aaron still gave a refresher.<\/p>\n<p>To begin, If you do not know about threads I recommend that you start with <a href=\"https:\/\/en.wikipedia.org\/wiki\/Thread_(computing)\">threads on Wikipedia<\/a> and read more about <a href=\"https:\/\/doc.rust-lang.org\/book\/second-edition\/ch16-01-threads.html\">threads in Rust book<\/a>. As you will read through that page, you will note that Rust wants to keep the runtime really small (\u2764\ufe0f). Rust\u2019s thread model is not <em>green<\/em> (used to be, but not anymore). That is, it has one OS thread mapped to one language thread.<\/p>\n<p>Uses of <em>threads<\/em> is mostly to facilitate parallelism in the program. In the workshop, I learnt in hands-on style that Rust\u2019s parallelism supports multiple paradigms, like Message passing and Mutable shared memory. Even though these methods have issues in general, like <em>data races<\/em>, Rust\u2019s borrow-checking and ownership makes sure that the compiler helps you eliminate potential <em>data races<\/em> early on.<\/p>\n<p><em>Traits<\/em> are Rust\u2019s <code class=\"inline\">interfaces<\/code> in many ways although not exactly. Most of the idiomatic Rust revolves around defining and using <em>types<\/em>. Usually, the <em>types<\/em> need to have some functions associated with them to be more useful. <em>Traits<\/em> allow you to define certain <em>methods<\/em> in them and you can <em>implement<\/em> these <em>Traits<\/em> for any <em>type<\/em> of your choice.<\/p>\n<p><a href=\"http:\/\/www.rust-tutorials.com\/exercises\/\">See all the tutorials for yourself<\/a> if you want some hands-on with Traits and Threads.<\/p>\n<h4 id=\"futures-and-async-programming-by-michael-gattozzi\">futures and async programming by Michael Gattozzi<\/h4>\n<ul>\n<li><a href=\"https:\/\/twitter.com\/mgattozzi\">Michael Gattozzi on Twitter<\/a><\/li>\n<li><a href=\"https:\/\/mgattozzi.com\/classes\/run-await-with-me\">Slides<\/a><\/li>\n<li><a href=\"https:\/\/github.com\/mgattozzi\/async-await-class\">Exercise code<\/a><\/li>\n<li><a href=\"https:\/\/github.com\/tokio-rs\/tokio\">Tokio: runtime for writing reliable, asynchronous, and slim applications with the Rust<\/a><\/li>\n<li><a href=\"https:\/\/github.com\/rust-lang-nursery\/futures-rs\">Futures-rs: Zero-cost asynchronous programming in Rust<\/a><\/li>\n<\/ul>\n<p>With this workshop, Michael took me to a place in Rustland, where I was afraid to go on my own. Up until I saw this workshop listed on RustConf\u2019s website, I was not sure whether I want to even try looking at Async stuff in Rust, given how little I know about Rust at the moment. Did I mention that Michael was very funny which is a good strategy to relax everyone.<\/p>\n<p>The workshop, for majority, dug deep into the details and components of Async in Rust which is helpful if you want to understand what\u2019s going on under the hood and at the same time how much work it requires to do manually. This is directly opposite to when the construct and semantics are provided by the language itself, which is where Rust is going, IIRC.<\/p>\n<p>I encourage you to go through <a href=\"https:\/\/mgattozzi.com\/classes\/run-await-with-me\">the slides<\/a>, as well as the <a href=\"https:\/\/github.com\/mgattozzi\/async-await-class\">exercise itself<\/a>. They are clear and helpful.<\/p>\n<h3 id=\"conference-day\">conference day<\/h3>\n<p>I am listing some of the resources I found during the talks. I am not going to write about the talks themselves as they will be online and you must watch them. \ud83d\ude0a<\/p>\n<ul>\n<li><a href=\"https:\/\/github.com\/japaric\/criterion.rs\">criterion.rs: Statistics-driven micro-benchmarking library<\/a>. Via \u201cBenchmarking and Optimization of Rust Libraries By Paul Mason\u201d<\/li>\n<li>The core ideas of Rust \u2014 Ownership and Borrowing \u2014 can be applied to hardware as well. Via \u201cGetting Something for Nothing By James Munns\u201d<\/li>\n<li><a href=\"https:\/\/github.com\/pingcap\/raft-rs\">raft-rs: Raft distributed consensus algorithm implemented in Rust<\/a>. Via \u201cUsing Raft in Rust By Siddon Tang\u201d<\/li>\n<li><a href=\"https:\/\/mozilla.github.io\/mentat\/\">Project Mentat: A relational (not key-value, not document-oriented) store that makes it easy to describe, grow, and reuse your domain schema<\/a>. Via \u201cProject Mentat: a store for evolving data in Rust By Emily Toop\u201d<\/li>\n<li>Just the talk \u201cEmbedding Rust in C\/C++ By Katharina\u201d. Just watch this talk. Just do it.<\/li>\n<li><a href=\"https:\/\/c2rust.com\/\">c2rust: a tool to translate C to semantically equivalent Rust<\/a>. Via \u201cC2Rust: Migrating Legacy Code to Rust By Per Larsen\u201d<\/li>\n<\/ul>\n"},{"title":"Terraform and YAML related notes","published":"2018-08-10T00:00:00+00:00","updated":"2018-08-10T00:00:00+00:00","author":{"name":"\n            \n              Unknown\n            \n          "},"link":{"@attributes":{"rel":"alternate","type":"text\/html","href":"https:\/\/amanjeev.com\/blog\/terraform-and-yaml\/"}},"id":"https:\/\/amanjeev.com\/blog\/terraform-and-yaml\/","content":"<p>If you manage applications, you likely come across a ton of structured data that stays mostly static but must be tracked, version-controlled and be visible, especially when someone modifies it. For some folks, that is a CMS, for others a stack of papyri and for some more it is just a long oral tradition of chanting or yelling. I, however, prefer readable, structured text files. That's why I like the idea of <a href=\"https:\/\/www.terraform.io\/\">Terraform<\/a> or infrastructure-as-code.\n<br>\nLast year, we faced a problem for management of some of our infrastructure-data. We wanted to manage the <a href=\"https:\/\/www.consul.io\/intro\/getting-started\/kv.html\">Consul KV store<\/a> data via Terraform and not depend on Consul's UI. The problem was that there was no default Terraform provider for the particular task of manipulation of <a href=\"https:\/\/www.consul.io\/intro\/getting-started\/kv.html\">Consul KV store<\/a>. There are provider resources like \"<a href=\"https:\/\/www.terraform.io\/docs\/providers\/consul\/r\/keys.html\">consul_keys<\/a>\" and \"<a href=\"https:\/\/www.terraform.io\/docs\/providers\/consul\/d\/key_prefix.html\">consul_key_prefix<\/a>\" but they alone were not adequate for a cleaner solution. We have multiple datacenters and it is tiresome to edit keys and values when needed. The data had a ton of repetition across various datacenters with some common parts. Moreover, Web UI does not keep a record of edits. As a result, the visibility within the team was close to zero.\n<br>\nGiven the lack of any such plugin, at that time, we went on a path to use <a href=\"https:\/\/www.terraform.io\/docs\/providers\/external\/data_source.html\">Terraform's external data source<\/a> and decided to store the KV data in a <a href=\"http:\/\/yaml.org\/\">YAML<\/a> file which will be read by the external data source and will populate the \"consul_key_prefix\". First, external data source is your custom program that reads from stdin and spits out on stdout, both as JSON objects. The official documentation for this \"external data source\" provider shows that you define it as<\/p>\n<pre data-lang=\"terraform\" style=\"background-color:#eff1f5;color:#4f5b66;\" class=\"language-terraform \"><code class=\"language-terraform\" data-lang=\"terraform\"><span>data &quot;external&quot; &quot;example&quot; {\n<\/span><span>  # Call &quot;python \/path\/to\/mod\/example.py&quot;\n<\/span><span>  program = [\n<\/span><span>      &quot;python&quot;, \n<\/span><span>      &quot;${path.module}\/example.py&quot;\n<\/span><span>    ]\n<\/span><span>\n<\/span><span>  # with JSON query\n<\/span><span>  query = {\n<\/span><span>    id = &quot;abc123&quot;\n<\/span><span>  }\n<\/span><span>}\n<\/span><span>\n<\/span><\/code><\/pre>\n<p>In the problem that I stated above, it was decided to store the Consul Key\/Value pairs as nested YAML file for each datacenter and a common section for common keys. Something like:<\/p>\n<pre data-lang=\"yaml\" style=\"background-color:#eff1f5;color:#4f5b66;\" class=\"language-yaml \"><code class=\"language-yaml\" data-lang=\"yaml\"><span style=\"color:#bf616a;\">common<\/span><span>:\n<\/span><span>    <\/span><span style=\"color:#bf616a;\">key_common_1<\/span><span>: <\/span><span style=\"color:#a3be8c;\">value_common_1\n<\/span><span>\n<\/span><span style=\"color:#bf616a;\">dc1<\/span><span>:\n<\/span><span>    <\/span><span style=\"color:#d08770;\">&lt;&lt;<\/span><span>: <\/span><span style=\"color:#b48ead;\">*<\/span><span style=\"color:#bf616a;\">common\n<\/span><span>\n<\/span><span>    <\/span><span style=\"color:#bf616a;\">key11<\/span><span>: <\/span><span style=\"color:#a3be8c;\">value11\n<\/span><span>    <\/span><span style=\"color:#bf616a;\">key12<\/span><span>: <\/span><span style=\"color:#a3be8c;\">value12\n<\/span><span>    <\/span><span style=\"color:#bf616a;\">key13<\/span><span>:\n<\/span><span>        <\/span><span style=\"color:#bf616a;\">key131<\/span><span>: <\/span><span style=\"color:#a3be8c;\">value131\n<\/span><span>\n<\/span><span style=\"color:#bf616a;\">dc2<\/span><span>:\n<\/span><span>    <\/span><span style=\"color:#d08770;\">&lt;&lt;<\/span><span>: <\/span><span style=\"color:#b48ead;\">*<\/span><span style=\"color:#bf616a;\">common\n<\/span><span>\n<\/span><span>    <\/span><span style=\"color:#bf616a;\">key21<\/span><span>: <\/span><span style=\"color:#a3be8c;\">value21\n<\/span><span>    <\/span><span style=\"color:#bf616a;\">key22<\/span><span>: <\/span><span style=\"color:#a3be8c;\">value22\n<\/span><span>    <\/span><span style=\"color:#bf616a;\">key23<\/span><span>:\n<\/span><span>        <\/span><span style=\"color:#bf616a;\">key231<\/span><span>: <\/span><span style=\"color:#a3be8c;\">value231\n<\/span><\/code><\/pre>\n<p>The wonderful part of YAML is that it is machine-and-human-readable, and can succinctly represent hierarchy in a document. Hence, it is well-suited for the representation of various datacenters and their respective KV data, which is what we wanted to do. Take the key \"key131\" in \"Datacenter 1\". You can depict the \"path\" to \"key131\" as Consul KV via <code>dc1\/key13\/key131<\/code><\/p>\n<p>Now, comes the question of what to do with external datasource. You can write that in any language\/platform that can read from stdin and write to stdout. In the official documentation they use bash. However, I chose Python for its simplicity, good YAML support and JSON interoperability. Also, I could get help from some team members if I ever got stuck. With these tools, the last thing I want to add is that you can use Python's <a href=\"https:\/\/flatdict.readthedocs.io\/\">flatdict<\/a> with delimiter=\"\/\" as<\/p>\n<pre data-lang=\"python\" style=\"background-color:#eff1f5;color:#4f5b66;\" class=\"language-python \"><code class=\"language-python\" data-lang=\"python\"><span style=\"color:#b48ead;\">import <\/span><span>flatdict\n<\/span><span>\n<\/span><span>\n<\/span><span style=\"color:#b48ead;\">with <\/span><span style=\"color:#96b5b4;\">open<\/span><span>(file_path, &#39;<\/span><span style=\"color:#a3be8c;\">r<\/span><span>&#39;) <\/span><span style=\"color:#b48ead;\">as <\/span><span>yf:\n<\/span><span>    yaml_dict = flatdict.<\/span><span style=\"color:#bf616a;\">FlatDict<\/span><span>(yaml.<\/span><span style=\"color:#bf616a;\">load<\/span><span>(yf), \n<\/span><span>                                  <\/span><span style=\"color:#bf616a;\">delimiter<\/span><span>=&#39;<\/span><span style=\"color:#a3be8c;\">\/<\/span><span>&#39;)\n<\/span><\/code><\/pre>\n<p>This creates a flattened dictionary, each key of which is the \"path\" with segments delimited by \"\/\". Example <code>dc1\/key13\/key131<\/code><\/p>\n<p>All that is left now is manipulation of the above dict if needed and writing JSON to stdout so it can be read by the Terraform resource \"consul_key_prefix\". \ud83d\ude07<\/p>\n<pre data-lang=\"terraform\" style=\"background-color:#eff1f5;color:#4f5b66;\" class=\"language-terraform \"><code class=\"language-terraform\" data-lang=\"terraform\"><span>resource &quot;consul_key_prefix&quot; &quot;keys&quot; {\n<\/span><span>  datacenter = &quot;${var.datacenter}&quot;\n<\/span><span>  path_prefix = &quot;${var.prefix}${var.path}&quot;\n<\/span><span>  subkeys = &quot;${data.external.mydata.result}&quot;\n<\/span><span>}\n<\/span><\/code><\/pre>\n<p>But, I am not done telling you about this yet. My friend and colleague <a href=\"https:\/\/ashald.net\/\">Borys Pierov<\/a> wrote new set of Terraform provider plugins because there was a need for a good Consul ACL management provider. He abstracted a bunch of stuff into independent plugins so you can go from flexible to powerful, if you want.<\/p>\n<ul>\n<li><a href=\"https:\/\/github.com\/Ashald\/terraform-provider-transform\">terraform-provider-transform<\/a>: Terraform data sources providing data transformations missing from core Terraform<\/li>\n<li><a href=\"https:\/\/github.com\/Ashald\/terraform-provider-stateful\">terraform-provider-stateful<\/a>: Generic abstract stateful resources to manage arbitrary objects by executing arbitrary commands<\/li>\n<li><a href=\"https:\/\/github.com\/Ashald\/terraform-provider-yaml\">terraform-provider-yaml<\/a>: Terraform data source that can consume YAML input<\/li>\n<li><a href=\"https:\/\/github.com\/Ashald\/terraform-provider-consulacl\">terraform-provider-consulacl<\/a>: Consul ACL Terraform Provider<\/li>\n<\/ul>\n<p>So, if you use Consul and you need to manage KV or ACL, you do not have any excuses to not use the above tools. Please let me know if I missed something or there are alternate approaches that you've taken. Thanks for reading. \ud83d\ude0a<\/p>\n"},{"title":"Leibniz\u2019s monads","published":"2016-02-14T00:00:00+00:00","updated":"2016-02-14T00:00:00+00:00","author":{"name":"\n            \n              Unknown\n            \n          "},"link":{"@attributes":{"rel":"alternate","type":"text\/html","href":"https:\/\/amanjeev.com\/blog\/leibniz-monads\/"}},"id":"https:\/\/amanjeev.com\/blog\/leibniz-monads\/","content":"<p><img src=\"https:\/\/upload.wikimedia.org\/wikipedia\/commons\/6\/6a\/Gottfried_Wilhelm_von_Leibniz.jpg\" alt=\"Photo image of Gottfried Wilhelm von Leibniz by Christoph Bernhard Francke, via Wikimedia Commons\" title=\"Photo image of Gottfried Wilhelm von Leibniz by Christoph Bernhard Francke, via Wikimedia Commons\" \/><\/p>\n<p>I recently discovered the philosophy of Gottfried Wilhelm Leibniz, the polymath and philosopher. Some believe that he developed Calculus independent of Sir Isaac Newton, as a mathematician. We have been using <a href=\"https:\/\/en.wikipedia.org\/wiki\/Leibniz%27s_notation\">Leibniz\u2019s notation<\/a>, in the infinitesimal calculus. He also contributed to what we know today as modern logic, binary system etc. It is his infinitesimal calculus and its relation with his philosophy that I find fascinating. The way he thought about the world can be seen in his math work, especially his calculus.<\/p>\n<p>\u201cMonadology\u201d is his seminal philosophy work. Monad, the idea and the word itself, has existed in philosophy for centuries. Pythagoreans are attribute for its origin. <a href=\"https:\/\/en.wikipedia.org\/wiki\/Monad_(philosophy)\">According to Wikipedia<\/a> \u2014<\/p>\n<blockquote>\n<p>Monad refers in cosmogony (creation theories) to the first being, divinity, or the totality of all beings. The concept was reportedly conceived by the Pythagoreans and may refer variously to a single source acting alone and\/or an indivisible origin.<\/p>\n<\/blockquote>\n<p>Monad as a concept was taken and enhanced by Leibniz where he referred to Monad as an <em>elementary particle<\/em>. He calls Monads as the \u201creal atoms of nature\u201d. According to Leibniz, Monads \u2014<\/p>\n<ul>\n<li>are indivisible<\/li>\n<li>can neither be made, nor destroyed<\/li>\n<li>do not have parts<\/li>\n<li>do not occupy space<\/li>\n<li>may possess qualities<\/li>\n<li>can cause changes to one another<\/li>\n<li>come into being by creation<\/li>\n<li>destroyed by annihilation<\/li>\n<\/ul>\n<p>Leibniz considered these infinite number of Monads made up the space. It means that the space is not empty, as Newton had suggested, but has Monads and since they indivisible, but do not take space, they are the basis for all the other \u2018stuff\u2019 somehow by interacting with each other. The part of Monads being indivisible is the key. That makes them infinitesimal, that cannot be detected by any apparatus we can build. Thus, we can see how the above is similar to his calculus work which deals with<\/p>\n<blockquote>\n<p>Sum of infinite number of infinitesimals equate a finite amount.<\/p>\n<\/blockquote>\n<p>As Ruth Lydia Saw writes in the best way possible \u2014<\/p>\n<blockquote>\n<p>If we consider his own discoveries in mathematics, we shall see how marvellously it must have seemed to Leibniz that everything pointed to and reinforced his metaphysical account of universe. For him, the universe is constituted by an infinite number of non-spatial beings presenting the appearance of large bodies moving in space. At the same time, the invention of the microscope shows minute differences which are lost in large perception and leads to possibility of ever more minute differences to be revealed by more powerful instruments. Then Leibniz himself invents the infinitesimal calculus which is based on the possibility and the usefulness of treating a finite amount as equivalent to the sum of an infinite number of infinitesimal amounts.<\/p>\n<\/blockquote>\n<p>Note: If you were looking for Monad in functional programming then please refer to <a href=\"https:\/\/en.wikipedia.org\/wiki\/Monad_(functional_programming)\">Wikipedia<\/a> or <a href=\"https:\/\/curiosity-driven.org\/monads-in-javascript\">Monads in JavaScript<\/a> or <a href=\"https:\/\/wiki.haskell.org\/Monad\">Monad in Haskell<\/a> or maybe <a href=\"https:\/\/stackoverflow.com\/questions\/2704652\/monad-in-plain-english-for-the-oop-programmer-with-no-fp-background\">Stackoverflow<\/a>.<\/p>\n"}]}