{"@attributes":{"version":"2.0"},"channel":{"title":{},"link":"https:\/\/remy.io\/","description":"Recent content on ","generator":"Hugo -- gohugo.io","language":"en-US","managingEditor":"contact@remy.io (R\u00e9my Mathieu)","webMaster":"contact@remy.io (R\u00e9my Mathieu)","lastBuildDate":"Mon, 01 Jun 2026 00:00:00 +0000","item":[{"title":"A Coding Benchmark: Step 3.7 vs Qwen 3.5 122B-A10B vs Qwen 3.6 27B vs Qwen 3.6 35B-A3B","link":"https:\/\/remy.io\/blog\/coding-benchmark-qwen-step\/","pubDate":"Mon, 01 Jun 2026 00:00:00 +0000","author":"contact@remy.io (R\u00e9my Mathieu)","guid":"https:\/\/remy.io\/blog\/coding-benchmark-qwen-step\/","description":"<h1 id=\"introduction\">Introduction<\/h1>\n<p>I recently ran a simple real-world benchmark with 4 different models, all quants from <a href=\"https:\/\/unsloth.ai\">Unsloth<\/a>:<\/p>\n<ul>\n<li>Step 3.7 - IQ4_XS<\/li>\n<li>Qwen 3.5 122B-A10B - UD-Q4_K_XL<\/li>\n<li>Qwen 3.6 27B - Q5_K_M<\/li>\n<li>Qwen 3.6 35B-A3B - UD-Q6_K<\/li>\n<\/ul>\n<p>The benchmark is quite simple: I have my own coding harness developed in Go, I want to add a <code>\/config<\/code> command displaying the runtime value of all configuration fields, and from where the value came from (environment variable, configuration file or default value).<\/p>\n<p>In order to implement this, I first asked Step 3.7 to write a plan, here it is the plan it wrote for reference:<\/p>\n<pre><code># PLAN-config.md\n\n## \/config Command\n\nDisplay the effective configuration from inside the TUI so the user can inspect model, endpoint, context limits, and other active settings without quitting.\n\n## Changes\n\n### `internal\/app\/app.go`\n\n1. **Add `case &quot;config&quot;:` in `handleCommand` (line ~809)**\n\n   Parse a `show` subcommand. Default to `show` when no subcommand is given.\n\n   ```go\n   case &quot;config&quot;:\n       sub := &quot;show&quot;\n       if len(parts) &gt; 1 {\n           sub = parts[1]\n       }\n       switch sub {\n       case &quot;show&quot;:\n           return m.handleConfigShow(), true\n       default:\n           msg := types.NewSystemMessage(&quot;Usage: \/config [show]&quot;)\n           m.Messages = append(m.Messages, msg)\n           m.Screen.GetMessageView().MessageGrew()\n           return nil, true\n       }\n   ```\n\n2. **Add `handleConfigShow` method**\n\n   Format key config values as a readable message. Mask the bearer token (show only last 4 chars or `***`). Indicate the source if it differs from the default (e.g. `[env]`, `[yaml]`, `[cli]`) \u2014 the config loader already knows the origin, so expose that via the `Config` struct or a separate `source` field.\n\n   Fields to display:\n\n   | Field | Notes |\n   |---|---|\n   | `Model` | always shown |\n   | `Endpoint` | always shown |\n   | `BearerToken` | masked \u2014 last 4 chars only |\n   | `CWD` | working directory |\n   | `MaxContextTokens` | context window size |\n   | `CompactionThreshold` | e.g. `0.8` |\n   | `KeepRecentTokens` | preserved after compaction |\n   | `MaxTokens` | response token limit |\n   | `SystemPrompt` | show length + first 80 chars, not the full text |\n   | `Debug` | on\/off |\n   | `UILogPath` | log file location |\n   | `EnableMultiline` | on\/off |\n   | `EnableAutocomplete` | on\/off |\n   | `AutocompleteMaxItems` | max suggestions |\n   | `TabCompletes` | on\/off |\n\n3. **Update help text (line ~841)**\n\n   Add:\n\n   ```\n   helpText += &quot;  \/config        - Show effective configuration\\n&quot;\n   ```\n\n### `internal\/app\/config_types.go`\n\n4. **Add `Source` field to `Config`**\n\n   Track where each value came from (default, yaml, env, cli) so `\/config show` can display it. Add a `source` map or per-field tags:\n\n   ```go\n   type ConfigSource string\n\n   const (\n       SourceDefault ConfigSource = &quot;default&quot;\n       SourceYAML     ConfigSource = &quot;yaml&quot;\n       SourceEnv      ConfigSource = &quot;env&quot;\n       SourceCLI      ConfigSource = &quot;cli&quot;\n   )\n   ```\n\n   Either embed a `map[string]ConfigSource` in `Config` or add a `GetSource(key string) ConfigSource` method. Keep it minimal \u2014 only needed for display.\n\n### `internal\/app\/config.go`\n\n5. **Populate source tracking in `LoadConfig` \/ `ApplyFlags`**\n\n   When a value is overridden, record the source. The loader already has the precedence chain (yaml \u2192 env \u2192 cli), so this is a straightforward addition alongside the existing assignment logic.\n\n## Notes\n\n- Bearer token masking: show `...xxxx` or empty string if not set. Never print the full token.\n- System prompt: only show length and a truncated preview to avoid flooding the message view.\n- Source display is additive \u2014 a `[env]` or `[cli]` tag next to the value makes it easy to understand why a non-default value is active.\n- Keep it read-only. No config mutation from `\/config` for now (that's a future stretch).\n<\/code><\/pre>\n<p>The first observation I can make is that the Step 3.7 prose is really different from Qwen&rsquo;s prose. Not that it is any better or worse, but it is a bit different and I like it.<\/p>\n<p>I then used <a href=\"https:\/\/remy.io\/blog\/local-inference-strix-halo-128gb\/\">llama.cpp on a Strix Halo<\/a> to implement this plan, one model at a time.<\/p>\n<h1 id=\"results-and-interpretation\">Results and interpretation<\/h1>\n\n\n\n\n\n\n\n\n  \n    \n    \n    \n      \n      \n      \n    \n  \n    \n    \n    \n      \n      \n      \n    \n  \n\n\n<table style=\"width: 100%; border-collapse: collapse; margin: 2rem 0; font-size: 0.95rem; font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, sans-serif;\">\n  <thead>\n    <tr style=\"background-color: #f5f5f5;\">\n      \n      <th style=\"border: 1px solid #e0e0e0; padding: 12px 16px; text-align: center; font-weight: 600; color: #333;\">Criteria<\/th>\n      \n      <th style=\"border: 1px solid #e0e0e0; padding: 12px 16px; text-align: center; font-weight: 600; color: #333;\">Step 3.7<\/th>\n      \n      <th style=\"border: 1px solid #e0e0e0; padding: 12px 16px; text-align: center; font-weight: 600; color: #333;\">Qwen 3.5 122B-A10B<\/th>\n      \n      <th style=\"border: 1px solid #e0e0e0; padding: 12px 16px; text-align: center; font-weight: 600; color: #333;\">Qwen 3.6 27B<\/th>\n      \n      <th style=\"border: 1px solid #e0e0e0; padding: 12px 16px; text-align: center; font-weight: 600; color: #333;\">Qwen 3.6 35B-A3B<\/th>\n      \n    <\/tr>\n  <\/thead>\n  <tbody>\n    \n      \n      <tr>\n        \n          \n          \n          \n            \n          \n          \n          <td style=\"border: 1px solid #e0e0e0; padding: 12px 16px; text-align: left; font-weight: 600; background-color: #f9f9f9;\">Quant<\/td>\n        \n          \n          \n          \n            \n          \n          \n          <td style=\"border: 1px solid #e0e0e0; padding: 12px 16px; text-align: center;\">IQ4_XS<\/td>\n        \n          \n          \n          \n            \n          \n          \n          <td style=\"border: 1px solid #e0e0e0; padding: 12px 16px; text-align: center;\">UD-Q4_K_XL<\/td>\n        \n          \n          \n          \n            \n          \n          \n          <td style=\"border: 1px solid #e0e0e0; padding: 12px 16px; text-align: center;\">Q5_K_M<\/td>\n        \n          \n          \n          \n            \n          \n          \n          <td style=\"border: 1px solid #e0e0e0; padding: 12px 16px; text-align: center;\">UD-Q6_K<\/td>\n        \n      <\/tr>\n    \n      \n      <tr style=\"background-color: #fafafa;\">\n        \n          \n          \n          \n            \n          \n          \n          <td style=\"border: 1px solid #e0e0e0; padding: 12px 16px; text-align: left; font-weight: 600; background-color: #f9f9f9;\">Token Counts<\/td>\n        \n          \n          \n          \n            \n          \n          \n          <td style=\"border: 1px solid #e0e0e0; padding: 12px 16px; text-align: center;\">69555<\/td>\n        \n          \n          \n          \n            \n          \n          \n          <td style=\"border: 1px solid #e0e0e0; padding: 12px 16px; text-align: center;\">67814<\/td>\n        \n          \n          \n          \n            \n          \n          \n          <td style=\"border: 1px solid #e0e0e0; padding: 12px 16px; text-align: center;\">68729<\/td>\n        \n          \n          \n          \n            \n          \n          \n            \n          \n          <td style=\"border: 1px solid #e0e0e0; padding: 12px 16px; text-align: center; color:#c62828;font-weight:600\">129280<\/td>\n        \n      <\/tr>\n    \n      \n      <tr>\n        \n          \n          \n          \n            \n          \n          \n          <td style=\"border: 1px solid #e0e0e0; padding: 12px 16px; text-align: left; font-weight: 600; background-color: #f9f9f9;\">Duration (wall time)<\/td>\n        \n          \n          \n          \n            \n          \n          \n          <td style=\"border: 1px solid #e0e0e0; padding: 12px 16px; text-align: center;\">24m36s<\/td>\n        \n          \n          \n          \n            \n          \n          \n            \n          \n          <td style=\"border: 1px solid #e0e0e0; padding: 12px 16px; text-align: center; color:#2e7d32;font-weight:600\">11m51s<\/td>\n        \n          \n          \n          \n            \n          \n          \n          <td style=\"border: 1px solid #e0e0e0; padding: 12px 16px; text-align: center;\">15m04s<\/td>\n        \n          \n          \n          \n            \n          \n          \n          <td style=\"border: 1px solid #e0e0e0; padding: 12px 16px; text-align: center;\">15m19s<\/td>\n        \n      <\/tr>\n    \n      \n      <tr style=\"background-color: #fafafa;\">\n        \n          \n          \n          \n            \n          \n          \n          <td style=\"border: 1px solid #e0e0e0; padding: 12px 16px; text-align: left; font-weight: 600; background-color: #f9f9f9;\">Fully implemented?<\/td>\n        \n          \n          \n          \n            \n          \n          \n          <td style=\"border: 1px solid #e0e0e0; padding: 12px 16px; text-align: center;\">Yes<\/td>\n        \n          \n          \n          \n            \n          \n          \n          <td style=\"border: 1px solid #e0e0e0; padding: 12px 16px; text-align: center;\">Yes<\/td>\n        \n          \n          \n          \n            \n          \n          \n          <td style=\"border: 1px solid #e0e0e0; padding: 12px 16px; text-align: center;\">Yes<\/td>\n        \n          \n          \n          \n            \n          \n          \n          <td style=\"border: 1px solid #e0e0e0; padding: 12px 16px; text-align: center;\">Yes<\/td>\n        \n      <\/tr>\n    \n      \n      <tr>\n        \n          \n          \n          \n            \n          \n          \n          <td style=\"border: 1px solid #e0e0e0; padding: 12px 16px; text-align: left; font-weight: 600; background-color: #f9f9f9;\">MTP supported in llama.cpp?<\/td>\n        \n          \n          \n          \n            \n          \n          \n          <td style=\"border: 1px solid #e0e0e0; padding: 12px 16px; text-align: center;\">No<\/td>\n        \n          \n          \n          \n            \n          \n          \n          <td style=\"border: 1px solid #e0e0e0; padding: 12px 16px; text-align: center;\">Yes<\/td>\n        \n          \n          \n          \n            \n          \n          \n          <td style=\"border: 1px solid #e0e0e0; padding: 12px 16px; text-align: center;\">Yes<\/td>\n        \n          \n          \n          \n            \n          \n          \n          <td style=\"border: 1px solid #e0e0e0; padding: 12px 16px; text-align: center;\">Yes<\/td>\n        \n      <\/tr>\n    \n      \n      <tr style=\"background-color: #fafafa;\">\n        \n          \n          \n          \n            \n          \n          \n          <td style=\"border: 1px solid #e0e0e0; padding: 12px 16px; text-align: left; font-weight: 600; background-color: #f9f9f9;\">Extra observation<\/td>\n        \n          \n          \n          \n            \n          \n          \n          <td style=\"border: 1px solid #e0e0e0; padding: 12px 16px; text-align: center;\">N\/A<\/td>\n        \n          \n          \n          \n            \n          \n          \n          <td style=\"border: 1px solid #e0e0e0; padding: 12px 16px; text-align: center;\">Slightly better than the others on adapting unit tests (simpler)<\/td>\n        \n          \n          \n          \n            \n          \n          \n          <td style=\"border: 1px solid #e0e0e0; padding: 12px 16px; text-align: center;\">Has added \/config to the commands autocompletion list<\/td>\n        \n          \n          \n          \n            \n          \n          \n          <td style=\"border: 1px solid #e0e0e0; padding: 12px 16px; text-align: center;\">N\/A<\/td>\n        \n      <\/tr>\n    \n  <\/tbody>\n<\/table>\n\n<p>It is interesting to see they all ended up generating a working implementation and with extremely similar results: it means that comparing their wall time and token counts make sense here in this use-case.<\/p>\n<p>For that exact use-case, I personally consider <code>Qwen 3.5 122B-A10B<\/code> the best for me:<\/p>\n<ol>\n<li>its wall-time is the shortest,<\/li>\n<li>adding <code>\/config<\/code> in the auto complete <strong>was not part of the plan<\/strong>, even though it sounds like <code>Qwen 3.6 27B<\/code> did better by doing this, it is a deviation from the plan!<\/li>\n<\/ol>\n<p>This is my verdict because I supervise the model results and I expect them to fully follow the plan when there is one. For someone purely vibe-coding everything and\/or not looking at results in depth, <code>Qwen 3.6 27B<\/code> could be a better choice since it did slightly better by going the extra mile.<\/p>\n<p>Some variance here and there could be a fluke, or could be a trait of the model or quant. In order to have something more reproducible in the future, I&rsquo;ll develop a testbench with a set of E2E tests and a list of criteria that will be interpreted by another [big] LLM. Stay tuned!<\/p>\n"},{"title":"Local Inference Infrastructure with AMD Strix Halo","link":"https:\/\/remy.io\/blog\/local-inference-strix-halo-128gb\/","pubDate":"Fri, 08 May 2026 00:00:00 +0000","author":"contact@remy.io (R\u00e9my Mathieu)","guid":"https:\/\/remy.io\/blog\/local-inference-strix-halo-128gb\/","description":"<h1 id=\"introduction\">Introduction<\/h1>\n<p>I wanted to build a personal LLM inference stack that doesn&rsquo;t rely on cloud providers or major tech companies.<\/p>\n<p>To achieve this, I purchased two <a href=\"https:\/\/www.corsair.com\/fr\/en\/p\/gaming-computers\/cs-9080003-pe\/corsair-ai-workstation-300-amd-ryzen-ai-max-395-processor-amd-radeon-8060s-igpu-128gb-up-to-96gb-vram-lpddr5x-memory-4tb-2tb-2tb-m2-ssd-win11-home-cs-9080003-pe\">Corsair AI Workstation 300<\/a> systems,\neach powered by an <code>AMD Ryzen AI MAX+ 395 APU<\/code> and each equipped with 128 GB of unified memory.<\/p>\n<h2 id=\"why-128gb-of-vram-changes-the-game\">Why 128GB of VRAM Changes the Game<\/h2>\n<p>The AMD Strix Halo platform uses a unified memory architecture, allowing the GPU to directly access system RAM.\nThis approach is similar to Apple Silicon&rsquo;s unified memory design on their MacBook M1 -&gt; M5.<\/p>\n<p>With 128 GB of shared memory, you can run locally hosted models that traditionally require a setup with multiple GPUs.\nFor example, the <code>Qwen3.5-122B-A10B<\/code> (a 122B parameters MoE model) fits comfortably within this headroom,\nespecially when using a quantized model.\nQuantization is about slightly reducing the precision of every parameter of the model, leading to a smaller footprint,\nbut also slightly decreasing model quality.<\/p>\n<p>Rather than trying to pool the two systems into a single 256 GB node, I want to leverage having two for parallel tasks, e.g.,\none machine can handle interactive prompts while the other runs background processes like agentic code generation or long-context reasoning.<\/p>\n<h1 id=\"goal\">Goal<\/h1>\n<p>Technically, the end goal is to have endpoints locally accessible on my network for every device that wants to run LLM inference.\nSince I can access my local network even from outside my house (using <a href=\"https:\/\/tailscale.com\/\">Tailscale<\/a>), it means I actually have access to this inference stack from anywhere.<\/p>\n<p>My three use-cases are:<\/p>\n<ul>\n<li>Agentic coding. I have a personal subscription to Claude Code (for now) and I know that local inference can&rsquo;t beat this (<a href=\"https:\/\/remy.io\/blog\/migrating-to-zig-0-16-0-with-a-local-llm\/\">despite being impressive<\/a>), still, there are usages where local agentic coding will be enough and I want to experiment.<\/li>\n<li>Web interface for prompting, with good capabilities (web search, vision, python code execution, &hellip;)<\/li>\n<li>General availability, having LLM inference for scripts, local automation, etc.<\/li>\n<\/ul>\n<p>Assistants such as OpenClaw are out of the picture, even if I would be able to run one instance easily in that infrastructure.<\/p>\n<h1 id=\"final-architecture\">Final architecture<\/h1>\n<p><em>Oh bo\u00ef<\/em>. Here is what I ended up creating:<\/p>\n<p><img src=\"https:\/\/remy.io\/img\/articles\/llm-infra\/llm-infra.webp\" alt=\"My local LLM inference infrastructure\"><\/p>\n<p><code>bob<\/code> and <code>rob<\/code> are the two Strix Halos running <a href=\"https:\/\/github.com\/ggml-org\/llama.cpp\">llama.cpp<\/a> for inference.<\/p>\n<p>As you can see, I also created (with Qwen) many small pieces for the infrastructure, but also my own coding agent <code>mok<\/code>.<\/p>\n<p><code>mehcp<\/code> is an MCP server I created exposing 4 functions: <code>fetch_url<\/code>, <code>web_search<\/code> using the Brave search API, <code>get_current_datetime<\/code> and <code>code_execute<\/code> (running python3 in a podman container), providing more capacities to the LLMs.<\/p>\n<p>Not part of this schema: every project code is hosted on my Git forge running on my NAS, which is also running an <a href=\"https:\/\/ntfy.sh\">ntfy<\/a> backend for messages and notifications.<\/p>\n<p>Mandatory picture of that stack running in my basement (temporary setup, they&rsquo;ll have their dedicated space in the basement soon):<\/p>\n<p><img src=\"https:\/\/remy.io\/img\/articles\/llm-infra\/basement_setup.webp\" alt=\"Setup of the Strix Halos in the basement\"><\/p>\n<h1 id=\"setting-up-the-environment\">Setting Up the Environment<\/h1>\n<h2 id=\"stack\">Stack<\/h2>\n<p>Running local LLM inference on Strix Halo can be approached in several ways: manual setup of a C++\/Python framework (e.g. <a href=\"https:\/\/github.com\/ggml-org\/llama.cpp\">llama.cpp<\/a>, <a href=\"https:\/\/github.com\/vllm-project\/vllm\">vLLM<\/a>, <a href=\"https:\/\/github.com\/ollama\/ollama\">Ollama<\/a>), <a href=\"https:\/\/github.com\/kyuz0\/amd-strix-halo-toolboxes\">containerized toolboxes<\/a>\nor even GUI applications (<a href=\"https:\/\/lmstudio.ai\/\">LM Studio<\/a>). The ecosystem is still consolidating, but mature enough for daily usage, still, expect some amount of maintenance in order to benefit from latest fixes and improvements.<\/p>\n<p>In this blogpost, I&rsquo;ll focus on the approach I chose: a manual installation of <code>llama.cpp<\/code>.\nThis setup provides good performance, reasonable ease of use and of maintenance, and solid compatibility with the Strix Halo hardware.<\/p>\n<h2 id=\"step-by-step-installation\">Step-by-Step Installation<\/h2>\n<p>Big shoutout to <a href=\"https:\/\/github.com\/carlosroman\">Carlos<\/a> who gave me a huge headstart on installing and configuring llama.cpp &amp; ROCm.<\/p>\n<h3 id=\"linux\">Linux<\/h3>\n<p>I installed a fresh <a href=\"https:\/\/fedoraproject.org\/\">Fedora<\/a> 43 in order to have a good mix of a stable distribution \/ recent packages and Linux kernel.<\/p>\n<p>In order to make sure the kernel would use all the VRAM, I&rsquo;ve added a few options to my grub boot options. In <code>\/etc\/default\/grub<\/code>, I appended these parameters to the <code>GRUB_CMDLINE_LINUX<\/code> field:<\/p>\n<div class=\"highlight\"><pre tabindex=\"0\" style=\"color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;\"><code class=\"language-bash\" data-lang=\"bash\"><span style=\"display:flex;\"><span>iommu<span style=\"color:#f92672\">=<\/span>pt amdgpu.gttsize<span style=\"color:#f92672\">=<\/span><span style=\"color:#ae81ff\">126976<\/span> ttm.pages_limit<span style=\"color:#f92672\">=<\/span><span style=\"color:#ae81ff\">32505856<\/span>\n<\/span><\/span><\/code><\/pre><\/div><p>Why these flags?<\/p>\n<ul>\n<li><code>iommu=pt<\/code> boosts performance by allowing devices to bypass unnecessary IOMMU translation overhead.<\/li>\n<li><code>amdgpu.gttsize=126976<\/code> tells the driver to use nearly all your system RAM as accessible VRAM for the GPU.<\/li>\n<li><code>ttm.pages_limit=32505856<\/code> increases the memory management limit so the kernel can handle the massive amount of RAM assigned to the GPU.<\/li>\n<\/ul>\n<p>You will have to make sure to run a recent Linux kernel to have the best performance possible, at least 6.12+.\nAs a reference, at the time of writing, I run the <code>6.19.13-200.fc43.x86_64<\/code> kernel.<\/p>\n<p>I had some random freezes\/unresponsive moment that I strongly suspect to come from the MT7925 wifi chip. I switched to ethernet and killed the wifi radio, no freeze happened ever since.<\/p>\n<h3 id=\"installing-rocm\">Installing ROCm<\/h3>\n<p>First, you need the ROCm stack, the AMD equivalent of CUDA from NVIDIA: the software stack for computing and AI acceleration on their GPUs.<\/p>\n<div class=\"highlight\"><pre tabindex=\"0\" style=\"color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;\"><code class=\"language-bash\" data-lang=\"bash\"><span style=\"display:flex;\"><span><span style=\"color:#75715e\"># create a Python 3.12 virtual env<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span>uv venv --python 3.12 <span style=\"color:#75715e\"># create a Python 3.12 virtualenv<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span><span style=\"color:#75715e\"># install latest ROCm (non-nightlies)<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span>uv pip install --index-url https:\/\/repo.amd.com\/rocm\/whl\/gfx1151\/ --upgrade <span style=\"color:#e6db74\">&#34;rocm[libraries,devel]&#34;<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span><span style=\"color:#75715e\"># init the SDK<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span>source .venv\/bin\/activate\n<\/span><\/span><span style=\"display:flex;\"><span>rocm-sdk init\n<\/span><\/span><\/code><\/pre><\/div><p>You now have a ROCm SDK available and ready.<\/p>\n<h3 id=\"compiling-llamacpp\">Compiling llama.cpp<\/h3>\n<p><a href=\"https:\/\/github.com\/ggml-org\/llama.cpp\">llama.cpp<\/a> is the inference framework that leverages ROCm to execute LLM workloads on the GPU.\nOn top of that, it also includes useful tools like a web-based UI prompt interface, model hot-loading capabilities and more.<\/p>\n<p>Before building it, we have to export some environment variables for <code>llama.cpp<\/code> to find the ROCm SDK we&rsquo;ve just installed:<\/p>\n<div class=\"highlight\"><pre tabindex=\"0\" style=\"color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;\"><code class=\"language-bash\" data-lang=\"bash\"><span style=\"display:flex;\"><span>export ROCM_PATH<span style=\"color:#f92672\">=<\/span><span style=\"color:#e6db74\">&#34;<\/span><span style=\"color:#66d9ef\">$(<\/span>uv run rocm-sdk path --root<span style=\"color:#66d9ef\">)<\/span><span style=\"color:#e6db74\">&#34;<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span>export HIP_DEVICE_LIB_PATH<span style=\"color:#f92672\">=<\/span><span style=\"color:#e6db74\">&#34;<\/span><span style=\"color:#66d9ef\">$(<\/span>uv run rocm-sdk path --root<span style=\"color:#66d9ef\">)<\/span><span style=\"color:#e6db74\">\/lib\/llvm\/amdgcn\/bitcode&#34;<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span>export HIPCXX<span style=\"color:#f92672\">=<\/span><span style=\"color:#e6db74\">&#34;<\/span><span style=\"color:#66d9ef\">$(<\/span>uv run rocm-sdk path --root<span style=\"color:#66d9ef\">)<\/span><span style=\"color:#e6db74\">\/llvm\/bin\/clang&#34;<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span>export HIP_PATH<span style=\"color:#f92672\">=<\/span><span style=\"color:#e6db74\">&#34;<\/span><span style=\"color:#66d9ef\">$(<\/span>uv run rocm-sdk path --root<span style=\"color:#66d9ef\">)<\/span><span style=\"color:#e6db74\">&#34;<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span>export HIP_PLATFORM<span style=\"color:#f92672\">=<\/span>amd\n<\/span><\/span><span style=\"display:flex;\"><span>export CMAKE_PREFIX_PATH<span style=\"color:#f92672\">=<\/span><span style=\"color:#e6db74\">&#34;<\/span><span style=\"color:#66d9ef\">$(<\/span>uv run rocm-sdk path --root<span style=\"color:#66d9ef\">)<\/span><span style=\"color:#e6db74\">:<\/span>$CMAKE_PREFIX_PATH<span style=\"color:#e6db74\">&#34;<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span>export GGML_CUDA_ENABLE_UNIFIED_MEMORY<span style=\"color:#f92672\">=<\/span>ON\n<\/span><\/span><span style=\"display:flex;\"><span>export LD_LIBRARY_PATH<span style=\"color:#f92672\">=<\/span><span style=\"color:#e6db74\">&#34;<\/span><span style=\"color:#66d9ef\">$(<\/span>uv run rocm-sdk path --root<span style=\"color:#66d9ef\">)<\/span><span style=\"color:#e6db74\">\/lib&#34;<\/span>\n<\/span><\/span><\/code><\/pre><\/div><p>Now, in your <code>llama.cpp<\/code> clone:<\/p>\n<div class=\"highlight\"><pre tabindex=\"0\" style=\"color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;\"><code class=\"language-bash\" data-lang=\"bash\"><span style=\"display:flex;\"><span>cmake -S . -B build <span style=\"color:#ae81ff\">\\\n<\/span><\/span><\/span><span style=\"display:flex;\"><span><span style=\"color:#ae81ff\"><\/span>        -DCMAKE_HIP_FLAGS<span style=\"color:#f92672\">=<\/span><span style=\"color:#e6db74\">&#34;-mllvm --amdgpu-unroll-threshold-local=600&#34;<\/span> <span style=\"color:#ae81ff\">\\\n<\/span><\/span><\/span><span style=\"display:flex;\"><span><span style=\"color:#ae81ff\"><\/span>        -DGGML_HIP<span style=\"color:#f92672\">=<\/span>ON <span style=\"color:#ae81ff\">\\\n<\/span><\/span><\/span><span style=\"display:flex;\"><span><span style=\"color:#ae81ff\"><\/span>        -DHIP_PLATFORM<span style=\"color:#f92672\">=<\/span>amd <span style=\"color:#ae81ff\">\\\n<\/span><\/span><\/span><span style=\"display:flex;\"><span><span style=\"color:#ae81ff\"><\/span>        -DGGML_HIPBLAS<span style=\"color:#f92672\">=<\/span>ON <span style=\"color:#ae81ff\">\\\n<\/span><\/span><\/span><span style=\"display:flex;\"><span><span style=\"color:#ae81ff\"><\/span>        -DGGML_HIP_ROCWMMA_FATTN<span style=\"color:#f92672\">=<\/span>OFF <span style=\"color:#ae81ff\">\\\n<\/span><\/span><\/span><span style=\"display:flex;\"><span><span style=\"color:#ae81ff\"><\/span>        -DGPU_TARGETS<span style=\"color:#f92672\">=<\/span>gfx1151 <span style=\"color:#ae81ff\">\\\n<\/span><\/span><\/span><span style=\"display:flex;\"><span><span style=\"color:#ae81ff\"><\/span>        -DCMAKE_BUILD_TYPE<span style=\"color:#f92672\">=<\/span>Release <span style=\"color:#ae81ff\">\\\n<\/span><\/span><\/span><span style=\"display:flex;\"><span><span style=\"color:#ae81ff\"><\/span>        -DLLAMA_OPENSSL<span style=\"color:#f92672\">=<\/span>ON <span style=\"color:#ae81ff\">\\\n<\/span><\/span><\/span><span style=\"display:flex;\"><span><span style=\"color:#ae81ff\"><\/span>        -DLLAMA_BUILD_EXAMPLES<span style=\"color:#f92672\">=<\/span>OFF <span style=\"color:#ae81ff\">\\\n<\/span><\/span><\/span><span style=\"display:flex;\"><span><span style=\"color:#ae81ff\"><\/span>        --fresh\n<\/span><\/span><\/code><\/pre><\/div><p>Then:<\/p>\n<div class=\"highlight\"><pre tabindex=\"0\" style=\"color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;\"><code class=\"language-bash\" data-lang=\"bash\"><span style=\"display:flex;\"><span>cmake --build build <span style=\"color:#ae81ff\">\\\n<\/span><\/span><\/span><span style=\"display:flex;\"><span><span style=\"color:#ae81ff\"><\/span>      --clean-first <span style=\"color:#ae81ff\">\\\n<\/span><\/span><\/span><span style=\"display:flex;\"><span><span style=\"color:#ae81ff\"><\/span>      --config Release -- -j<span style=\"color:#66d9ef\">$(<\/span>nproc<span style=\"color:#66d9ef\">)<\/span>\n<\/span><\/span><\/code><\/pre><\/div><p>Some of these build flags are critical for achieving good performance on Strix Halo.<\/p>\n<p>For example, while <code>GGML_HIP_ROCWMMA_FATTN=OFF<\/code> may seem counter-intuitive, it tells <code>llama.cpp<\/code> not to use the <code>rocWMMA<\/code> implementation for fast-attention,\nsince it performs worse on RDNA3.5 (the Strix Halo architecture). In fact, we get much better prompt processing performance with this flag set to <code>OFF<\/code> than when it&rsquo;s <code>ON<\/code>.<\/p>\n<p>This said, you are now ready to run some inference!<\/p>\n<h4 id=\"quick-validation-with-qwen36-35b-a3b\">Quick validation with Qwen3.6-35B-A3B<\/h4>\n<p>To run inference, you first need to download a quantized model (usually in GGUF format) and then pass it to the <code>llama-cli<\/code> (or <code>llama-server<\/code>) binary you just built.<\/p>\n<p>If you want to test a model which is a good mix of speed and quality, the <code>Qwen3.6-35B-A3B<\/code> at <code>Q6_K_L<\/code> quantization is a great choice.\nIt fits easily into the memory of a Strix Halo and offers excellent speed-to-quality ratio.<\/p>\n<p>You can download it (using <code>huggingface-cli<\/code> or using your browsers <a href=\"https:\/\/huggingface.co\/unsloth\/Qwen3.6-35B-A3B-GGUF\">here<\/a>) and run it:<\/p>\n<div class=\"highlight\"><pre tabindex=\"0\" style=\"color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;\"><code class=\"language-bash\" data-lang=\"bash\"><span style=\"display:flex;\"><span><span style=\"color:#75715e\"># Example command (assuming you have the model file)<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span>.\/build\/bin\/llama-cli --model path\/to\/qwen3.6-35b-a3b-q6_k.gguf -p <span style=\"color:#e6db74\">&#34;You are a helpful assistant.&#34;<\/span> --gpu-layers <span style=\"color:#ae81ff\">999<\/span>\n<\/span><\/span><\/code><\/pre><\/div><p>The <code>--gpu-layers 999<\/code> flag ensures all layers are offloaded to the GPU and that nothing is running in your CPU.<\/p>\n<p><img src=\"https:\/\/remy.io\/img\/articles\/llm-infra\/llama-cli.webp\" alt=\"llama.cpp running Qwen3.6 35B-A3B\"><\/p>\n<p><code>llama-cli<\/code> is only an ad-hoc tool to do inference with a model.\nNow, what I really want, is a server able to receive inference requests and send the results, in order\nto have my own tools able to run their inference workload.<\/p>\n<h3 id=\"llama-server\">llama-server<\/h3>\n<p>In order to serve different models, <code>llama.cpp<\/code> also implements a server exposing OpenAI compatible endpoints, named <code>llama-server<\/code>.\nIt is configurable through a <code>models.ini<\/code> file, where you list all the models you intend to expose, and <code>llama-server<\/code> make sure to\nload and unload models on demand.<\/p>\n<p>Here is a glance at my <code>models.ini<\/code> with Qwen 3.5 122B configured as an example:<\/p>\n<div class=\"highlight\"><pre tabindex=\"0\" style=\"color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;\"><code class=\"language-bash\" data-lang=\"bash\"><span style=\"display:flex;\"><span>version <span style=\"color:#f92672\">=<\/span> <span style=\"color:#ae81ff\">1<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span><span style=\"color:#f92672\">[<\/span>*<span style=\"color:#f92672\">]<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span><span style=\"color:#75715e\"># offload all layers to the GPU VRAM<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span>n-gpu-layers <span style=\"color:#f92672\">=<\/span> <span style=\"color:#ae81ff\">999<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span><span style=\"color:#75715e\"># Use all the threads for HTTP requests<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span>threads <span style=\"color:#f92672\">=<\/span> -1\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span><span style=\"color:#75715e\"># no mmap, direct-io for faster access<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span>mmap <span style=\"color:#f92672\">=<\/span> <span style=\"color:#ae81ff\">0<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span>direct-io <span style=\"color:#f92672\">=<\/span> <span style=\"color:#ae81ff\">1<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span>mlock <span style=\"color:#f92672\">=<\/span> <span style=\"color:#ae81ff\">1<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span>fit <span style=\"color:#f92672\">=<\/span> <span style=\"color:#ae81ff\">1<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span>parallel <span style=\"color:#f92672\">=<\/span> <span style=\"color:#ae81ff\">2<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span><span style=\"color:#75715e\"># flash attention<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span>flash-attn <span style=\"color:#f92672\">=<\/span> <span style=\"color:#ae81ff\">1<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span><span style=\"color:#75715e\"># prompt caching<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span>cache-prompt <span style=\"color:#f92672\">=<\/span> <span style=\"color:#ae81ff\">1<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span>cache-type-k <span style=\"color:#f92672\">=<\/span> q8_0\n<\/span><\/span><span style=\"display:flex;\"><span>cache-type-v <span style=\"color:#f92672\">=<\/span> q8_0\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span><span style=\"color:#f92672\">[<\/span>qwen3.5-122b-a10b-coder<span style=\"color:#f92672\">]<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span>model <span style=\"color:#f92672\">=<\/span> \/home\/remy\/opt\/models\/bartowski\/Qwen3.5-122B-A10B-Q4_K_M.gguf\n<\/span><\/span><span style=\"display:flex;\"><span>temp <span style=\"color:#f92672\">=<\/span> 0.6\n<\/span><\/span><span style=\"display:flex;\"><span>top-p <span style=\"color:#f92672\">=<\/span> 0.95\n<\/span><\/span><span style=\"display:flex;\"><span>top-k <span style=\"color:#f92672\">=<\/span> <span style=\"color:#ae81ff\">20<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span>min-p <span style=\"color:#f92672\">=<\/span> 0.00\n<\/span><\/span><span style=\"display:flex;\"><span>repeat-penalty <span style=\"color:#f92672\">=<\/span> 1.0\n<\/span><\/span><span style=\"display:flex;\"><span>presence-penalty <span style=\"color:#f92672\">=<\/span> 0.00\n<\/span><\/span><span style=\"display:flex;\"><span>swa-full <span style=\"color:#f92672\">=<\/span> true\n<\/span><\/span><span style=\"display:flex;\"><span>ctx-size <span style=\"color:#f92672\">=<\/span> <span style=\"color:#ae81ff\">262144<\/span>\n<\/span><\/span><\/code><\/pre><\/div><p>In the same manner, the <code>llama-server<\/code> can expose a neat web interface to run\ninteractive conversations (with MCPs support):<\/p>\n<p><img src=\"https:\/\/remy.io\/img\/articles\/llm-infra\/llama-webui.webp\" alt=\"llama.cpp webui with Qwen3.6 35B-A3B\"><\/p>\n<p>And finally, here is how I start it in a dedicated systemd service:<\/p>\n<div class=\"highlight\"><pre tabindex=\"0\" style=\"color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;\"><code class=\"language-bash\" data-lang=\"bash\"><span style=\"display:flex;\"><span>llama-server <span style=\"color:#ae81ff\">\\\n<\/span><\/span><\/span><span style=\"display:flex;\"><span><span style=\"color:#ae81ff\"><\/span>        --models-preset \/home\/remy\/opt\/models\/models.ini <span style=\"color:#ae81ff\">\\\n<\/span><\/span><\/span><span style=\"display:flex;\"><span><span style=\"color:#ae81ff\"><\/span>        --models-max <span style=\"color:#ae81ff\">2<\/span> <span style=\"color:#ae81ff\">\\\n<\/span><\/span><\/span><span style=\"display:flex;\"><span><span style=\"color:#ae81ff\"><\/span>        --webui-mcp-proxy <span style=\"color:#ae81ff\">\\\n<\/span><\/span><\/span><span style=\"display:flex;\"><span><span style=\"color:#ae81ff\"><\/span>        --metrics <span style=\"color:#ae81ff\">\\\n<\/span><\/span><\/span><span style=\"display:flex;\"><span><span style=\"color:#ae81ff\"><\/span>        --host 0.0.0.0 <span style=\"color:#ae81ff\">\\\n<\/span><\/span><\/span><span style=\"display:flex;\"><span><span style=\"color:#ae81ff\"><\/span>        --port <span style=\"color:#ae81ff\">8080<\/span>\n<\/span><\/span><\/code><\/pre><\/div><p>All of this setup through Ansible playbooks makes it easily maintanable across the servers.<\/p>\n<h1 id=\"conclusion\">Conclusion<\/h1>\n<p>With this setup, I have successfully moved some of my LLM inference workload from the cloud to my own basement.\nI am genuinely pleased with the results; the performance is surprisingly capable, and the learning curve has been\na rewarding detour.<\/p>\n<p>I can use the web interface to query random questions to my LLMs. With a MCP server I developed they are\ncapable of fetching the net or searching the web to help with better understanding something, or to run Python 3\ncode in order to draft some random prototypes.<\/p>\n<p>I can use my custom agent to do agentic development locally, reviews, and more.\nI developed a router\/load-balancer to load-balance the coding tasks between my two servers, using\nsession affinity and load usage to make sure both were used appropriately.<\/p>\n<p>What&rsquo;s next?<\/p>\n<ul>\n<li>I want to add <strong>observability<\/strong> on token throughput, VRAM pressure, the load-balancing efficiency and more.<\/li>\n<li>The router I developed load-balances the work \/ routing appropriately, but I have more plans: I work on an <strong>orchestration<\/strong> layer, one of the Strix Halo does the developement while the other with a faster LLM would study the diff for eventual improvements on-the-fly. And more.<\/li>\n<li>I monitor a couple PRs in <code>llama.cpp<\/code> that have the potential to greatly accelerate <a href=\"https:\/\/github.com\/ggml-org\/llama.cpp\/pull\/21344\">the prefill<\/a>, and the <a href=\"https:\/\/github.com\/ggml-org\/llama.cpp\/pull\/22673\">token generation<\/a>!<\/li>\n<\/ul>\n<p>If you&rsquo;re interested in knowing more, stay tuned. I might write an article or two on developing the coding agent,\nthe custom MCP server and on the orchestration layer.<\/p>\n"},{"title":"Migrating to Zig 0.16.0 With a Local LLM (Qwen3.6-27B)","link":"https:\/\/remy.io\/blog\/migrating-to-zig-0-16-0-with-a-local-llm\/","pubDate":"Sat, 25 Apr 2026 21:20:44 +0100","author":"contact@remy.io (R\u00e9my Mathieu)","guid":"https:\/\/remy.io\/blog\/migrating-to-zig-0-16-0-with-a-local-llm\/","description":"<h1 id=\"introduction\">Introduction<\/h1>\n<p>Using my local LLM inference setup (blog post coming soon), I wanted to take a shot at a\nsomewhat large and complex refactor using <a href=\"https:\/\/huggingface.co\/Qwen\/Qwen3.6-27B\">Qwen3.6-27B<\/a>.<\/p>\n<p>For years I&rsquo;ve been using <a href=\"https:\/\/github.com\/remeh\/meh\">my own code editor<\/a> developed in Zig that I compile with Zig <code>0.15.x<\/code>,\nand recently, the Zig team released a new stable version, <a href=\"https:\/\/ziglang.org\/download\/0.16.0\/release-notes.html\">Zig 0.16.0<\/a>.<\/p>\n<p>Zig is still in development, so new versions imply breaking changes\nand having to adapt your codebase. I thought that it would be a perfect project for my\nlocal LLMs:<\/p>\n<ul>\n<li>while being complex, it is not extremely hard,<\/li>\n<li>the Zig release notes include most of what&rsquo;s necessary to migrate,<\/li>\n<li>I can have both compilers at the same time on the machine, letting the LLM validate its work.<\/li>\n<\/ul>\n<p><img src=\"https:\/\/remy.io\/img\/articles\/zig-llm\/screenshot-20260425.png\" alt=\"Screenshot of my code editor meh\"><\/p>\n<h2 id=\"prompt\">Prompt<\/h2>\n<p>Here is the prompt I gave to <code>Qwen3.6-27B<\/code>:<\/p>\n<div class=\"highlight\"><pre tabindex=\"0\" style=\"color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;\"><code class=\"language-markdown\" data-lang=\"markdown\"><span style=\"display:flex;\"><span>You are an incredibly smart Zig developer. We are in a text code editor codebase, compiling on Zig 0.15.2\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span>However, Zig 0.16.0 is out. The compiler is available at: \/home\/remy\/opt\/zig-x86_64-linux-0.16.0\/zig\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span>I want you to port current codebase to Zig 0.16.0. To build with it, use this command:\n<\/span><\/span><span style=\"display:flex;\"><span><span style=\"color:#e6db74\">`\/home\/remy\/opt\/zig-x86_64-linux-0.16.0\/zig build`<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span>Consider reading 0.16.0 release notes in https:\/\/ziglang.org\/download\/0.16.0\/release-notes.html\n<\/span><\/span><span style=\"display:flex;\"><span>it contains information on how to port to 0.16.0\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span>If you need to rebuild with Zig 0.15.2 for some reasons, but you shouldn&#39;t have to, just run: <span style=\"color:#e6db74\">`zig build`<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span>Port the codebase now.\n<\/span><\/span><\/code><\/pre><\/div><p>I kicked it off in the early afternoon and went for a small hike with the kids,\nhoping the harness would not do anything crazy while I was away.<\/p>\n<h2 id=\"results\">Results<\/h2>\n<p>When I came back, I was welcomed with this message:<\/p>\n<div class=\"highlight\"><pre tabindex=\"0\" style=\"color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;\"><code class=\"language-markdown\" data-lang=\"markdown\"><span style=\"display:flex;\"><span> The port to Zig 0.16.0 is complete. Here&#39;s a summary of the key changes made:\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span> Major API Changes\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span> ### std.Io threading\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#66d9ef\">-<\/span> main.zig: Migrated to &#34;Juicy Main&#34; pattern - pub fn main(init: std.process.Init) !void\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#66d9ef\">-<\/span> io parameter threaded through App, Buffer, LSP, WidgetLookup, Editor.save\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#66d9ef\">-<\/span> std.testing.io used in all tests\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span> ### File I\/O (std.fs \u2192 std.Io)\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#66d9ef\">-<\/span> std.fs.realpathAlloc \u2192 std.Io.Dir.cwd().realPathFileAlloc(io, path, allocator)\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#66d9ef\">-<\/span> std.fs.cwd().openFile \u2192 std.Io.Dir.cwd().openFile(io, path, options)\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#66d9ef\">-<\/span> std.fs.cwd().createFile \u2192 std.Io.Dir.cwd().createFile(io, path, options)\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#66d9ef\">-<\/span> file.read(buff) \u2192 file.readStreaming(io, <span style=\"color:#960050;background-color:#1e0010\">&amp;<\/span>[_][]u8{slice[0..]}) (scatter-gather)\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#66d9ef\">-<\/span> file.write(data) \u2192 file.writeStreamingAll(io, data)\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#66d9ef\">-<\/span> file.close() \u2192 file.close(io)\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#66d9ef\">-<\/span> file.sync() \u2192 file.sync(io)\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#66d9ef\">-<\/span> std.fs.deleteFileAbsolute \u2192 std.Io.Dir.deleteFileAbsolute(io, path)\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span> ### Process (std.process.Child \u2192 std.process.spawn)\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#66d9ef\">-<\/span> Child.init + spawn \u2192 std.process.spawn(io, SpawnOptions{...})\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#66d9ef\">-<\/span> stdin_behavior \u2192 stdin: .pipe\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#66d9ef\">-<\/span> max_output_bytes \u2192 stdout_limit \/ stderr_limit\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#66d9ef\">-<\/span> cwd \u2192 cwd: .{ .path = path }\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span> ### Time <span style=\"color:#960050;background-color:#1e0010\">&amp;<\/span> Sleep\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#66d9ef\">-<\/span> std.time.milliTimestamp() \u2192 @divTrunc(std.Io.Timestamp.now(io, .boot).nanoseconds, std.time.ns_per_ms)\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#66d9ef\">-<\/span> std.Thread.sleep(ns) \u2192 io.sleep(.fromMilliseconds(ms), .awake)\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#66d9ef\">-<\/span> std.posix.nanosleep \u2192 io.sleep\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span> ### Synchronization\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#66d9ef\">-<\/span> std.Thread.Mutex \u2192 std.Io.Mutex with lock(io) catch {}\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span> ### Other\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#66d9ef\">-<\/span> std.io.Writer.Allocating \u2192 std.Io.Writer.Allocating\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#66d9ef\">-<\/span> Dir.Iterator.next() \u2192 next(io)\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#66d9ef\">-<\/span> Dir.close() \u2192 close(io)\n<\/span><\/span><span style=\"display:flex;\"><span> <span style=\"color:#66d9ef\">-<\/span> Child.kill() \u2192 kill(io) (returns void, no error)\n<\/span><\/span><\/code><\/pre><\/div><p>That&rsquo;s it. I checked the changes: they compile and they work, with the exception of the parts spawning an external\nprocess (e.g. ripgrep): it failed to adapt this part to the new <code>std.Io<\/code> implementation.<\/p>\n<p>But it got me 99% there&hellip; I just had to send an additional 4-line commit to finish its work.<\/p>\n<p><a href=\"https:\/\/github.com\/remeh\/meh\/commit\/8f46b05443b84100695499778fdda09832119466\">Here is the commit containing all its changes<\/a>\nand the <a href=\"https:\/\/github.com\/remeh\/meh\/commit\/c617deb7b3f1b532196e8bccd5374593032edc4b\">simple additional commit<\/a> I did\nmanually to complete the migration to Zig <code>0.16.0<\/code>.<\/p>\n<h1 id=\"conclusion\">Conclusion<\/h1>\n<p>It ran for 2h33 on my AMD Strix Halo (AMD APU with 128GB of unified memory).<\/p>\n<p>Is it fast? Not by any means.\nIs the generated code working, useful, and produced entirely unattended on local hardware? Yes.<\/p>\n<p>I think that several things helped it be successful here:<\/p>\n<ul>\n<li>having access to the &ldquo;migration&rdquo; guide in the release notes must have been great context for the LLM,<\/li>\n<li>being able to compile it itself, and check the output\/errors,<\/li>\n<li>all the unit tests in the codebase, that it ran several times.<\/li>\n<\/ul>\n<p>While I won&rsquo;t do the exercise of comparing &ldquo;Qwen3.6-27B running on a Strix Halo&rdquo; and a paid Claude Code subscription,\nI thought this was a piece of practical, real-world feedback worth sharing.<\/p>\n<p>Stay tuned for a future blog post on the architecture of my setup for local LLM inference.<\/p>\n"},{"title":"mok","link":"https:\/\/remy.io\/projects\/mok\/","pubDate":"Thu, 01 Jan 2026 19:20:44 +0100","author":"contact@remy.io (R\u00e9my Mathieu)","guid":"https:\/\/remy.io\/projects\/mok\/","description":{}},{"title":"Scroll Story","link":"https:\/\/remy.io\/projects\/scroll-story\/","pubDate":"Mon, 01 Sep 2025 19:20:44 +0100","author":"contact@remy.io (R\u00e9my Mathieu)","guid":"https:\/\/remy.io\/projects\/scroll-story\/","description":{}},{"title":"meh","link":"https:\/\/remy.io\/projects\/meh\/","pubDate":"Sat, 01 Jan 2022 19:20:44 +0100","author":"contact@remy.io (R\u00e9my Mathieu)","guid":"https:\/\/remy.io\/projects\/meh\/","description":{}},{"title":"Diago","link":"https:\/\/remy.io\/projects\/diago\/","pubDate":"Sun, 15 Mar 2020 19:20:44 +0100","author":"contact@remy.io (R\u00e9my Mathieu)","guid":"https:\/\/remy.io\/projects\/diago\/","description":{}},{"title":"How to use Diago to diagnose CPU and memory usage in Go programs","link":"https:\/\/remy.io\/blog\/how-to-use-diago-to-diagnose-cpu-and-memory-usage-in-go-programs\/","pubDate":"Sun, 15 Mar 2020 08:10:44 +0100","author":"contact@remy.io (R\u00e9my Mathieu)","guid":"https:\/\/remy.io\/blog\/how-to-use-diago-to-diagnose-cpu-and-memory-usage-in-go-programs\/","description":"<p><a href=\"https:\/\/github.com\/remeh\/diago\">Diago<\/a> is a visualization tool for profiles and heap snapshots generated with <code>pprof<\/code>.<\/p>\n<p>It is a stand-alone application displaying a call tree of the CPU usage per functions\n(or per line of codes in functions) but also capable of displaying heap snapshots.<\/p>\n<p>We&rsquo;ll see in this short article how to use Diago to get more insight in your Go programs and to help you debug them.<\/p>\n<p><img src=\"https:\/\/remy.io\/img\/articles\/diago\/profile.png\" alt=\"Diago displaying the CPU usage of each function in a call tree\"><\/p>\n<h2 id=\"run-pprof-in-the-go-program\">Run pprof in the Go program<\/h2>\n<p>If you know how to use <code>pprof<\/code> with Go binaries, this chapter won&rsquo;t add\nmuch to what you already know. If you don&rsquo;t, you will see that it is quite\nstraight-forward.<\/p>\n<p>There are a few ways of starting profiling using <code>pprof<\/code>, we will focus on the\neasiest one: using its HTTP handler.<\/p>\n<p>The Go standard library ships an HTTP handler exposing profiling samples, heap snashots,\ngoroutines information, traces and much more. In order to enable it, you have\nto import <code>net\/http\/pprof<\/code> if you&rsquo;re already using a HTTP server in your program:<\/p>\n<div class=\"highlight\"><pre tabindex=\"0\" style=\"color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;\"><code class=\"language-go\" data-lang=\"go\"><span style=\"display:flex;\"><span><span style=\"color:#f92672\">import<\/span> <span style=\"color:#a6e22e\">_<\/span> <span style=\"color:#e6db74\">&#34;net\/http\/pprof&#34;<\/span>\n<\/span><\/span><\/code><\/pre><\/div><p>If you are not already serving HTTP in your program, the easiest way to get started is to import\nthe packages as mentioned above and to add this snippet to your program:<\/p>\n<div class=\"highlight\"><pre tabindex=\"0\" style=\"color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;\"><code class=\"language-go\" data-lang=\"go\"><span style=\"display:flex;\"><span><span style=\"color:#66d9ef\">go<\/span> <span style=\"color:#66d9ef\">func<\/span>() {\n<\/span><\/span><span style=\"display:flex;\"><span>    <span style=\"color:#a6e22e\">log<\/span>.<span style=\"color:#a6e22e\">Println<\/span>(<span style=\"color:#a6e22e\">http<\/span>.<span style=\"color:#a6e22e\">ListenAndServe<\/span>(<span style=\"color:#e6db74\">&#34;localhost:6060&#34;<\/span>, <span style=\"color:#66d9ef\">nil<\/span>))\n<\/span><\/span><span style=\"display:flex;\"><span>}()\n<\/span><\/span><\/code><\/pre><\/div><p>With this done, <code>pprof<\/code> is exposing all the information on <code>http:\/\/localhost:6060\/debug\/pprof<\/code>.<\/p>\n<h2 id=\"get-profiles-and-heap-snapshots\">Get profiles and heap snapshots<\/h2>\n<p>Now that pprof is exposing its data through HTTP, it&rsquo;s quite easy to retrieve\nprofiles and heap snashots.<\/p>\n<h3 id=\"profile\">Profile<\/h3>\n<p>A profile will contain information on the amount of time spent by the CPU in each\nfunction. It is a proportional value: pprof won&rsquo;t observe exactly how long each\nfunction is running on the CPU, instead, pprof is frequently capturing samples\nof the CPU usage and they are later aggregated by the visualization tools.<\/p>\n<p>Because the samples are frequently captured by pprof, you can decide how long you\nwant pprof to gather them. Let&rsquo;s say we want to let pprof run for 30s, we will use\n<code>curl<\/code> to download a 30s profile:<\/p>\n<div class=\"highlight\"><pre tabindex=\"0\" style=\"color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;\"><code class=\"language-bash\" data-lang=\"bash\"><span style=\"display:flex;\"><span>curl <span style=\"color:#e6db74\">&#34;http:\/\/localhost:6060\/debug\/pprof\/profile?seconds=30&#34;<\/span> &gt; profile\n<\/span><\/span><\/code><\/pre><\/div><h3 id=\"heap-snapshots\">Heap snapshots<\/h3>\n<p>Pprof lets you capture a snapshot of the heap as well. Here&rsquo;s the command to capture it:<\/p>\n<div class=\"highlight\"><pre tabindex=\"0\" style=\"color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;\"><code class=\"language-bash\" data-lang=\"bash\"><span style=\"display:flex;\"><span>curl <span style=\"color:#e6db74\">&#34;http:\/\/localhost:6060\/debug\/pprof\/heap&#34;<\/span> &gt; heap\n<\/span><\/span><\/code><\/pre><\/div><p>We will see in the visualization chapter that there is two kind of information in this file:\nthe total amount of memory allocated per functions since the start of the program\nand the amount of memory used by each function at the moment of capture.<\/p>\n<h2 id=\"visualize-the-profile-and-the-heap-snapshot-with-diago\">Visualize the profile and the heap snapshot with Diago<\/h2>\n<p>First, if you want to compile and install Diago, use the command:<\/p>\n<div class=\"highlight\"><pre tabindex=\"0\" style=\"color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;\"><code class=\"language-bash\" data-lang=\"bash\"><span style=\"display:flex;\"><span>go get -u github.com\/remeh\/diago\n<\/span><\/span><\/code><\/pre><\/div><p>In order to visualize the profile, start Diago and use the <code>-file<\/code> flag to provide\nthe filepath of the file to open:<\/p>\n<div class=\"highlight\"><pre tabindex=\"0\" style=\"color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;\"><code class=\"language-bash\" data-lang=\"bash\"><span style=\"display:flex;\"><span>.\/diago -file profile\n<\/span><\/span><\/code><\/pre><\/div><p>and here you go, you should have a visualization such as this one:<\/p>\n<p><img src=\"https:\/\/remy.io\/img\/articles\/diago\/profile.png\" alt=\"Diago displaying the CPU usage of each function in a call tree\"><\/p>\n<p>You can do the same to open the heap snapshot:<\/p>\n<div class=\"highlight\"><pre tabindex=\"0\" style=\"color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;\"><code class=\"language-bash\" data-lang=\"bash\"><span style=\"display:flex;\"><span>.\/diago -file heap\n<\/span><\/span><\/code><\/pre><\/div><p><img src=\"https:\/\/remy.io\/img\/articles\/diago\/heap-snapshot.png\" alt=\"Diago displaying the memory allocated of each function in a call tree\"><\/p>\n<p>In the screenshot above, you can observe that we&rsquo;re looking at the total memory\nallocated by each function of the program since it has started (the observed heap\nin this screenshot is actually a heap snapshot of Diago).<\/p>\n<p>On the top right, you can see a switch to select the other mode displaying the memory\nusage at the moment of the capture.<\/p>\n<h2 id=\"conclusion\">Conclusion<\/h2>\n<p>There is other tools helping you visualize these files in flamegraph or in a dot\ndiagram, you can even use the pprof command-line tool to display the data by top\nvalues and more.<\/p>\n<p>I&rsquo;ve developed this tool because I sometimes want to visualize this kind of information in\na tree and I hope that it could be useful to you as well.<\/p>\n<p>Don&rsquo;t hesitate to ping me on <a href=\"https:\/\/twitter.com\/remeh\">Twitter<\/a> or to check the\n<a href=\"https:\/\/github.com\/remeh\/diago\">source code available here<\/a>.<\/p>\n"},{"title":"Bloom effect in Go","link":"https:\/\/remy.io\/blog\/bloom-effect-in-go\/","pubDate":"Wed, 12 Feb 2020 19:20:44 +0100","author":"contact@remy.io (R\u00e9my Mathieu)","guid":"https:\/\/remy.io\/blog\/bloom-effect-in-go\/","description":"<h1 id=\"introduction\">Introduction<\/h1>\n<p>While working on <a href=\"https:\/\/strmaker.com\">one of my side-project<\/a>, I&rsquo;ve had the need\nto render a kind of neon or glow effect on images (think the effect used everywhere in Tron).\nSomething very close to the bloom effect used in video-games.\nI&rsquo;ve needed to do this using software rendering (computed on the CPU), because it\nwill run on a server without any GPU (note that it must not be confused with the <a href=\"https:\/\/wikipedia.org\/wiki\/Bloom_filter\">bloom filter data structure<\/a>).<\/p>\n<p>In reality, the bloom effect is produced by the lens of cameras which can never\nperfectly focus, and, to cite the <a href=\"https:\/\/wikipedia.org\/wiki\/Bloom_(shader_effect)\">Wikipedia article<\/a> :<\/p>\n<div class=\"highlight\"><pre tabindex=\"0\" style=\"color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;\"><code class=\"language-markdown\" data-lang=\"markdown\"><span style=\"display:flex;\"><span>    Under normal circumstances, these imperfections are not noticeable, but an intensely\n<\/span><\/span><span style=\"display:flex;\"><span>      bright light source will cause the imperfections [the bloom] to become visible.\n<\/span><\/span><\/code><\/pre><\/div><h1 id=\"how-to-replicate-the-effect\">How to replicate the effect<\/h1>\n<p>This image is composed of three rendered elements: a gray background, a red rectangle and a red circle.<\/p>\n<p><img src=\"https:\/\/remy.io\/img\/articles\/bloom-effect-in-go\/bloom_red_1.png\" alt=\"Basic rendering with no bloom effect\"><\/p>\n<p>Everything looks flat and a bit boring: for a nice effect, we will want the red\nrectangle and the red circle to have some bloom effect. We want:<\/p>\n<p><img src=\"https:\/\/remy.io\/img\/articles\/bloom-effect-in-go\/bloom_red_4.png\" alt=\"Rendering with bloom effect\"><\/p>\n<p>In a rendering pipeline, you control what you draw and in which order, it will\nthen be quite easy to apply an effect only on some parts of what you&rsquo;re rendering\nand in this case, on the red lines.<\/p>\n<p>The rendering process will be:<\/p>\n<ul>\n<li>Create a first buffer representing the empty image<\/li>\n<li>Draw the gray background on this buffer (it doesn&rsquo;t need any processing)<\/li>\n<li>Draw the red rectangle and red circle in a second buffer<\/li>\n<li>Apply the bloom effect on this second buffer<\/li>\n<li>Draw this second buffer on top of the first one<\/li>\n<\/ul>\n<p>After this, the first buffer should be containing neat red lines with bloom effect\non top of a gray background. But in this rendering process, what does <code>Apply the bloom effect<\/code> mean?<\/p>\n<h1 id=\"bloom-effect-using-go\">Bloom effect using Go<\/h1>\n<h2 id=\"graphic-libraries\">Graphic libraries<\/h2>\n<p>For the rendering, I&rsquo;ve used the excellent <a href=\"https:\/\/github.com\/fogleman\/gg\">gg<\/a> library.\nDrawing lines, rectangles and texts is straight-forward with it.<\/p>\n<p>For the graphic effects, I&rsquo;ve selected <a href=\"https:\/\/github.com\/anthonynsimon\/bild\">bild<\/a>\nwhich is really easy to use and which I also recommend.<\/p>\n<h2 id=\"implementation\">Implementation<\/h2>\n<p>Basically, having a camera out of focus means that the captured image is blurry,\nand we previously said that this is noticeable with bright sources of light.<\/p>\n<p>Then, what we will want to do is to blur the red lines, because here, it is our\nsource of light. After having blurred their lines, we draw them on top of the gray\nbackground. Let&rsquo;s check out the result using the Gaussian blur shipped in the bild library:<\/p>\n<p><img src=\"https:\/\/remy.io\/img\/articles\/bloom-effect-in-go\/bloom_red_2.png\" alt=\"Basic rendering with blurred red lines\"><\/p>\n<p>We immediatly see that it is not really the final effect that we want: the source\nof light has kind of disappeared, becoming blurry but way darker.<\/p>\n<p>We&rsquo;ve lost our source of light? OK, let&rsquo;s draw it again on top of the blurred one:<\/p>\n<p><img src=\"https:\/\/remy.io\/img\/articles\/bloom-effect-in-go\/bloom_red_3.png\" alt=\"Basic rendering blurred red lines and original red lines on top of it\"><\/p>\n<p>We are getting there, however, in this image I think that the effect is&hellip; too light. <em>pun<\/em><\/p>\n<p>Because we are blurring the red lines, they get transparent around their borders.\nDrawing the original lines on top helps but how could we increase the amount of\nlight in the blurred one, without having this much transparency?\nApplying the blur a second time (or increasing its radius) won&rsquo;t help, it will\ninstead decrease the amount of light by making it sparse.<\/p>\n<p>The solution is to simply use a bigger source of light <em>before<\/em> applying the blur.\nIn order to have this improved source of light from the rectangle and the circle:\nwe want to dilate them first.  Their lines will be bigger and we will then apply\na blur on a larger source of light.  Hopefully, the bild library is shipping a dilate effect.<\/p>\n<p>Rendering with different colors:<\/p>\n<p><img src=\"https:\/\/remy.io\/img\/articles\/bloom-effect-in-go\/bloom_red_4.png\" alt=\"Rendering red with bloom effect\">\n<img src=\"https:\/\/remy.io\/img\/articles\/bloom-effect-in-go\/bloom_blue_4.png\" alt=\"Rendering blue with bloom effect\">\n<img src=\"https:\/\/remy.io\/img\/articles\/bloom-effect-in-go\/bloom_green_4.png\" alt=\"Rendering green with bloom effect\"><\/p>\n<p>At this point, I think it is what we were looking for!<\/p>\n<h1 id=\"conclusion\">Conclusion<\/h1>\n<p>We&rsquo;ve seen that to imitate the glow or bloom effect on an image by:<\/p>\n<ul>\n<li>Dilating the source of light to make it bigger<\/li>\n<li>Blurring it to make it look out of focus<\/li>\n<li>On top of this blurred light, draw the original one (which is in focus)<\/li>\n<\/ul>\n<p>Please note that I&rsquo;m not an expert in image processing, especially not in colors and\nlights theory. I hope that this article could be helpful to someone else.<\/p>\n<h1 id=\"code\">Code<\/h1>\n<p>Here&rsquo;s the full code for this article:<\/p>\n<div class=\"highlight\"><pre tabindex=\"0\" style=\"color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;\"><code class=\"language-go\" data-lang=\"go\"><span style=\"display:flex;\"><span><span style=\"color:#f92672\">package<\/span> <span style=\"color:#a6e22e\">main<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span><span style=\"color:#f92672\">import<\/span> (\n<\/span><\/span><span style=\"display:flex;\"><span>\t<span style=\"color:#e6db74\">&#34;image&#34;<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span>\t<span style=\"color:#e6db74\">&#34;github.com\/anthonynsimon\/bild\/blur&#34;<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span>\t<span style=\"color:#e6db74\">&#34;github.com\/anthonynsimon\/bild\/effect&#34;<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span>\t<span style=\"color:#e6db74\">&#34;github.com\/fogleman\/gg&#34;<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span>)\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span><span style=\"color:#66d9ef\">func<\/span> <span style=\"color:#a6e22e\">main<\/span>() {\n<\/span><\/span><span style=\"display:flex;\"><span>\t<span style=\"color:#75715e\">\/\/ draw the source of light\n<\/span><\/span><\/span><span style=\"display:flex;\"><span><span style=\"color:#75715e\"><\/span>\t<span style=\"color:#75715e\">\/\/ it&#39;s the buffer on which we will apply the bloom effect\n<\/span><\/span><\/span><span style=\"display:flex;\"><span><span style=\"color:#75715e\"><\/span>\t<span style=\"color:#a6e22e\">dc<\/span> <span style=\"color:#f92672\">:=<\/span> <span style=\"color:#a6e22e\">gg<\/span>.<span style=\"color:#a6e22e\">NewContext<\/span>(<span style=\"color:#ae81ff\">200<\/span>, <span style=\"color:#ae81ff\">200<\/span>)\n<\/span><\/span><span style=\"display:flex;\"><span>\t<span style=\"color:#a6e22e\">dc<\/span>.<span style=\"color:#a6e22e\">SetLineWidth<\/span>(<span style=\"color:#ae81ff\">3.0<\/span>)\n<\/span><\/span><span style=\"display:flex;\"><span>\t<span style=\"color:#a6e22e\">dc<\/span>.<span style=\"color:#a6e22e\">SetRGB255<\/span>(<span style=\"color:#ae81ff\">147<\/span>, <span style=\"color:#ae81ff\">112<\/span>, <span style=\"color:#ae81ff\">219<\/span>)\n<\/span><\/span><span style=\"display:flex;\"><span>\t<span style=\"color:#a6e22e\">dc<\/span>.<span style=\"color:#a6e22e\">DrawRectangle<\/span>(<span style=\"color:#ae81ff\">10<\/span>, <span style=\"color:#ae81ff\">10<\/span>, <span style=\"color:#ae81ff\">180<\/span>, <span style=\"color:#ae81ff\">180<\/span>)\n<\/span><\/span><span style=\"display:flex;\"><span>\t<span style=\"color:#a6e22e\">dc<\/span>.<span style=\"color:#a6e22e\">DrawCircle<\/span>(<span style=\"color:#ae81ff\">100<\/span>, <span style=\"color:#ae81ff\">100<\/span>, <span style=\"color:#ae81ff\">50<\/span>)\n<\/span><\/span><span style=\"display:flex;\"><span>\t<span style=\"color:#a6e22e\">dc<\/span>.<span style=\"color:#a6e22e\">Stroke<\/span>()\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span>\t<span style=\"color:#75715e\">\/\/ store the original source of light to draw it back later\n<\/span><\/span><\/span><span style=\"display:flex;\"><span><span style=\"color:#75715e\"><\/span>\t<span style=\"color:#a6e22e\">original<\/span> <span style=\"color:#f92672\">:=<\/span> <span style=\"color:#a6e22e\">dc<\/span>.<span style=\"color:#a6e22e\">Image<\/span>()\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span>\t<span style=\"color:#75715e\">\/\/ bloom this source of light\n<\/span><\/span><\/span><span style=\"display:flex;\"><span><span style=\"color:#75715e\"><\/span>\t<span style=\"color:#a6e22e\">bloomed<\/span> <span style=\"color:#f92672\">:=<\/span> <span style=\"color:#a6e22e\">Bloom<\/span>(<span style=\"color:#a6e22e\">dc<\/span>.<span style=\"color:#a6e22e\">Image<\/span>())\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span>\t<span style=\"color:#75715e\">\/\/ now, let&#39;s do our final rendering, let&#39;s starts by rendering\n<\/span><\/span><\/span><span style=\"display:flex;\"><span><span style=\"color:#75715e\"><\/span>\t<span style=\"color:#75715e\">\/\/ a gray rectangle in a new buffer\n<\/span><\/span><\/span><span style=\"display:flex;\"><span><span style=\"color:#75715e\"><\/span>\t<span style=\"color:#a6e22e\">dc<\/span> = <span style=\"color:#a6e22e\">gg<\/span>.<span style=\"color:#a6e22e\">NewContext<\/span>(<span style=\"color:#ae81ff\">220<\/span>, <span style=\"color:#ae81ff\">220<\/span>)\n<\/span><\/span><span style=\"display:flex;\"><span>\t<span style=\"color:#a6e22e\">dc<\/span>.<span style=\"color:#a6e22e\">SetRGB255<\/span>(<span style=\"color:#ae81ff\">40<\/span>, <span style=\"color:#ae81ff\">40<\/span>, <span style=\"color:#ae81ff\">40<\/span>)\n<\/span><\/span><span style=\"display:flex;\"><span>\t<span style=\"color:#a6e22e\">dc<\/span>.<span style=\"color:#a6e22e\">DrawRectangle<\/span>(<span style=\"color:#ae81ff\">0<\/span>, <span style=\"color:#ae81ff\">0<\/span>, <span style=\"color:#ae81ff\">220<\/span>, <span style=\"color:#ae81ff\">220<\/span>)\n<\/span><\/span><span style=\"display:flex;\"><span>\t<span style=\"color:#a6e22e\">dc<\/span>.<span style=\"color:#a6e22e\">Fill<\/span>()\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span>\t<span style=\"color:#75715e\">\/\/ draw our bloomed light\n<\/span><\/span><\/span><span style=\"display:flex;\"><span><span style=\"color:#75715e\"><\/span>\t<span style=\"color:#a6e22e\">dc<\/span>.<span style=\"color:#a6e22e\">DrawImage<\/span>(<span style=\"color:#a6e22e\">bloomed<\/span>, <span style=\"color:#ae81ff\">0<\/span>, <span style=\"color:#ae81ff\">0<\/span>)\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span>\t<span style=\"color:#75715e\">\/\/ re-apply the original source of light\n<\/span><\/span><\/span><span style=\"display:flex;\"><span><span style=\"color:#75715e\"><\/span>\t<span style=\"color:#a6e22e\">dc<\/span>.<span style=\"color:#a6e22e\">DrawImage<\/span>(<span style=\"color:#a6e22e\">original<\/span>, <span style=\"color:#ae81ff\">10<\/span>, <span style=\"color:#ae81ff\">10<\/span>)\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span>\t<span style=\"color:#75715e\">\/\/ save the result in a PNG\n<\/span><\/span><\/span><span style=\"display:flex;\"><span><span style=\"color:#75715e\"><\/span>\t<span style=\"color:#a6e22e\">dc<\/span>.<span style=\"color:#a6e22e\">SavePNG<\/span>(<span style=\"color:#e6db74\">&#34;output.png&#34;<\/span>)\n<\/span><\/span><span style=\"display:flex;\"><span>}\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span><span style=\"color:#75715e\">\/\/ Bloom applies a bloom effect on the given image.\n<\/span><\/span><\/span><span style=\"display:flex;\"><span><span style=\"color:#75715e\">\/\/ Because of the nature of the effect, a larger image is returned.\n<\/span><\/span><\/span><span style=\"display:flex;\"><span><span style=\"color:#75715e\">\/\/ 10px padding is added to each side of the image, growing it by\n<\/span><\/span><\/span><span style=\"display:flex;\"><span><span style=\"color:#75715e\">\/\/ 20px on X and 20px on Y.\n<\/span><\/span><\/span><span style=\"display:flex;\"><span><span style=\"color:#75715e\"><\/span><span style=\"color:#66d9ef\">func<\/span> <span style=\"color:#a6e22e\">Bloom<\/span>(<span style=\"color:#a6e22e\">img<\/span> <span style=\"color:#a6e22e\">image<\/span>.<span style=\"color:#a6e22e\">Image<\/span>) <span style=\"color:#a6e22e\">image<\/span>.<span style=\"color:#a6e22e\">Image<\/span> {\n<\/span><\/span><span style=\"display:flex;\"><span>\t<span style=\"color:#75715e\">\/\/ create a larger image\n<\/span><\/span><\/span><span style=\"display:flex;\"><span><span style=\"color:#75715e\"><\/span>\t<span style=\"color:#a6e22e\">size<\/span> <span style=\"color:#f92672\">:=<\/span> <span style=\"color:#a6e22e\">img<\/span>.<span style=\"color:#a6e22e\">Bounds<\/span>().<span style=\"color:#a6e22e\">Size<\/span>()\n<\/span><\/span><span style=\"display:flex;\"><span>\t<span style=\"color:#a6e22e\">newSize<\/span> <span style=\"color:#f92672\">:=<\/span> <span style=\"color:#a6e22e\">image<\/span>.<span style=\"color:#a6e22e\">Rect<\/span>(<span style=\"color:#ae81ff\">0<\/span>, <span style=\"color:#ae81ff\">0<\/span>, <span style=\"color:#a6e22e\">size<\/span>.<span style=\"color:#a6e22e\">X<\/span><span style=\"color:#f92672\">+<\/span><span style=\"color:#ae81ff\">20<\/span>, <span style=\"color:#a6e22e\">size<\/span>.<span style=\"color:#a6e22e\">Y<\/span><span style=\"color:#f92672\">+<\/span><span style=\"color:#ae81ff\">20<\/span>)\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span>\t<span style=\"color:#75715e\">\/\/ copy the original in this larger image, slightly translated to the center\n<\/span><\/span><\/span><span style=\"display:flex;\"><span><span style=\"color:#75715e\"><\/span>\t<span style=\"color:#66d9ef\">var<\/span> <span style=\"color:#a6e22e\">extended<\/span> <span style=\"color:#a6e22e\">image<\/span>.<span style=\"color:#a6e22e\">Image<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span>\t<span style=\"color:#a6e22e\">extended<\/span> = <span style=\"color:#a6e22e\">translateImage<\/span>(<span style=\"color:#a6e22e\">img<\/span>, <span style=\"color:#a6e22e\">newSize<\/span>, <span style=\"color:#ae81ff\">10<\/span>, <span style=\"color:#ae81ff\">10<\/span>)\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span>\t<span style=\"color:#75715e\">\/\/ dilate the image to have a bigger source of light\n<\/span><\/span><\/span><span style=\"display:flex;\"><span><span style=\"color:#75715e\"><\/span>\t<span style=\"color:#a6e22e\">dilated<\/span> <span style=\"color:#f92672\">:=<\/span> <span style=\"color:#a6e22e\">effect<\/span>.<span style=\"color:#a6e22e\">Dilate<\/span>(<span style=\"color:#a6e22e\">extended<\/span>, <span style=\"color:#ae81ff\">3<\/span>)\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span>\t<span style=\"color:#75715e\">\/\/ blur the image\n<\/span><\/span><\/span><span style=\"display:flex;\"><span><span style=\"color:#75715e\"><\/span>\t<span style=\"color:#a6e22e\">bloomed<\/span> <span style=\"color:#f92672\">:=<\/span> <span style=\"color:#a6e22e\">blur<\/span>.<span style=\"color:#a6e22e\">Gaussian<\/span>(<span style=\"color:#a6e22e\">dilated<\/span>, <span style=\"color:#ae81ff\">10.0<\/span>)\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span>\t<span style=\"color:#66d9ef\">return<\/span> <span style=\"color:#a6e22e\">bloomed<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span>}\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span><span style=\"color:#75715e\">\/\/ translateImage copies the src image applying the given offset on a new Image\n<\/span><\/span><\/span><span style=\"display:flex;\"><span><span style=\"color:#75715e\">\/\/ bounds is the size of the resulting image.\n<\/span><\/span><\/span><span style=\"display:flex;\"><span><span style=\"color:#75715e\"><\/span><span style=\"color:#66d9ef\">func<\/span> <span style=\"color:#a6e22e\">translateImage<\/span>(<span style=\"color:#a6e22e\">src<\/span> <span style=\"color:#a6e22e\">image<\/span>.<span style=\"color:#a6e22e\">Image<\/span>, <span style=\"color:#a6e22e\">bounds<\/span> <span style=\"color:#a6e22e\">image<\/span>.<span style=\"color:#a6e22e\">Rectangle<\/span>, <span style=\"color:#a6e22e\">xOffset<\/span>, <span style=\"color:#a6e22e\">yOffset<\/span> <span style=\"color:#66d9ef\">int<\/span>) <span style=\"color:#a6e22e\">image<\/span>.<span style=\"color:#a6e22e\">Image<\/span> {\n<\/span><\/span><span style=\"display:flex;\"><span>\t<span style=\"color:#a6e22e\">rv<\/span> <span style=\"color:#f92672\">:=<\/span> <span style=\"color:#a6e22e\">image<\/span>.<span style=\"color:#a6e22e\">NewRGBA<\/span>(<span style=\"color:#a6e22e\">bounds<\/span>)\n<\/span><\/span><span style=\"display:flex;\"><span>\t<span style=\"color:#a6e22e\">size<\/span> <span style=\"color:#f92672\">:=<\/span> <span style=\"color:#a6e22e\">src<\/span>.<span style=\"color:#a6e22e\">Bounds<\/span>().<span style=\"color:#a6e22e\">Size<\/span>()\n<\/span><\/span><span style=\"display:flex;\"><span>\t<span style=\"color:#66d9ef\">for<\/span> <span style=\"color:#a6e22e\">x<\/span> <span style=\"color:#f92672\">:=<\/span> <span style=\"color:#ae81ff\">0<\/span>; <span style=\"color:#a6e22e\">x<\/span> &lt; <span style=\"color:#a6e22e\">size<\/span>.<span style=\"color:#a6e22e\">X<\/span>; <span style=\"color:#a6e22e\">x<\/span><span style=\"color:#f92672\">++<\/span> {\n<\/span><\/span><span style=\"display:flex;\"><span>\t\t<span style=\"color:#66d9ef\">for<\/span> <span style=\"color:#a6e22e\">y<\/span> <span style=\"color:#f92672\">:=<\/span> <span style=\"color:#ae81ff\">0<\/span>; <span style=\"color:#a6e22e\">y<\/span> &lt; <span style=\"color:#a6e22e\">size<\/span>.<span style=\"color:#a6e22e\">Y<\/span>; <span style=\"color:#a6e22e\">y<\/span><span style=\"color:#f92672\">++<\/span> {\n<\/span><\/span><span style=\"display:flex;\"><span>\t\t\t<span style=\"color:#a6e22e\">rv<\/span>.<span style=\"color:#a6e22e\">Set<\/span>(<span style=\"color:#a6e22e\">xOffset<\/span><span style=\"color:#f92672\">+<\/span><span style=\"color:#a6e22e\">x<\/span>, <span style=\"color:#a6e22e\">yOffset<\/span><span style=\"color:#f92672\">+<\/span><span style=\"color:#a6e22e\">y<\/span>, <span style=\"color:#a6e22e\">src<\/span>.<span style=\"color:#a6e22e\">At<\/span>(<span style=\"color:#a6e22e\">x<\/span>, <span style=\"color:#a6e22e\">y<\/span>))\n<\/span><\/span><span style=\"display:flex;\"><span>\t\t}\n<\/span><\/span><span style=\"display:flex;\"><span>\t}\n<\/span><\/span><span style=\"display:flex;\"><span>\t<span style=\"color:#66d9ef\">return<\/span> <span style=\"color:#a6e22e\">rv<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span>}\n<\/span><\/span><\/code><\/pre><\/div>"},{"title":"Build a Linux kernel running only a Go binary","link":"https:\/\/remy.io\/blog\/custom-built-kernel-running-go-binary\/","pubDate":"Sun, 13 Oct 2019 17:20:44 +0100","author":"contact@remy.io (R\u00e9my Mathieu)","guid":"https:\/\/remy.io\/blog\/custom-built-kernel-running-go-binary\/","description":"<h1 id=\"introduction\">Introduction<\/h1>\n<p>I&rsquo;ve recently thought that it&rsquo;s been long since I&rsquo;ve booted a Linux kernel that\nI&rsquo;ve myself compiled. For some reasons, I also thought about the fact that it would\nbe very easy to boot a Linux running only a Go binary, as they&rsquo;re statically built\nif you don&rsquo;t use external C or anything exotic.<\/p>\n<p>This article is going through the Linux compilation process and a tiny initramfs creation\nto be able to boot a Go binary.<\/p>\n<h2 id=\"compile-linux\">Compile Linux<\/h2>\n<h2 id=\"configure\">Configure<\/h2>\n<p>First stop: <a href=\"https:\/\/kernel.org\">https:\/\/kernel.org<\/a><\/p>\n<p>At the time of writing, Linux 5.3.6 is the stable version so, let&rsquo;s go for it.<\/p>\n<div class=\"highlight\"><pre tabindex=\"0\" style=\"color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;\"><code class=\"language-bash\" data-lang=\"bash\"><span style=\"display:flex;\"><span>$ curl https:\/\/cdn.kernel.org\/pub\/linux\/kernel\/v5.x\/linux-5.3.6.tar.xz| tar Jx\n<\/span><\/span><span style=\"display:flex;\"><span>$ cd linux-5.3.6\n<\/span><\/span><\/code><\/pre><\/div><p>Before compiling Linux, you&rsquo;ll want to configure what you actually want shipped\nin the kernel. Linux comes with a menu to help you configure that.<\/p>\n<div class=\"highlight\"><pre tabindex=\"0\" style=\"color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;\"><code class=\"language-bash\" data-lang=\"bash\"><span style=\"display:flex;\"><span>$ make menuconfig\n<\/span><\/span><\/code><\/pre><\/div><p>At this point, it&rsquo;s up to you. If you want to run the image on your machine without virtualization\nand you&rsquo;re doing this using Linux, you can use the command <code>make localmodconfig<\/code>\nwhich configures the kernel with what&rsquo;s available in the running one.<\/p>\n<p>If the plan is to run it in qemu (or something else), I would advise\nyou to remove everything that will be unused to speed up the build:<\/p>\n<ul>\n<li>bluetooth<\/li>\n<li>drm \/ intel drivers<\/li>\n<li>sound card support<\/li>\n<li>pcmcia<\/li>\n<li>even network drivers could be removed&hellip;<\/li>\n<\/ul>\n<p><img src=\"https:\/\/remy.io\/img\/articles\/go_binary\/custom_linux_menuconfig.png\" alt=\"The Linux menu to configure the build\"><\/p>\n<h2 id=\"build\">Build<\/h2>\n<p>When you are happy with your configuration, build the kernel:<\/p>\n<div class=\"highlight\"><pre tabindex=\"0\" style=\"color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;\"><code class=\"language-bash\" data-lang=\"bash\"><span style=\"display:flex;\"><span>$ make -j4\n<\/span><\/span><\/code><\/pre><\/div><p>Use <code>-j<\/code> with the value which fits the best your CPU to indicate to <code>make<\/code> how\nmany jobs it can run simultaneously.<\/p>\n<p>Here&rsquo;s the result for my compilation configuration:<\/p>\n<div class=\"highlight\"><pre tabindex=\"0\" style=\"color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;\"><code class=\"language-bash\" data-lang=\"bash\"><span style=\"display:flex;\"><span>Setup is <span style=\"color:#ae81ff\">16764<\/span> bytes <span style=\"color:#f92672\">(<\/span>padded to <span style=\"color:#ae81ff\">16896<\/span> bytes<span style=\"color:#f92672\">)<\/span>.\n<\/span><\/span><span style=\"display:flex;\"><span>System is <span style=\"color:#ae81ff\">6493<\/span> kB\n<\/span><\/span><span style=\"display:flex;\"><span>CRC 6e1285d1\n<\/span><\/span><span style=\"display:flex;\"><span>Kernel: arch\/x86\/boot\/bzImage is ready  <span style=\"color:#f92672\">(<\/span><span style=\"color:#75715e\">#1)<\/span>\n<\/span><\/span><\/code><\/pre><\/div><p>Where we can see that a 6493 kB kernel has been built.<\/p>\n<h2 id=\"run\">Run<\/h2>\n<p>In order to test the kernel you have just built, there is a convenient option\nin <code>qemu<\/code> allowing you to directly boot a <code>bzImage<\/code> file:<\/p>\n<div class=\"highlight\"><pre tabindex=\"0\" style=\"color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;\"><code class=\"language-bash\" data-lang=\"bash\"><span style=\"display:flex;\"><span>$ qemu-system-x86_64 --enable-kvm -kernel linux-5.3.6\/arch\/x86\/boot\/bzImage\n<\/span><\/span><\/code><\/pre><\/div><p><img src=\"https:\/\/remy.io\/img\/articles\/go_binary\/custom_linux_1.png\" alt=\"First run of this kernel\"><\/p>\n<p>Looks like it is booting! But something is wrong: it can&rsquo;t mount a rootfs, which\nmakes total sense since we didn&rsquo;t provide one.<\/p>\n<p>If you prefer to run it directly in your terminal:<\/p>\n<div class=\"highlight\"><pre tabindex=\"0\" style=\"color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;\"><code class=\"language-bash\" data-lang=\"bash\"><span style=\"display:flex;\"><span>$ qemu-system-x86_64 --enable-kvm -kernel linux-5.3.6\/arch\/x86\/boot\/bzImage -nographic -append <span style=\"color:#e6db74\">&#34;console=ttyS0&#34;<\/span>\n<\/span><\/span><\/code><\/pre><\/div><p><img src=\"https:\/\/remy.io\/img\/articles\/go_binary\/custom_linux_2.png\" alt=\"Running it in the terminal\"><\/p>\n<p>Use <code>C-a x<\/code> to exit qemu.<\/p>\n<h1 id=\"use-initramfs-as-a-rootfs\">Use initramfs as a rootfs<\/h1>\n<p>Since Linux 2.6.13, the kernel is capable of booting directly on a cpio archive,\nwithout requiring a file system or a block driver. Basically, everything will run\nfrom the RAM and you won&rsquo;t need to mount anything. All you have to do to get\nstarted is to use the command <code>cpio<\/code> to create the archive that will be loaded in\nRAM and used as the rootfs.<\/p>\n<p>What you want to put inside is up to you (binaries, libraries, etc.).<\/p>\n<h2 id=\"create-an-initramfs-containing-busybox\">Create an initramfs containing busybox<\/h2>\n<p>If for example you want <code>ls<\/code>, <code>sh<\/code>, <code>cat<\/code> and other useful binaries, you can\ncompile <a href=\"https:\/\/busybox.net\">busybox<\/a> and add these binaries into your initramfs.\nMake sure to compile busybox statically though!<\/p>\n<div class=\"highlight\"><pre tabindex=\"0\" style=\"color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;\"><code class=\"language-bash\" data-lang=\"bash\"><span style=\"display:flex;\"><span>$ wget https:\/\/busybox.net\/downloads\/busybox-1.31.0.tar.bz2|tar xJ\n<\/span><\/span><span style=\"display:flex;\"><span>$ cd busybox-1.31.0\n<\/span><\/span><span style=\"display:flex;\"><span>$ make menuconfig <span style=\"color:#75715e\"># select the option to build static binaries<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span>$ make -j4\n<\/span><\/span><span style=\"display:flex;\"><span>$ make install\n<\/span><\/span><\/code><\/pre><\/div><p>Now that busybox is compiled, let&rsquo;s fill a directory with the content of our\nfuture initramfs:<\/p>\n<div class=\"highlight\"><pre tabindex=\"0\" style=\"color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;\"><code class=\"language-bash\" data-lang=\"bash\"><span style=\"display:flex;\"><span>$ mkdir -p initramfs-busybox\/<span style=\"color:#f92672\">{<\/span>bin,sbin,etc,proc,sys,usr\/bin,usr\/sbin<span style=\"color:#f92672\">}<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span>$ cp -a busybox-1.31.0\/_install\/* initramfs-busybox\/\n<\/span><\/span><\/code><\/pre><\/div><p>What we also want is an <code>init<\/code> binary available in the root of our initramfs,\nbecause Linux will have the good idea of starting this binary just after it has\nbooted!<\/p>\n<p>As we have <code>sh<\/code> available, we can create an executable script calling&hellip; <code>sh<\/code> to\nboot on a shell interpreter.<\/p>\n<div class=\"highlight\"><pre tabindex=\"0\" style=\"color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;\"><code class=\"language-bash\" data-lang=\"bash\"><span style=\"display:flex;\"><span>$ echo <span style=\"color:#e6db74\">&#39;#!\/bin\/sh\\necho &#34;Welcome to your custom Linux.&#34;\\n\/bin\/sh&#39;<\/span> &gt; initramfs-busybox\/init\n<\/span><\/span><span style=\"display:flex;\"><span>$ chmod +x initramfs-busybox\/init\n<\/span><\/span><\/code><\/pre><\/div><p>Finally, create the initramfs archive with the content of the directory:<\/p>\n<div class=\"highlight\"><pre tabindex=\"0\" style=\"color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;\"><code class=\"language-bash\" data-lang=\"bash\"><span style=\"display:flex;\"><span>$ cd initramfs-busybox\n<\/span><\/span><span style=\"display:flex;\"><span>$ find . -print0 | cpio --null -ov --format<span style=\"color:#f92672\">=<\/span>newc | gzip -9 &gt; ..\/initramfs.cpio.gz\n<\/span><\/span><span style=\"display:flex;\"><span>$ cd ..\/\n<\/span><\/span><\/code><\/pre><\/div><p>And voil\u00e0, <code>initramfs.cpio.gz<\/code> is our compressed initramfs!  In order to boot with it:<\/p>\n<div class=\"highlight\"><pre tabindex=\"0\" style=\"color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;\"><code class=\"language-bash\" data-lang=\"bash\"><span style=\"display:flex;\"><span>$ qemu-system-x86_64 --enable-kvm -kernel linux-5.3.6\/arch\/x86\/boot\/bzImage <span style=\"color:#ae81ff\">\\\n<\/span><\/span><\/span><span style=\"display:flex;\"><span><span style=\"color:#ae81ff\"><\/span>                -nographic -append <span style=\"color:#e6db74\">&#34;console=ttyS0&#34;<\/span> -initrd initramfs.cpio.gz\n<\/span><\/span><\/code><\/pre><\/div><p>If this doesn&rsquo;t work, make sure you&rsquo;ve compiled the kernel with the initramfs support.<\/p>\n<p><img src=\"https:\/\/remy.io\/img\/articles\/go_binary\/custom_linux_3.png\" alt=\"A minimal Linux kernel with busybox binaries.\"><\/p>\n<h2 id=\"create-a-minimal-initramfs-with-only-a-go-binary\">Create a minimal initramfs with only a Go binary<\/h2>\n<p>Now that we know how to create an initramfs and do something useful with it,\nback to our initial goal: let&rsquo;s boot on a Go binary.<\/p>\n<p>From there, it will actually be very easy: we just have to build our binary,\nmake sure it is static, send it to an empty directory named as <code>init<\/code> and create\nan initramfs with the content of this directory!<\/p>\n<p>Let&rsquo;s say we have this Go file in a directory called <code>source<\/code>:<\/p>\n<div class=\"highlight\"><pre tabindex=\"0\" style=\"color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;\"><code class=\"language-go\" data-lang=\"go\"><span style=\"display:flex;\"><span><span style=\"color:#f92672\">package<\/span> <span style=\"color:#a6e22e\">main<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span><span style=\"color:#f92672\">import<\/span> (\n<\/span><\/span><span style=\"display:flex;\"><span>\t<span style=\"color:#e6db74\">&#34;time&#34;<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span>\t<span style=\"color:#e6db74\">&#34;fmt&#34;<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span>)\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span><span style=\"color:#66d9ef\">func<\/span> <span style=\"color:#a6e22e\">main<\/span>() {\n<\/span><\/span><span style=\"display:flex;\"><span>\t<span style=\"color:#a6e22e\">fmt<\/span>.<span style=\"color:#a6e22e\">Println<\/span>(<span style=\"color:#e6db74\">&#34;Hello from your custom built Linux kernel.&#34;<\/span>)\n<\/span><\/span><span style=\"display:flex;\"><span>\t<span style=\"color:#a6e22e\">time<\/span>.<span style=\"color:#a6e22e\">Sleep<\/span>(<span style=\"color:#a6e22e\">time<\/span>.<span style=\"color:#a6e22e\">Second<\/span><span style=\"color:#f92672\">*<\/span><span style=\"color:#ae81ff\">1<\/span>)\n<\/span><\/span><span style=\"display:flex;\"><span>\t<span style=\"color:#a6e22e\">fmt<\/span>.<span style=\"color:#a6e22e\">Println<\/span>(<span style=\"color:#e6db74\">&#34;Booting on a Go binary!&#34;<\/span>)\n<\/span><\/span><span style=\"display:flex;\"><span>    <span style=\"color:#66d9ef\">for<\/span> {} <span style=\"color:#75715e\">\/\/ never stop!\n<\/span><\/span><\/span><span style=\"display:flex;\"><span><span style=\"color:#75715e\"><\/span>}\n<\/span><\/span><\/code><\/pre><\/div><p>Let&rsquo;s compile it and build an initramfs with only this binary:<\/p>\n<div class=\"highlight\"><pre tabindex=\"0\" style=\"color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;\"><code class=\"language-bash\" data-lang=\"bash\"><span style=\"display:flex;\"><span>$ cd source\n<\/span><\/span><span style=\"display:flex;\"><span>$ go build -o init\n<\/span><\/span><span style=\"display:flex;\"><span>$ cd ..\/\n<\/span><\/span><span style=\"display:flex;\"><span>$ mkdir initramfs-go\n<\/span><\/span><span style=\"display:flex;\"><span>$ mv source\/init initramfs-go\/init\n<\/span><\/span><span style=\"display:flex;\"><span>$ cd initramfs-go\n<\/span><\/span><span style=\"display:flex;\"><span>$ find . -print0 | cpio --null -ov --format<span style=\"color:#f92672\">=<\/span>newc | gzip -9 &gt; ..\/initramfs.cpio.gz\n<\/span><\/span><\/code><\/pre><\/div><p>And&hellip; that&rsquo;s pretty much it. We can now boot on this initramfs:<\/p>\n<div class=\"highlight\"><pre tabindex=\"0\" style=\"color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;\"><code class=\"language-bash\" data-lang=\"bash\"><span style=\"display:flex;\"><span>$ qemu-system-x86_64 --enable-kvm -kernel linux-5.3.6\/arch\/x86\/boot\/bzImage <span style=\"color:#ae81ff\">\\\n<\/span><\/span><\/span><span style=\"display:flex;\"><span><span style=\"color:#ae81ff\"><\/span>                -nographic -append <span style=\"color:#e6db74\">&#34;console=ttyS0&#34;<\/span> -initrd initramfs.cpio.gz\n<\/span><\/span><\/code><\/pre><\/div><p><img src=\"https:\/\/remy.io\/img\/articles\/go_binary\/custom_linux_4.png\" alt=\"A minimal Linux kernel running only a Go binary.\"><\/p>\n<h1 id=\"conclusion\">Conclusion<\/h1>\n<p>Since the Go binaries are static, it&rsquo;s very easy to boot them as the init process\nof a Linux kernel. We have to keep in mind that &ldquo;it just works&rdquo; because I&rsquo;m building\neverything on the same CPU architecture: we would have to involve cross-compilation\nif we wanted to do this for another architecture.<\/p>\n<p>However, starting from here, there are many things that would be very interesting\nto develop such as an actual init system!<\/p>\n<p>If you&rsquo;ve reached this point and liked this article, thank you and don&rsquo;t hesitate\nto ping me on <a href=\"https:\/\/twitter.com\/remeh\">Twitter<\/a>.<\/p>\n<p>Some links I&rsquo;ve used:<\/p>\n<ul>\n<li><a href=\"https:\/\/en.wikipedia.org\/wiki\/Initial_ramdisk\">Initial ramdisk wikipedia article<\/a><\/li>\n<li><a href=\"http:\/\/www.linuxfromscratch.org\/blfs\/view\/svn\/postlfs\/initramfs.html\">LFS initramfs page<\/a><\/li>\n<li><a href=\"https:\/\/gist.github.com\/chrisdone\/02e165a0004be33734ac2334f215380e\">This gist<\/a><\/li>\n<\/ul>\n"},{"title":"Rendering a map using Go, Mapbox and OpenStreetMap","link":"https:\/\/remy.io\/blog\/rendering-a-map-using-go-mapbox-and-openstreetmap\/","pubDate":"Sun, 21 Oct 2018 20:00:51 +0100","author":"contact@remy.io (R\u00e9my Mathieu)","guid":"https:\/\/remy.io\/blog\/rendering-a-map-using-go-mapbox-and-openstreetmap\/","description":"<h1 id=\"introduction\">Introduction<\/h1>\n<p>Fantasy books or novels have a lot of success all around the world, such as the famous <a href=\"https:\/\/en.wikipedia.org\/wiki\/The_Lord_of_the_Rings\">The Lord of the Rings<\/a> or <a href=\"https:\/\/en.wikipedia.org\/wiki\/Magician_(Feist_novel)\">Magician<\/a>. In those books, in order to help you feel the atmosphere of the world your heroes are evolving in, you often find a map of the world. Most of the times they look hand-drawn and they also have a fantasy design.\nA well-known example would be the map of the <a href=\"https:\/\/en.wikipedia.org\/wiki\/Land_of_Oz#\/media\/File:Map-of-Oz.jpg\">Land of Oz<\/a>.<\/p>\n<p>In the past, I had a project of hand-drawing such a map of the place where I&rsquo;ve grown. I guess that it was an inspiration when I started this one.<\/p>\n<p>As a first step, this article will cover how to retrieve map data and how to draw it, but without any fantasy style. The second step of rendering a fantasy map will be for another article if I actually succeed to do so!<\/p>\n<h2 id=\"how-to-retrieve-the-map-data\">How to retrieve the map data<\/h2>\n<h3 id=\"locate-the-place-to-render\">Locate the place to render<\/h3>\n<p>First, I want to textually look for the zone I want to draw. But what actually happens when you type your address on Google Maps? How does it locate the map on the address you typed? Turning an address into latitude\/longitude coordinates is called \u00ab forward geocoding \u00bb. The reverse, turning latitude\/longitude coordinates into text, is called \u00ab reverse geocoding \u00bb.<\/p>\n<p>One would need a lot of information and data to actually implement a geocoding feature and this is where the <a href=\"https:\/\/www.mapbox.com\/api-documentation\/#geocoding\">Mapbox Geocoding API<\/a> will be great: they provide a geocoding API that will help us retrieve latitude\/longitude coordinates from text inputs.<\/p>\n<p><img src=\"https:\/\/remy.io\/img\/articles\/rendering-map\/mapbox-geocode.png\" alt=\"Geocoding &ldquo;Annecy, 74000, France&rdquo;\"><\/p>\n<h3 id=\"get-a-bounding-box\">Get a bounding box<\/h3>\n<p>A bounding box, also called an extent, is basically the rectangle containing all the data we&rsquo;re interested in.<\/p>\n<p><img src=\"https:\/\/remy.io\/img\/articles\/rendering-map\/boundingbox.png\" alt=\"Drawing of a bounding box\"><\/p>\n<p>The <a href=\"https:\/\/www.mapbox.com\/api-documentation\/#geocoding\">Mapbox API documentation<\/a> explains that I retrieve \u00ab Feature \u00bb objects. There is many feature types in the Mapbox API but those interesting me are the type \u00ab place \u00bb and the type \u00ab poi \u00bb.<\/p>\n<p>The good thing with the type \u00ab place \u00bb is that it ships a bounding box. However it could be smart to check whether or not this bounding box is very large: if it is, too much objects will be returned and drawn. Let&rsquo;s not bother too much about this for now.<\/p>\n<p>The other type \u00ab poi \u00bb means Point Of Interest, it is a simple position with no bounding box, for example the position of a fountain in the middle of a place. But it&rsquo;s enough because, with approximate math, I&rsquo;m able to compute a bounding box.<\/p>\n<h3 id=\"get-the-data-from-openstreetmap\">Get the data from OpenStreetMap<\/h3>\n<h4 id=\"openstreetmap\">OpenStreetMap<\/h4>\n<p><a href=\"https:\/\/openstreetmap.org\">OpenStreetMap<\/a> is an open &ldquo;database&rdquo; \u00ab built by a community of mappers that contribute and maintain data about roads, trails, caf\u00e9s, railway stations, and much more, all over the world. \u00bb<\/p>\n<p>It is a tremendous source of data and this is in this huge database that I will retrieve everything I need to start rendering the maps.<\/p>\n<p>However, I will host this project on a small VPS with only 10 Gb of storage. At the time of writing, the XML file containing all the data for the whole planet is 69GB compressed. I need to fetch the data from an external source, moreover, I need to fetch only the data inside the bounding box I&rsquo;ve previously retrieved or computed.<\/p>\n<h4 id=\"overpass-api\">Overpass API<\/h4>\n<p>OpenStreetMap offers such a solution: the <a href=\"https:\/\/wiki.openstreetmap.org\/wiki\/Overpass_API\">Overpass API<\/a>. Very powerful, it offers a complete language of query in order to ask for specific data in a given zone. Thus, getting the data only consist of an HTTP call to this service. We must be cautious to not spam it as, again, it seems to be runned by volunteers. Example of a call on the API to retrieve objects inside a given bounding box:<\/p>\n<div class=\"highlight\"><pre tabindex=\"0\" style=\"color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;\"><code class=\"language-bash\" data-lang=\"bash\"><span style=\"display:flex;\"><span>curl https:\/\/overpass-api.de\/api\/map?bbox<span style=\"color:#f92672\">=<\/span>5.656179,45.708292,5.730329,45.776623\n<\/span><\/span><\/code><\/pre><\/div><p>This API route returns XML data using the OpenStreetMap format: in order to turn it into useful Go objects, I&rsquo;ve used the library <a href=\"https:\/\/github.com\/glaslos\/go-osm\">go-osm<\/a>.<\/p>\n<p>This done, using the OpenStreetMap objects definition in their documentation and their website which gives all information on each object in the map, I was able to characterize most of the objects that I want to draw.<\/p>\n<p>Example of the description of the OpenStreetMap object type \u00ab Relation \u00bb for the \u00ab Lac d&rsquo;Annecy \u00bb:<\/p>\n<p><img src=\"https:\/\/remy.io\/img\/articles\/rendering-map\/lac-annecy.png\" alt=\"Example of the description of the OpenStreetMap object type \u00ab Relation \u00bb for the &ldquo;Lac d&rsquo;Annecy&rdquo;\"><\/p>\n<h2 id=\"rendering\">Rendering<\/h2>\n<p>Now that I&rsquo;ve gathered all the objects contained in the zone I want to draw, we can start the rendering process to a PNG file.<\/p>\n<p>I chose the great <a href=\"https:\/\/github.com\/fogleman\">gg library<\/a> library in order to draw primitives such as areas and lines which actually represents the roads, the buildings, the rivers, etc.<\/p>\n<p>Using this library, the rendering is the easiest part as the gg API fits this use-case very well.<\/p>\n<h4 id=\"rendering-a-road\">Rendering a road<\/h4>\n<p>A road will simply be drawn as a line, not much to say here. OpenStreetMap does the difference between major roads and minor roads, paths, etc. I&rsquo;ve just changed the thinness depending on this information.<\/p>\n<p>First rendering I&rsquo;ve done, drawing only roads around \u00ab Lyon, France \u00bb:<\/p>\n<p><img src=\"https:\/\/remy.io\/img\/articles\/rendering-map\/roads.png\" alt=\"First rendering I&rsquo;ve done, drawing only roads around Lyon, France\"><\/p>\n<h4 id=\"rendering-a-river\">Rendering a river<\/h4>\n<p>A river is not a simple line: it can be large or it can be very thin but it will be better to render it as an area. For example: when you get the object \u00ab Lac d&rsquo;Annecy \u00bb from OpenStreetMap, you actually get an object with many inner and outer lines. In order to draw it we will want to use the \u00ab outer \u00bb line as the borders of a polygon, and fill its content with a blue-ish color.<\/p>\n<p>Example of rendering only the water areas around Lyon, France:<\/p>\n<p><img src=\"https:\/\/remy.io\/img\/articles\/rendering-map\/water.png\" alt=\"Rendering water areas around Lyon\"><\/p>\n<p>I won&rsquo;t detail the rendering of each type of objects but you got the main idea.<\/p>\n<h2 id=\"results\">Results<\/h2>\n<p>Finally, here are some rendering of different zones of the world:<\/p>\n<h3 id=\"luxembourg\">Luxembourg<\/h3>\n<p><img src=\"https:\/\/remy.io\/img\/articles\/rendering-map\/luxembourg.png\" alt=\"Rendering of Luxembourg\"><\/p>\n<h3 id=\"lyon-france\">Lyon, France<\/h3>\n<p><img src=\"https:\/\/remy.io\/img\/articles\/rendering-map\/lyon.png\" alt=\"Rendering of Lyon\"><\/p>\n<h3 id=\"a-street-of-montreal-quebec\">A street of Montreal, Quebec<\/h3>\n<p><img src=\"https:\/\/remy.io\/img\/articles\/rendering-map\/montreal.png\" alt=\"Rendering a street of Montreal\"><\/p>\n<h1 id=\"conclusion\">Conclusion<\/h1>\n<p>There is still many things wrong with this map rendering, for exemple, I didn&rsquo;t apply any <a href=\"https:\/\/en.wikipedia.org\/wiki\/Map_projection\">projection<\/a> on the data while drawing it on a 2D plan, thus, it is not really accurate and even badly render some parts of the world. It is a complex topic and I didn&rsquo;t want to deal with it for now.<\/p>\n<p>Moreover, there is still some OpenStreetMap objects that I don&rsquo;t process and that I don&rsquo;t draw.<\/p>\n<p>Obviously, the next step will be to render all these objects using a hand-drawn fantasy style, and this is where the real challenge will appear!<\/p>\n<p>Please, let me know if you liked or disliked this article. Don&rsquo;t hesitate to contact me on <a href=\"https:\/\/twitter.com\/remeh\">Twitter<\/a> or to follow me on <a href=\"https:\/\/github.com\/remeh\">Github<\/a>.<\/p>\n"},{"title":"Stream Maker","link":"https:\/\/remy.io\/projects\/stream-maker\/","pubDate":"Wed, 21 Feb 2018 19:20:44 +0100","author":"contact@remy.io (R\u00e9my Mathieu)","guid":"https:\/\/remy.io\/projects\/stream-maker\/","description":{}},{"title":"Les 12 Derniers","link":"https:\/\/remy.io\/projects\/les-12-derniers\/","pubDate":"Wed, 10 Jan 2018 19:20:44 +0100","author":"contact@remy.io (R\u00e9my Mathieu)","guid":"https:\/\/remy.io\/projects\/les-12-derniers\/","description":{}},{"title":"Simple tools to improve your Go code","link":"https:\/\/remy.io\/blog\/simple-tools-to-improve-your-go-code\/","pubDate":"Thu, 09 Nov 2017 14:00:51 +0100","author":"contact@remy.io (R\u00e9my Mathieu)","guid":"https:\/\/remy.io\/blog\/simple-tools-to-improve-your-go-code\/","description":"<h1 id=\"tools\">Tools<\/h1>\n<p>Go has the chance to have many static checkers available, they help you to produce simpler and more reliable code. Nowadays, some editors directly integrate them but they are also stand-alone tools that you can run by yourself directly on your source code.<\/p>\n<p>I&rsquo;ll go through a list of the ones I find really helpful, please note that this list is not exhaustive.<\/p>\n<h2 id=\"gosimple\">gosimple<\/h2>\n<p>Starting easy, <code>gosimple<\/code> looks for things which can be simplified in the code. Let&rsquo;s look through this example:<\/p>\n<div class=\"highlight\"><pre tabindex=\"0\" style=\"color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;\"><code class=\"language-go\" data-lang=\"go\"><span style=\"display:flex;\"><span><span style=\"color:#75715e\">\/\/ validateContent returns whether or not this text\/category\n<\/span><\/span><\/span><span style=\"display:flex;\"><span><span style=\"color:#75715e\">\/\/ is applicable.\n<\/span><\/span><\/span><span style=\"display:flex;\"><span><span style=\"color:#75715e\"><\/span><span style=\"color:#66d9ef\">func<\/span> (<span style=\"color:#a6e22e\">s<\/span> <span style=\"color:#f92672\">*<\/span><span style=\"color:#a6e22e\">Struct<\/span>) <span style=\"color:#a6e22e\">validateContent<\/span>() <span style=\"color:#66d9ef\">bool<\/span> {\n<\/span><\/span><span style=\"display:flex;\"><span>\t<span style=\"color:#66d9ef\">if<\/span> <span style=\"color:#a6e22e\">strings<\/span>.<span style=\"color:#a6e22e\">Count<\/span>(<span style=\"color:#a6e22e\">w<\/span>.<span style=\"color:#a6e22e\">text<\/span>, <span style=\"color:#e6db74\">&#34; &#34;<\/span>) &gt; <span style=\"color:#ae81ff\">4<\/span> {\n<\/span><\/span><span style=\"display:flex;\"><span>\t\t<span style=\"color:#66d9ef\">return<\/span> <span style=\"color:#66d9ef\">false<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span>\t}\n<\/span><\/span><span style=\"display:flex;\"><span>\t<span style=\"color:#66d9ef\">return<\/span> <span style=\"color:#66d9ef\">true<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span>}\n<\/span><\/span><\/code><\/pre><\/div><p>Let&rsquo;s say this code is somewhere in my project, I can start <code>gosimple<\/code> on my project by running the command: <code>gosimple .\/...<\/code> This will be the case for every tools presented in this article.<\/p>\n<p><code>gosimple<\/code> will indicate you that it is better to directly return the value of the boolean test:<\/p>\n<div class=\"highlight\"><pre tabindex=\"0\" style=\"color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;\"><code class=\"language-bash\" data-lang=\"bash\"><span style=\"display:flex;\"><span>\tshould use <span style=\"color:#e6db74\">&#39;return &lt;expr&gt;&#39;<\/span> instead of <span style=\"color:#e6db74\">&#39;if &lt;expr&gt; { return &lt;bool&gt; }; return &lt;bool&gt;&#39;<\/span>\n<\/span><\/span><\/code><\/pre><\/div><p>Obviously, this is an example but there is a lot of other rules managed by <code>gosimple<\/code> which <a href=\"https:\/\/staticcheck.io\/docs\/gosimple\">can be found here<\/a>.<\/p>\n<h2 id=\"golint\">golint<\/h2>\n<p><code>golint<\/code> checks for style mistakes.<\/p>\n<p>For example, it&rsquo;ll report you when a public method doesn&rsquo;t have any comment.  If there is a comment, it reports if you don&rsquo;t follow the best practice. If you wrote a method called <code>newJsonData()<\/code>, it&rsquo;ll report you that it is a better practice to write <code>newJSONData()<\/code>, and so on.<\/p>\n<p><a href=\"https:\/\/github.com\/golang\/lint\">Link<\/a><\/p>\n<h2 id=\"go-vet--staticcheck\">go vet \/ staticcheck<\/h2>\n<p>Shipped with go, there is a tool that you can call using the command <code>go vet<\/code>.<\/p>\n<p>This tool applies static checks to your code, meaning that some defined rules are tested, such as suspicious Printf. You know, the <code>fmt.Sprintf(&quot;%s%d&quot;, var1)<\/code> where you forget the second var in the parameters!<\/p>\n<p>I would advice you to also use <code>staticcheck<\/code>, it is the same things but with other rules tested.<\/p>\n<p><a href=\"https:\/\/staticcheck.io\">Link<\/a> \u2014 <a href=\"https:\/\/staticcheck.io\/docs\/staticcheck\">Link of applied rules<\/a><\/p>\n<h2 id=\"unused--unparam\">unused \/ unparam<\/h2>\n<p>When you use <code>go fmt<\/code>, the linter + your brain, you may think it&rsquo;s enough to not let unused things here and there in the code.<\/p>\n<p><code>unused<\/code> is here to remind you that it is actually wrong and that it probably stays unused things: it reports when there is unused function or methods in your code, unused constants and so on.<\/p>\n<p>While the detected parts are not harmful as they are unused, it is great to remove these extra informations, for example for newcomers on the codebase, it avoids having them trying to understand these things for nothing.<\/p>\n<p><a href=\"https:\/\/github.com\/dominikh\/go-tools\/tree\/master\/cmd\/unused\">Link<\/a><\/p>\n<p><code>unparam<\/code> is even more useful in my opinion as it indicates you unused parameters in a method or a function. An unused parameter in a function is probably due to the purpose of the function having changed (maybe it should be better to rename it?) or even a critical oversight as the go compiler doesn&rsquo;t notify of unused method parameters.<\/p>\n<p><a href=\"https:\/\/github.com\/mvdan\/unparam\">Link<\/a><\/p>\n<h2 id=\"errcheck\">errcheck<\/h2>\n<p><code>errcheck<\/code> looks for errors returned by methods and not checked in your code. The compiler is not complaining when you silent an error, you do need someone to remind you that it is not necessarily a good idea! This is where errcheck enters the game.<\/p>\n<p><a href=\"https:\/\/github.com\/kisielk\/errcheck\">Link<\/a><\/p>\n<h2 id=\"gometalinter\">gometalinter<\/h2>\n<p>It is a tool trying to regroup all the existing ones: it concurrently runs every tools and normalises their output. <code>gometalinter<\/code> avoids you the pain of running these tools one by one, but in exchange, you can be flooded by a whole lot of results.<\/p>\n<p><a href=\"https:\/\/github.com\/alecthomas\/gometalinter\">The github repository<\/a> is therefore a good place to start to have information on the major parts of existing static checkers for Go.<\/p>\n<p><a href=\"https:\/\/github.com\/alecthomas\/gometalinter\">Link<\/a><\/p>\n<h1 id=\"conclusion\">Conclusion<\/h1>\n<p>You probably can run them every times you save your files such as we do with <code>go fmt<\/code>, some great plugins such as <a href=\"https:\/\/github.com\/fatih\/vim-go\">vim-go<\/a> or editors such as <a href=\"https:\/\/code.visualstudio.com\/\">Visual Studio Code<\/a> can directly integrate <code>gometalinter<\/code>, some plugins are even providing inline hints. For myself, I occasionally start them manually enjoying to improve my code and my best practices, step by step.<\/p>\n<p>Finally, if you&rsquo;re brave enough and patient, you can directly integrate them in your CI, before the build step.<\/p>\n<p>This is up to you to decide when you want to apply these tools and that&rsquo;s the great part.<\/p>\n<h1 id=\"links\">Links<\/h1>\n<p>For more information on all these tools, I&rsquo;ll redirect you to the <a href=\"https:\/\/github.com\/alecthomas\/gometalinter\">gometalinter<\/a> repository or to <a href=\"https:\/\/github.com\/dominikh\/go-tools\">this github repository<\/a> which also contains some useful information.<\/p>\n<p>If you&rsquo;ve find useful or erroneous information, please don&rsquo;t hesitate to send me a tweet on <a href=\"https:\/\/twitter.com\/remeh\">@remeh<\/a>, I&rsquo;ll be glad to answer or to communicate with you.<\/p>\n"},{"title":"Smartwitter","link":"https:\/\/remy.io\/projects\/smartwitter\/","pubDate":"Thu, 10 Aug 2017 19:20:44 +0100","author":"contact@remy.io (R\u00e9my Mathieu)","guid":"https:\/\/remy.io\/projects\/smartwitter\/","description":{}},{"title":"Memoiz","link":"https:\/\/remy.io\/projects\/memoiz\/","pubDate":"Fri, 10 Feb 2017 19:20:44 +0100","author":"contact@remy.io (R\u00e9my Mathieu)","guid":"https:\/\/remy.io\/projects\/memoiz\/","description":{}},{"title":"Sized Wait Group","link":"https:\/\/remy.io\/projects\/sized-wait-group\/","pubDate":"Mon, 21 Nov 2016 19:20:44 +0100","author":"contact@remy.io (R\u00e9my Mathieu)","guid":"https:\/\/remy.io\/projects\/sized-wait-group\/","description":{}},{"title":"SizedWaitGroup: a Golang WaitGroup with throttling","link":"https:\/\/remy.io\/blog\/sized-wait-group\/","pubDate":"Mon, 21 Nov 2016 19:20:44 +0100","author":"contact@remy.io (R\u00e9my Mathieu)","guid":"https:\/\/remy.io\/blog\/sized-wait-group\/","description":"<h1 id=\"what-is-a-waitgroup-\">What is a WaitGroup ?<\/h1>\n<p>In <a href=\"https:\/\/golang.org\">Golang<\/a>, there is a very nice feature available in the <code>sync<\/code> package: Wait Groups.<\/p>\n<p>The purpose of this feature is to help developers to launch many concurrent goroutines and to easily <strong>wait<\/strong> for each routine of this <strong>group<\/strong> to end, hence its name.<\/p>\n<p>Basic example:<\/p>\n<div class=\"highlight\"><pre tabindex=\"0\" style=\"color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;\"><code class=\"language-go\" data-lang=\"go\"><span style=\"display:flex;\"><span><span style=\"color:#f92672\">package<\/span> <span style=\"color:#a6e22e\">main<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span><span style=\"color:#f92672\">import<\/span> (\n<\/span><\/span><span style=\"display:flex;\"><span>        <span style=\"color:#e6db74\">&#34;fmt&#34;<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span>        <span style=\"color:#e6db74\">&#34;math\/rand&#34;<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span>\t<span style=\"color:#e6db74\">&#34;sync&#34;<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span>        <span style=\"color:#e6db74\">&#34;time&#34;<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span>)\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span><span style=\"color:#66d9ef\">func<\/span> <span style=\"color:#a6e22e\">main<\/span>() {\n<\/span><\/span><span style=\"display:flex;\"><span>        <span style=\"color:#a6e22e\">rand<\/span>.<span style=\"color:#a6e22e\">Seed<\/span>(<span style=\"color:#a6e22e\">time<\/span>.<span style=\"color:#a6e22e\">Now<\/span>().<span style=\"color:#a6e22e\">UnixNano<\/span>())\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span>\t<span style=\"color:#66d9ef\">var<\/span> <span style=\"color:#a6e22e\">wg<\/span> <span style=\"color:#a6e22e\">sync<\/span>.<span style=\"color:#a6e22e\">WaitGroup<\/span>{}\n<\/span><\/span><span style=\"display:flex;\"><span>        <span style=\"color:#66d9ef\">for<\/span> <span style=\"color:#a6e22e\">i<\/span> <span style=\"color:#f92672\">:=<\/span> <span style=\"color:#ae81ff\">0<\/span>; <span style=\"color:#a6e22e\">i<\/span> &lt; <span style=\"color:#ae81ff\">50<\/span>; <span style=\"color:#a6e22e\">i<\/span><span style=\"color:#f92672\">++<\/span> {\n<\/span><\/span><span style=\"display:flex;\"><span>                <span style=\"color:#a6e22e\">wg<\/span>.<span style=\"color:#a6e22e\">Add<\/span>(<span style=\"color:#ae81ff\">1<\/span>)\n<\/span><\/span><span style=\"display:flex;\"><span>                <span style=\"color:#66d9ef\">go<\/span> <span style=\"color:#66d9ef\">func<\/span>() {\n<\/span><\/span><span style=\"display:flex;\"><span>\t\t\t<span style=\"color:#66d9ef\">defer<\/span> <span style=\"color:#a6e22e\">wg<\/span>.<span style=\"color:#a6e22e\">Done<\/span>()\n<\/span><\/span><span style=\"display:flex;\"><span>                        <span style=\"color:#a6e22e\">query<\/span>()\n<\/span><\/span><span style=\"display:flex;\"><span>                }()\n<\/span><\/span><span style=\"display:flex;\"><span>        }\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span>        <span style=\"color:#a6e22e\">wg<\/span>.<span style=\"color:#a6e22e\">Wait<\/span>() <span style=\"color:#75715e\">\/\/ Past this point, all routines are executed.\n<\/span><\/span><\/span><span style=\"display:flex;\"><span><span style=\"color:#75715e\"><\/span>}\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span><span style=\"color:#66d9ef\">func<\/span> <span style=\"color:#a6e22e\">query<\/span>() {\n<\/span><\/span><span style=\"display:flex;\"><span>\t<span style=\"color:#f92672\">...<\/span> <span style=\"color:#a6e22e\">query<\/span> <span style=\"color:#a6e22e\">the<\/span> <span style=\"color:#a6e22e\">database<\/span> <span style=\"color:#f92672\">...<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span>}\n<\/span><\/span><\/code><\/pre><\/div><p>In this example, we start 50 goroutines which will be executed as fast as possible.<\/p>\n<p>The <code>wg.Wait()<\/code> ensures that past this point, all routines are ended. More precisely: all routines have called <code>Done()<\/code>. Even more precisely: <code>Done()<\/code> has been called the same amount of time as <code>Add(1)<\/code>.<\/p>\n<p>But sometimes, even if we want to execute things as fast as possible, we don&rsquo;t necessarily want to overload everything.<\/p>\n<h1 id=\"introducing-sizedwaitgroup\">Introducing SizedWaitGroup<\/h1>\n<p><code>SizedWaitGroup<\/code> is quite similar to <code>WaitGroup<\/code> but adds the principle of throttling: you can specify the <strong>maximum<\/strong> amount of routines to spawn concurrently.<\/p>\n<p>A typical use-case would be to execute as fast as possible a set of queries but without overloading the called database. Example:<\/p>\n<div class=\"highlight\"><pre tabindex=\"0\" style=\"color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;\"><code class=\"language-go\" data-lang=\"go\"><span style=\"display:flex;\"><span><span style=\"color:#f92672\">package<\/span> <span style=\"color:#a6e22e\">main<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span><span style=\"color:#f92672\">import<\/span> (\n<\/span><\/span><span style=\"display:flex;\"><span>        <span style=\"color:#e6db74\">&#34;fmt&#34;<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span>        <span style=\"color:#e6db74\">&#34;math\/rand&#34;<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span>        <span style=\"color:#e6db74\">&#34;time&#34;<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span>\t<span style=\"color:#e6db74\">&#34;github.com\/remeh\/sizedwaitgroup&#34;<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span>)\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span><span style=\"color:#66d9ef\">func<\/span> <span style=\"color:#a6e22e\">main<\/span>() {\n<\/span><\/span><span style=\"display:flex;\"><span>        <span style=\"color:#a6e22e\">rand<\/span>.<span style=\"color:#a6e22e\">Seed<\/span>(<span style=\"color:#a6e22e\">time<\/span>.<span style=\"color:#a6e22e\">Now<\/span>().<span style=\"color:#a6e22e\">UnixNano<\/span>())\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span>        <span style=\"color:#75715e\">\/\/ Typical use-case:\n<\/span><\/span><\/span><span style=\"display:flex;\"><span><span style=\"color:#75715e\"><\/span>        <span style=\"color:#75715e\">\/\/ 50 queries must be executed as quick as possible\n<\/span><\/span><\/span><span style=\"display:flex;\"><span><span style=\"color:#75715e\"><\/span>        <span style=\"color:#75715e\">\/\/ but without overloading the database, so only\n<\/span><\/span><\/span><span style=\"display:flex;\"><span><span style=\"color:#75715e\"><\/span>        <span style=\"color:#75715e\">\/\/ 8 routines should be started concurrently.\n<\/span><\/span><\/span><span style=\"display:flex;\"><span><span style=\"color:#75715e\"><\/span>        <span style=\"color:#a6e22e\">swg<\/span> <span style=\"color:#f92672\">:=<\/span> <span style=\"color:#a6e22e\">sizedwaitgroup<\/span>.<span style=\"color:#a6e22e\">New<\/span>(<span style=\"color:#ae81ff\">8<\/span>)\n<\/span><\/span><span style=\"display:flex;\"><span>        <span style=\"color:#66d9ef\">for<\/span> <span style=\"color:#a6e22e\">i<\/span> <span style=\"color:#f92672\">:=<\/span> <span style=\"color:#ae81ff\">0<\/span>; <span style=\"color:#a6e22e\">i<\/span> &lt; <span style=\"color:#ae81ff\">50<\/span>; <span style=\"color:#a6e22e\">i<\/span><span style=\"color:#f92672\">++<\/span> {\n<\/span><\/span><span style=\"display:flex;\"><span>                <span style=\"color:#a6e22e\">swg<\/span>.<span style=\"color:#a6e22e\">Add<\/span>()\n<\/span><\/span><span style=\"display:flex;\"><span>                <span style=\"color:#66d9ef\">go<\/span> <span style=\"color:#66d9ef\">func<\/span>() {\n<\/span><\/span><span style=\"display:flex;\"><span>\t\t\t<span style=\"color:#66d9ef\">defer<\/span> <span style=\"color:#a6e22e\">swg<\/span>.<span style=\"color:#a6e22e\">Done<\/span>()\n<\/span><\/span><span style=\"display:flex;\"><span>                        <span style=\"color:#a6e22e\">query<\/span>()\n<\/span><\/span><span style=\"display:flex;\"><span>                }()\n<\/span><\/span><span style=\"display:flex;\"><span>        }\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span>        <span style=\"color:#a6e22e\">swg<\/span>.<span style=\"color:#a6e22e\">Wait<\/span>()\n<\/span><\/span><span style=\"display:flex;\"><span>}\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span><span style=\"color:#66d9ef\">func<\/span> <span style=\"color:#a6e22e\">query<\/span>() {\n<\/span><\/span><span style=\"display:flex;\"><span>\t<span style=\"color:#f92672\">...<\/span> <span style=\"color:#a6e22e\">query<\/span> <span style=\"color:#a6e22e\">the<\/span> <span style=\"color:#a6e22e\">database<\/span> <span style=\"color:#f92672\">...<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span>}\n<\/span><\/span><\/code><\/pre><\/div><p>In this example, we can see that the use of the <code>SizedWaitGroup<\/code> is quite similar to <code>WaitGroup<\/code> but there is two differences:<\/p>\n<ul>\n<li>You provide a <code>limit<\/code> when creating the <code>SizedWaitGroup<\/code>: in the example, only 8 routines can be spawned concurrently. In order to let start a new routine after having already started 8, one of those 8 should call <code>Done()<\/code>.<\/li>\n<li><code>Add()<\/code> doesn&rsquo;t take any argument on how much routines you want to count in the <code>WaitGroup<\/code>: this difference is because in a <code>SizedWaitGroup<\/code>, a call to <code>Add()<\/code> can be blocking!<\/li>\n<\/ul>\n<h1 id=\"conclusion\">Conclusion<\/h1>\n<p>Most of the time, this kind of tool is not needed: even for a database, receiving 100 queries \u00ab simultaneously \u00bb may not be a problem. But it doesn&rsquo;t mean that you want to use the full set of available connections of a database and this is another use-case where the <code>SizedWaitGroup<\/code> would reveal itself useful.<\/p>\n<p><a href=\"https:\/\/github.com\/remeh\/sizedwaitgroup\">Sources are available here<\/a> and feedbacks are welcome!<\/p>\n"},{"title":"How I've created an intruder detector - Part 3","link":"https:\/\/remy.io\/blog\/how-ive-created-an-intruder-detector-part-3\/","pubDate":"Sun, 13 Nov 2016 18:53:44 +0100","author":"contact@remy.io (R\u00e9my Mathieu)","guid":"https:\/\/remy.io\/blog\/how-ive-created-an-intruder-detector-part-3\/","description":"<h1 id=\"introduction\">Introduction<\/h1>\n<p>This article is the Part 3 of the series <code>How I've created an intruder detector<\/code>. For new readers, here&rsquo;s a link to the <a href=\"https:\/\/remy.io\/articles\/how-ive-created-an-intruder-detector-part-1\">Part 1 - Get the picture<\/a> where we discover how to manually retrieve the image from a D-Link DCS 932L and to the <a href=\"https:\/\/remy.io\/articles\/how-ive-created-an-intruder-detector-part-2\">Part 2 - Motion Detection<\/a>, where we store pictures if motion is detected.<\/p>\n<h1 id=\"part-3---notification\">Part 3 - Notification<\/h1>\n<p>At this point, I have an application writing files on disk when there is motion detected through the camera. What I can already do is regularly watch the output directory to check whether something new appears but this is not really what we would expect for \u00ab an intruder detector \u00bb system.<\/p>\n<p>What I really want is to receive notifications on my mobile phone in real-time when the camera sees something and to have access to the actual picture. In order to send push notifications, there are many solutions:<\/p>\n<ul>\n<li>Create my own Android or iOS application and send real push notifications to the app: doable but longer than other options,<\/li>\n<li>Pushing the images to a private Instagram or Twitter account and subscribe with my real account to receive notifications or,<\/li>\n<li>Use applications\/services created to do this such as <a href=\"https:\/\/www.pushbullet.com\/\">Pushbullet<\/a> or <a href=\"https:\/\/pushover.net\/\">Pushover<\/a>.<\/li>\n<\/ul>\n<p>In favor of validating my proof-of-concept first, I&rsquo;ll go with the option <a href=\"https:\/\/pushover.net\">Pushover<\/a>. Actually, I&rsquo;ve first implemented a version using <a href=\"https:\/\/www.justyo.co\/\">Yo<\/a> but the API was quite slow and I chose to switch to something else.<\/p>\n<p>The next steps are straight-forward:<\/p>\n<ul>\n<li>Serve the image on one unique route per picture (hopefully not discoverable from the outside),<\/li>\n<li>Send a push notification containing a link to the picture.<\/li>\n<\/ul>\n<p>A hand-drawn schema of what will happen in case of motion detection:<\/p>\n<p><img src=\"https:\/\/remy.io\/img\/articles\/cam-part-3\/schema_small.png\" alt=\"Architecture schema\"><\/p>\n<p>Probably seems a bit complicated but:<\/p>\n<ul>\n<li><strong>1<\/strong> \u2014 is already implemented.<\/li>\n<li><strong>2<\/strong> \u2014 will be a simple HTTP call using the <code>net\/http<\/code> Golang package.<\/li>\n<li><strong>3<\/strong> \u2014 this is the service offered by Pushover, not much to do here.<\/li>\n<li><strong>4<\/strong> \u2014 will also be really simple to implement thanks to Golang.<\/li>\n<\/ul>\n<p>What is also great in this architecture is that the D-Link DCS 932L camera doesn&rsquo;t need to be connected to the Internet, it only needs to be connected  to my private network.<\/p>\n<h2 id=\"serve-the-image\">Serve the image<\/h2>\n<p>When motion is suspected, I currently do only one thing: storing the picture on disk. What I want to add is a simple HTTP server providing the picture on a route not discoverable by someone else.<\/p>\n<p>Let&rsquo;s again start with a naive implementation: we will map a unique ID to the filepath of a stored picture, thus, a controller will be able to serve the content of a picture by its ID only if the given ID exists.<\/p>\n<p>This code can be found in <a href=\"https:\/\/github.com\/remeh\/mehcam\/blob\/master\/serve.go\">serve.go<\/a>:<\/p>\n<div class=\"highlight\"><pre tabindex=\"0\" style=\"color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;\"><code class=\"language-go\" data-lang=\"go\"><span style=\"display:flex;\"><span><span style=\"color:#75715e\">\/\/ mapping from a unique ID to the actual filepath on the disk.\n<\/span><\/span><\/span><span style=\"display:flex;\"><span><span style=\"color:#75715e\"><\/span><span style=\"color:#66d9ef\">var<\/span> <span style=\"color:#a6e22e\">mapping<\/span> <span style=\"color:#66d9ef\">map<\/span>[<span style=\"color:#66d9ef\">string<\/span>]<span style=\"color:#66d9ef\">string<\/span> = make(<span style=\"color:#66d9ef\">map<\/span>[<span style=\"color:#66d9ef\">string<\/span>]<span style=\"color:#66d9ef\">string<\/span>)\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span><span style=\"color:#75715e\">\/\/ addPic adds a filename indexed by a generated\n<\/span><\/span><\/span><span style=\"display:flex;\"><span><span style=\"color:#75715e\">\/\/ unique ID to the mapping map.\n<\/span><\/span><\/span><span style=\"display:flex;\"><span><span style=\"color:#75715e\"><\/span><span style=\"color:#66d9ef\">func<\/span> <span style=\"color:#a6e22e\">addPic<\/span>(<span style=\"color:#a6e22e\">filename<\/span> <span style=\"color:#66d9ef\">string<\/span>) <span style=\"color:#66d9ef\">string<\/span> {\n<\/span><\/span><span style=\"display:flex;\"><span>\t<span style=\"color:#75715e\">\/\/ generate a random ID\n<\/span><\/span><\/span><span style=\"display:flex;\"><span><span style=\"color:#75715e\"><\/span>\t<span style=\"color:#a6e22e\">id<\/span> <span style=\"color:#f92672\">:=<\/span> make([]<span style=\"color:#66d9ef\">byte<\/span>, <span style=\"color:#ae81ff\">64<\/span>)\n<\/span><\/span><span style=\"display:flex;\"><span>\t<span style=\"color:#66d9ef\">for<\/span> <span style=\"color:#a6e22e\">i<\/span> <span style=\"color:#f92672\">:=<\/span> <span style=\"color:#66d9ef\">range<\/span> <span style=\"color:#a6e22e\">id<\/span> {\n<\/span><\/span><span style=\"display:flex;\"><span>\t\t<span style=\"color:#a6e22e\">id<\/span>[<span style=\"color:#a6e22e\">i<\/span>] = <span style=\"color:#a6e22e\">idsChar<\/span>[<span style=\"color:#a6e22e\">rand<\/span>.<span style=\"color:#a6e22e\">Intn<\/span>(len(<span style=\"color:#a6e22e\">idsChar<\/span>))]\n<\/span><\/span><span style=\"display:flex;\"><span>\t}\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span>\t<span style=\"color:#75715e\">\/\/ we&#39;ll now serve this file using this newly generated ID\n<\/span><\/span><\/span><span style=\"display:flex;\"><span><span style=\"color:#75715e\"><\/span>\t<span style=\"color:#a6e22e\">sid<\/span> <span style=\"color:#f92672\">:=<\/span> string(<span style=\"color:#a6e22e\">id<\/span>)\n<\/span><\/span><span style=\"display:flex;\"><span>\t<span style=\"color:#a6e22e\">mapping<\/span>[<span style=\"color:#a6e22e\">sid<\/span>] = <span style=\"color:#a6e22e\">filename<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span>\t<span style=\"color:#66d9ef\">return<\/span> <span style=\"color:#a6e22e\">sid<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span>}\n<\/span><\/span><\/code><\/pre><\/div><p>In this part of the code we can see the generation of a unique ID for the given picture filename.\nThis ID will later be used to access the picture from the link in the push notification, so, we store it in a map pointing to the actual filepath of the picture. This way, we can resolve this ID to a filepath in this controller serving the picture:<\/p>\n<div class=\"highlight\"><pre tabindex=\"0\" style=\"color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;\"><code class=\"language-go\" data-lang=\"go\"><span style=\"display:flex;\"><span><span style=\"color:#75715e\">\/\/ picHandler resolves the given file ID using the map mapping\n<\/span><\/span><\/span><span style=\"display:flex;\"><span><span style=\"color:#75715e\">\/\/ to read the actual image from fs and to serve it.\n<\/span><\/span><\/span><span style=\"display:flex;\"><span><span style=\"color:#75715e\"><\/span><span style=\"color:#66d9ef\">type<\/span> <span style=\"color:#a6e22e\">picHandler<\/span> <span style=\"color:#66d9ef\">struct<\/span>{}\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span><span style=\"color:#66d9ef\">func<\/span> (<span style=\"color:#a6e22e\">p<\/span> <span style=\"color:#a6e22e\">picHandler<\/span>) <span style=\"color:#a6e22e\">ServeHTTP<\/span>(<span style=\"color:#a6e22e\">w<\/span> <span style=\"color:#a6e22e\">http<\/span>.<span style=\"color:#a6e22e\">ResponseWriter<\/span>, <span style=\"color:#a6e22e\">r<\/span> <span style=\"color:#f92672\">*<\/span><span style=\"color:#a6e22e\">http<\/span>.<span style=\"color:#a6e22e\">Request<\/span>) {\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span>\t<span style=\"color:#a6e22e\">r<\/span>.<span style=\"color:#a6e22e\">ParseForm<\/span>()\n<\/span><\/span><span style=\"display:flex;\"><span>\t<span style=\"color:#a6e22e\">filename<\/span> <span style=\"color:#f92672\">:=<\/span> <span style=\"color:#a6e22e\">r<\/span>.<span style=\"color:#a6e22e\">Form<\/span>.<span style=\"color:#a6e22e\">Get<\/span>(<span style=\"color:#e6db74\">&#34;f&#34;<\/span>)\n<\/span><\/span><span style=\"display:flex;\"><span>\t<span style=\"color:#a6e22e\">filepath<\/span>, <span style=\"color:#a6e22e\">exists<\/span> <span style=\"color:#f92672\">:=<\/span> <span style=\"color:#a6e22e\">mapping<\/span>[<span style=\"color:#a6e22e\">filename<\/span>]\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span>\t<span style=\"color:#75715e\">\/\/ unknown image\n<\/span><\/span><\/span><span style=\"display:flex;\"><span><span style=\"color:#75715e\"><\/span>\t<span style=\"color:#66d9ef\">if<\/span> !<span style=\"color:#a6e22e\">exists<\/span> {\n<\/span><\/span><span style=\"display:flex;\"><span>\t\t<span style=\"color:#a6e22e\">w<\/span>.<span style=\"color:#a6e22e\">WriteHeader<\/span>(<span style=\"color:#ae81ff\">404<\/span>)\n<\/span><\/span><span style=\"display:flex;\"><span>\t\t<span style=\"color:#66d9ef\">return<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span>\t}\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span>\t<span style=\"color:#75715e\">\/\/ read the file on disk\n<\/span><\/span><\/span><span style=\"display:flex;\"><span><span style=\"color:#75715e\"><\/span>\t<span style=\"color:#a6e22e\">data<\/span>, <span style=\"color:#a6e22e\">err<\/span> <span style=\"color:#f92672\">:=<\/span> <span style=\"color:#a6e22e\">ioutil<\/span>.<span style=\"color:#a6e22e\">ReadFile<\/span>(<span style=\"color:#a6e22e\">filepath<\/span>)\n<\/span><\/span><span style=\"display:flex;\"><span>\t<span style=\"color:#66d9ef\">if<\/span> <span style=\"color:#a6e22e\">err<\/span> <span style=\"color:#f92672\">!=<\/span> <span style=\"color:#66d9ef\">nil<\/span> {\n<\/span><\/span><span style=\"display:flex;\"><span>\t\t<span style=\"color:#a6e22e\">log<\/span>.<span style=\"color:#a6e22e\">Printf<\/span>(<span style=\"color:#e6db74\">&#34;error: while serving file %s during read: %v&#34;<\/span>, <span style=\"color:#a6e22e\">err<\/span>)\n<\/span><\/span><span style=\"display:flex;\"><span>\t\t<span style=\"color:#a6e22e\">w<\/span>.<span style=\"color:#a6e22e\">WriteHeader<\/span>(<span style=\"color:#ae81ff\">404<\/span>)\n<\/span><\/span><span style=\"display:flex;\"><span>\t\t<span style=\"color:#66d9ef\">return<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span>\t}\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span>\t<span style=\"color:#75715e\">\/\/ render the file\n<\/span><\/span><\/span><span style=\"display:flex;\"><span><span style=\"color:#75715e\"><\/span>\t<span style=\"color:#a6e22e\">w<\/span>.<span style=\"color:#a6e22e\">Header<\/span>().<span style=\"color:#a6e22e\">Set<\/span>(<span style=\"color:#e6db74\">&#34;Content-Type&#34;<\/span>, <span style=\"color:#a6e22e\">http<\/span>.<span style=\"color:#a6e22e\">DetectContentType<\/span>(<span style=\"color:#a6e22e\">data<\/span>))\n<\/span><\/span><span style=\"display:flex;\"><span>\t<span style=\"color:#a6e22e\">w<\/span>.<span style=\"color:#a6e22e\">Write<\/span>(<span style=\"color:#a6e22e\">data<\/span>)\n<\/span><\/span><span style=\"display:flex;\"><span>}\n<\/span><\/span><\/code><\/pre><\/div><p>First the controller checks that the ID really exists in RAM in <code>mapping<\/code>. When it&rsquo;s the case, the second part of the controller reads this file and writes it in the HTTP response.<\/p>\n<p>I&rsquo;ve decided to keep this map of IDs in RAM only: it won&rsquo;t survive a reboot of the server but it won&rsquo;t be critical at the moment. Now that we can serve the pictures, let&rsquo;s send a push notification.<\/p>\n<h2 id=\"push-notification\">Push notification<\/h2>\n<p><a href=\"https:\/\/pushover.net\">Pushover<\/a> is a notification service: it offers applications (iOS, Android, Chrome, &hellip;) to receive the notifications and a REST API to send push notifications. Only thing you have to create is a consumer of their API to send custom push notifications.\nApps they provide are free for 7 days and then cost $4.99. They offer 7,500 push\/month per consumer of the API.<\/p>\n<p>It will be more than enough for my proof-of-concept and after having registered and created a consumer to get an API token, I was ready to use their API.<\/p>\n<p>Changes in code to send the notifications were quite easy: after the storage of the picture on the filesystem, I call the <code>send<\/code> method in a goroutine. The <code>send<\/code> method calls the Pushover API through an HTTP POST.<\/p>\n<p>This part of the code can be found in <a href=\"https:\/\/github.com\/remeh\/mehcam\/blob\/master\/notif.go\">notif.go<\/a>:<\/p>\n<div class=\"highlight\"><pre tabindex=\"0\" style=\"color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;\"><code class=\"language-go\" data-lang=\"go\"><span style=\"display:flex;\"><span><span style=\"color:#75715e\">\/\/ send uses the Pushover API to send a push notification to my phone\n<\/span><\/span><\/span><span style=\"display:flex;\"><span><span style=\"color:#75715e\">\/\/ with a link to serve the image using its unique ID.\n<\/span><\/span><\/span><span style=\"display:flex;\"><span><span style=\"color:#75715e\"><\/span><span style=\"color:#66d9ef\">func<\/span> <span style=\"color:#a6e22e\">send<\/span>(<span style=\"color:#a6e22e\">t<\/span> <span style=\"color:#a6e22e\">time<\/span>.<span style=\"color:#a6e22e\">Time<\/span>, <span style=\"color:#a6e22e\">id<\/span> <span style=\"color:#66d9ef\">string<\/span>) {\n<\/span><\/span><span style=\"display:flex;\"><span>\t<span style=\"color:#66d9ef\">if<\/span> len(<span style=\"color:#a6e22e\">id<\/span>) <span style=\"color:#f92672\">==<\/span> <span style=\"color:#ae81ff\">0<\/span> {\n<\/span><\/span><span style=\"display:flex;\"><span>\t\t<span style=\"color:#66d9ef\">return<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span>\t}\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span>\t<span style=\"color:#a6e22e\">values<\/span> <span style=\"color:#f92672\">:=<\/span> <span style=\"color:#a6e22e\">url<\/span>.<span style=\"color:#a6e22e\">Values<\/span>{}\n<\/span><\/span><span style=\"display:flex;\"><span>\t<span style=\"color:#a6e22e\">values<\/span>.<span style=\"color:#a6e22e\">Add<\/span>(<span style=\"color:#e6db74\">&#34;token&#34;<\/span>, <span style=\"color:#a6e22e\">config<\/span>.<span style=\"color:#a6e22e\">PoToken<\/span>)\n<\/span><\/span><span style=\"display:flex;\"><span>\t<span style=\"color:#a6e22e\">values<\/span>.<span style=\"color:#a6e22e\">Add<\/span>(<span style=\"color:#e6db74\">&#34;user&#34;<\/span>, <span style=\"color:#a6e22e\">config<\/span>.<span style=\"color:#a6e22e\">PoUser<\/span>)\n<\/span><\/span><span style=\"display:flex;\"><span>\t<span style=\"color:#a6e22e\">values<\/span>.<span style=\"color:#a6e22e\">Add<\/span>(<span style=\"color:#e6db74\">&#34;title&#34;<\/span>, <span style=\"color:#e6db74\">&#34;Motion detection&#34;<\/span>)\n<\/span><\/span><span style=\"display:flex;\"><span>\t<span style=\"color:#a6e22e\">values<\/span>.<span style=\"color:#a6e22e\">Add<\/span>(<span style=\"color:#e6db74\">&#34;message&#34;<\/span>, <span style=\"color:#a6e22e\">fmt<\/span>.<span style=\"color:#a6e22e\">Sprintf<\/span>(<span style=\"color:#e6db74\">&#34;Motion detected at %s&#34;<\/span>, <span style=\"color:#a6e22e\">t<\/span>.<span style=\"color:#a6e22e\">Format<\/span>(<span style=\"color:#a6e22e\">time<\/span>.<span style=\"color:#a6e22e\">ANSIC<\/span>)))\n<\/span><\/span><span style=\"display:flex;\"><span>\t<span style=\"color:#a6e22e\">values<\/span>.<span style=\"color:#a6e22e\">Add<\/span>(<span style=\"color:#e6db74\">&#34;url&#34;<\/span>, <span style=\"color:#a6e22e\">config<\/span>.<span style=\"color:#a6e22e\">BaseLink<\/span><span style=\"color:#f92672\">+<\/span><span style=\"color:#e6db74\">&#34;?f=&#34;<\/span><span style=\"color:#f92672\">+<\/span><span style=\"color:#a6e22e\">id<\/span>)\n<\/span><\/span><span style=\"display:flex;\"><span>\t<span style=\"color:#a6e22e\">values<\/span>.<span style=\"color:#a6e22e\">Add<\/span>(<span style=\"color:#e6db74\">&#34;url_title&#34;<\/span>, <span style=\"color:#e6db74\">&#34;Open picture&#34;<\/span>)\n<\/span><\/span><span style=\"display:flex;\"><span>\t<span style=\"color:#a6e22e\">values<\/span>.<span style=\"color:#a6e22e\">Add<\/span>(<span style=\"color:#e6db74\">&#34;sound&#34;<\/span>, <span style=\"color:#e6db74\">&#34;gamelan&#34;<\/span>)\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span>\t<span style=\"color:#a6e22e\">resp<\/span>, <span style=\"color:#a6e22e\">err<\/span> <span style=\"color:#f92672\">:=<\/span> <span style=\"color:#a6e22e\">http<\/span>.<span style=\"color:#a6e22e\">PostForm<\/span>(<span style=\"color:#a6e22e\">PoApiUrl<\/span>, <span style=\"color:#a6e22e\">values<\/span>)\n<\/span><\/span><span style=\"display:flex;\"><span>\t<span style=\"color:#66d9ef\">if<\/span> <span style=\"color:#a6e22e\">err<\/span> <span style=\"color:#f92672\">!=<\/span> <span style=\"color:#66d9ef\">nil<\/span> {\n<\/span><\/span><span style=\"display:flex;\"><span>\t\t<span style=\"color:#a6e22e\">log<\/span>.<span style=\"color:#a6e22e\">Printf<\/span>(<span style=\"color:#e6db74\">&#34;error while sending the notification: %v&#34;<\/span>, <span style=\"color:#a6e22e\">err<\/span>)\n<\/span><\/span><span style=\"display:flex;\"><span>\t\t<span style=\"color:#66d9ef\">return<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span>\t}\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span>\t<span style=\"color:#66d9ef\">if<\/span> <span style=\"color:#a6e22e\">resp<\/span>.<span style=\"color:#a6e22e\">StatusCode<\/span> <span style=\"color:#f92672\">!=<\/span> <span style=\"color:#ae81ff\">200<\/span> {\n<\/span><\/span><span style=\"display:flex;\"><span>\t\t<span style=\"color:#a6e22e\">log<\/span>.<span style=\"color:#a6e22e\">Printf<\/span>(<span style=\"color:#e6db74\">&#34;error while sending the notification: %v&#34;<\/span>, <span style=\"color:#a6e22e\">resp<\/span>.<span style=\"color:#a6e22e\">Status<\/span>)\n<\/span><\/span><span style=\"display:flex;\"><span>\t}\n<\/span><\/span><span style=\"display:flex;\"><span>}\n<\/span><\/span><\/code><\/pre><\/div><p>I&rsquo;ve also changed the configuration file to add settings for the Pushover token and user key. A <a href=\"https:\/\/github.com\/remeh\/mehcam\/blob\/master\/mehcam.conf\">sample of the configuration file<\/a> is available in the repository.<\/p>\n<p>And I&rsquo;m done for the notification part.<\/p>\n<p>This is the result of the push notification on my mobile and the click on the <strong>Open picture<\/strong> button:<\/p>\n<p><img src=\"https:\/\/remy.io\/img\/articles\/cam-part-3\/notif.png\" alt=\"Intrusion detected and sent by push notification to my mobile phone\"><\/p>\n<h1 id=\"conclusion\">Conclusion<\/h1>\n<p>I&rsquo;m quite happy with this naive implementation of my intruder detector, the ratio features\/development time is really good due to available resources and services. To go further, a lot of things could be added:<\/p>\n<ul>\n<li>Limit the motion detection to some hours of the day,<\/li>\n<li>Delete stored pictures after a few days,<\/li>\n<li>Replace the IP camera with a webcam to read the picture directly from the webcam,<\/li>\n<li>Real mobile application having more features (even if we can already provide more features in the served page),<\/li>\n<li>Improve the detection motion,<\/li>\n<li>And so on&hellip;<\/li>\n<\/ul>\n<p>I hope you liked this series where I try to share the fun of learning and creating something new. Feedbacks are welcome, either about the articles or the code and feel free to create issues or to send pull-requests on the <a href=\"https:\/\/github.com\/remeh\/mehcam\">Github page of the project<\/a>. Also, I&rsquo;m available on <a href=\"https:\/\/twitter.com\/remeh\">Twitter<\/a> so don&rsquo;t hesitate to contact me!<\/p>\n"},{"title":"How I've created an intruder detector - Part 2","link":"https:\/\/remy.io\/blog\/how-ive-created-an-intruder-detector-part-2\/","pubDate":"Mon, 07 Nov 2016 22:53:44 +0100","author":"contact@remy.io (R\u00e9my Mathieu)","guid":"https:\/\/remy.io\/blog\/how-ive-created-an-intruder-detector-part-2\/","description":"<h1 id=\"introduction\">Introduction<\/h1>\n<p>This article is the Part 2 of the series <code>How I've created an intruder detector<\/code>. For new readers, here&rsquo;s a link to the <a href=\"https:\/\/remy.io\/articles\/how-ive-created-an-intruder-detector-part-1\">Part 1 - Get the picture<\/a> where we discover how to manually retrieve the image from a D-Link DCS 932L.<\/p>\n<h1 id=\"part-2---motion-detection\">Part 2 - Motion detection<\/h1>\n<p>Now that we&rsquo;ve access to what the camera sees, we can start the motion detection work.<\/p>\n<p>In order to create a simple solution, I&rsquo;ve first wanted to simply detect when an image is different from the previous one and to know how much delta there is between these two images. At first, I prefer to have false positives rather than false negatives (too much pictures rather than not enough).\nThe quality of the image provided by this camera is quite bad so I need something for creating a kind of sum representing the image, then, computing the differences must provide me the \u00ab amount \u00bb of changes between two images. Naive, but I like to first test a naive approach and then re-iterate if something more complex is needed.<\/p>\n<p>After some internet lookups, I&rsquo;ve found an existing Golang library providing a function of <a href=\"https:\/\/www.safaribooksonline.com\/blog\/2013\/11\/26\/image-hashing-with-python\/\">Average Hash<\/a> on image and a function of <a href=\"https:\/\/en.wikipedia.org\/wiki\/Hamming_distance\">Hamming Distance<\/a>:<\/p>\n<p><a href=\"https:\/\/github.com\/jteeuwen\/imghash\">https:\/\/github.com\/jteeuwen\/imghash<\/a><\/p>\n<p>It provides other features but I&rsquo;ll import it only to use its <a href=\"https:\/\/github.com\/jteeuwen\/imghash\/blob\/master\/average.go\">Average<\/a> implementation and its <a href=\"https:\/\/github.com\/jteeuwen\/imghash\/blob\/master\/hash.go\">Distance<\/a> one.<\/p>\n<h2 id=\"code\">Code<\/h2>\n<p>Of course, I know this is something doable in many, many languages. There is not a lot to do here and there is no reason I&rsquo;ve chosen Go except I&rsquo;m efficient with it and I already knew all I needed is available in its stdlib.<\/p>\n<p>Conceptually, we&rsquo;ll need to do <strong>four<\/strong> things, which will be executed regularly:<\/p>\n<ul>\n<li>Get the image from the cam<\/li>\n<li>Compute the hash of this image<\/li>\n<li>Compute the distance of this hash to the previous one<\/li>\n<li>Store the image if the distance is big<\/li>\n<\/ul>\n<h2 id=\"get-the-image-and-compute-the-hash\">Get the image and compute the hash<\/h2>\n<p>The first two bullets are realized in the file <a href=\"https:\/\/github.com\/remeh\/mehcam\/blob\/master\/image.go\">image.go<\/a>:<\/p>\n<div class=\"highlight\"><pre tabindex=\"0\" style=\"color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;\"><code class=\"language-go\" data-lang=\"go\"><span style=\"display:flex;\"><span><span style=\"color:#75715e\">\/\/ get queries the camera HTTP server to get the image.\n<\/span><\/span><\/span><span style=\"display:flex;\"><span><span style=\"color:#75715e\">\/\/ It provides authentication through an HTTP\n<\/span><\/span><\/span><span style=\"display:flex;\"><span><span style=\"color:#75715e\">\/\/ header set in the configuration.\n<\/span><\/span><\/span><span style=\"display:flex;\"><span><span style=\"color:#75715e\"><\/span><span style=\"color:#66d9ef\">func<\/span> <span style=\"color:#a6e22e\">get<\/span>() ([]<span style=\"color:#66d9ef\">byte<\/span>, <span style=\"color:#66d9ef\">error<\/span>) {\n<\/span><\/span><span style=\"display:flex;\"><span>\t<span style=\"color:#a6e22e\">req<\/span>, <span style=\"color:#a6e22e\">err<\/span> <span style=\"color:#f92672\">:=<\/span> <span style=\"color:#a6e22e\">http<\/span>.<span style=\"color:#a6e22e\">NewRequest<\/span>(<span style=\"color:#e6db74\">&#34;GET&#34;<\/span>, <span style=\"color:#a6e22e\">config<\/span>.<span style=\"color:#a6e22e\">Url<\/span>, <span style=\"color:#66d9ef\">nil<\/span>)\n<\/span><\/span><span style=\"display:flex;\"><span>\t<span style=\"color:#66d9ef\">if<\/span> <span style=\"color:#a6e22e\">err<\/span> <span style=\"color:#f92672\">!=<\/span> <span style=\"color:#66d9ef\">nil<\/span> {\n<\/span><\/span><span style=\"display:flex;\"><span>\t\t<span style=\"color:#66d9ef\">return<\/span> <span style=\"color:#66d9ef\">nil<\/span>, <span style=\"color:#a6e22e\">err<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span>\t}\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span>\t<span style=\"color:#75715e\">\/\/ Basic Authentication\n<\/span><\/span><\/span><span style=\"display:flex;\"><span><span style=\"color:#75715e\"><\/span>\t<span style=\"color:#a6e22e\">req<\/span>.<span style=\"color:#a6e22e\">SetBasicAuth<\/span>(<span style=\"color:#a6e22e\">config<\/span>.<span style=\"color:#a6e22e\">Login<\/span>, <span style=\"color:#a6e22e\">config<\/span>.<span style=\"color:#a6e22e\">Password<\/span>)\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span>\t<span style=\"color:#66d9ef\">var<\/span> <span style=\"color:#a6e22e\">resp<\/span> <span style=\"color:#f92672\">*<\/span><span style=\"color:#a6e22e\">http<\/span>.<span style=\"color:#a6e22e\">Response<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span>\t<span style=\"color:#66d9ef\">if<\/span> <span style=\"color:#a6e22e\">resp<\/span>, <span style=\"color:#a6e22e\">err<\/span> = <span style=\"color:#a6e22e\">http<\/span>.<span style=\"color:#a6e22e\">DefaultClient<\/span>.<span style=\"color:#a6e22e\">Do<\/span>(<span style=\"color:#a6e22e\">req<\/span>); <span style=\"color:#a6e22e\">err<\/span> <span style=\"color:#f92672\">!=<\/span> <span style=\"color:#66d9ef\">nil<\/span> {\n<\/span><\/span><span style=\"display:flex;\"><span>\t\t<span style=\"color:#66d9ef\">return<\/span> <span style=\"color:#66d9ef\">nil<\/span>, <span style=\"color:#a6e22e\">err<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span>\t}\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span>\t<span style=\"color:#66d9ef\">if<\/span> <span style=\"color:#a6e22e\">resp<\/span>.<span style=\"color:#a6e22e\">StatusCode<\/span> <span style=\"color:#f92672\">!=<\/span> <span style=\"color:#ae81ff\">200<\/span> {\n<\/span><\/span><span style=\"display:flex;\"><span>\t\t<span style=\"color:#66d9ef\">return<\/span> <span style=\"color:#66d9ef\">nil<\/span>, <span style=\"color:#a6e22e\">fmt<\/span>.<span style=\"color:#a6e22e\">Errorf<\/span>(<span style=\"color:#e6db74\">&#34;status: %d&#34;<\/span>, <span style=\"color:#a6e22e\">resp<\/span>.<span style=\"color:#a6e22e\">StatusCode<\/span>)\n<\/span><\/span><span style=\"display:flex;\"><span>\t}\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span>\t<span style=\"color:#66d9ef\">defer<\/span> <span style=\"color:#a6e22e\">resp<\/span>.<span style=\"color:#a6e22e\">Body<\/span>.<span style=\"color:#a6e22e\">Close<\/span>()\n<\/span><\/span><span style=\"display:flex;\"><span>\t<span style=\"color:#66d9ef\">return<\/span> <span style=\"color:#a6e22e\">ioutil<\/span>.<span style=\"color:#a6e22e\">ReadAll<\/span>(<span style=\"color:#a6e22e\">resp<\/span>.<span style=\"color:#a6e22e\">Body<\/span>)\n<\/span><\/span><span style=\"display:flex;\"><span>}\n<\/span><\/span><\/code><\/pre><\/div><p>We create a <code>GET<\/code> request, putting the Authorization header into what we&rsquo;ve previously found using our browser, send the query and finally if everything went well, we return the body (or an error) which would be the image seen by the camera.<\/p>\n<div class=\"highlight\"><pre tabindex=\"0\" style=\"color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;\"><code class=\"language-go\" data-lang=\"go\"><span style=\"display:flex;\"><span><span style=\"color:#75715e\">\/\/ hash hashes the given bytes using the\n<\/span><\/span><\/span><span style=\"display:flex;\"><span><span style=\"color:#75715e\">\/\/ Average Hash method.\n<\/span><\/span><\/span><span style=\"display:flex;\"><span><span style=\"color:#75715e\">\/\/ Errors on unknown file format.\n<\/span><\/span><\/span><span style=\"display:flex;\"><span><span style=\"color:#75715e\"><\/span><span style=\"color:#66d9ef\">func<\/span> <span style=\"color:#a6e22e\">hash<\/span>(<span style=\"color:#a6e22e\">data<\/span> []<span style=\"color:#66d9ef\">byte<\/span>) (<span style=\"color:#66d9ef\">uint64<\/span>, <span style=\"color:#66d9ef\">error<\/span>) {\n<\/span><\/span><span style=\"display:flex;\"><span>\t<span style=\"color:#a6e22e\">buff<\/span> <span style=\"color:#f92672\">:=<\/span> <span style=\"color:#a6e22e\">bytes<\/span>.<span style=\"color:#a6e22e\">NewBuffer<\/span>(<span style=\"color:#a6e22e\">data<\/span>)\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span>\t<span style=\"color:#a6e22e\">img<\/span>, <span style=\"color:#a6e22e\">_<\/span>, <span style=\"color:#a6e22e\">err<\/span> <span style=\"color:#f92672\">:=<\/span> <span style=\"color:#a6e22e\">image<\/span>.<span style=\"color:#a6e22e\">Decode<\/span>(<span style=\"color:#a6e22e\">buff<\/span>)\n<\/span><\/span><span style=\"display:flex;\"><span>\t<span style=\"color:#66d9ef\">if<\/span> <span style=\"color:#a6e22e\">err<\/span> <span style=\"color:#f92672\">!=<\/span> <span style=\"color:#66d9ef\">nil<\/span> {\n<\/span><\/span><span style=\"display:flex;\"><span>\t\t<span style=\"color:#66d9ef\">return<\/span> <span style=\"color:#ae81ff\">0<\/span>, <span style=\"color:#a6e22e\">err<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span>\t}\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span>\t<span style=\"color:#66d9ef\">return<\/span> <span style=\"color:#a6e22e\">imghash<\/span>.<span style=\"color:#a6e22e\">Average<\/span>(<span style=\"color:#a6e22e\">img<\/span>), <span style=\"color:#66d9ef\">nil<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span>}\n<\/span><\/span><\/code><\/pre><\/div><p>Here we create a Golang image object (because it&rsquo;s the type wanted by the <code>imghash.Average<\/code> method) and we return the hash.<\/p>\n<h2 id=\"compute-the-distance-of-two-pictures-and-store-the-image\">Compute the distance of two pictures and store the image<\/h2>\n<p>The distance and the picture storage will be done in the main loop of the application, you can find it in the <a href=\"https:\/\/github.com\/remeh\/mehcam\/blob\/master\/main.go\">main.go<\/a> file:<\/p>\n<div class=\"highlight\"><pre tabindex=\"0\" style=\"color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;\"><code class=\"language-go\" data-lang=\"go\"><span style=\"display:flex;\"><span><span style=\"color:#66d9ef\">var<\/span> <span style=\"color:#a6e22e\">lastHash<\/span>, <span style=\"color:#a6e22e\">currHash<\/span> <span style=\"color:#66d9ef\">uint64<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span><span style=\"color:#66d9ef\">var<\/span> <span style=\"color:#a6e22e\">lastImg<\/span>, <span style=\"color:#a6e22e\">currImg<\/span> []<span style=\"color:#66d9ef\">byte<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span><span style=\"color:#66d9ef\">var<\/span> <span style=\"color:#a6e22e\">err<\/span> <span style=\"color:#66d9ef\">error<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span><span style=\"color:#a6e22e\">duration<\/span>, <span style=\"color:#a6e22e\">_<\/span> <span style=\"color:#f92672\">:=<\/span> <span style=\"color:#a6e22e\">time<\/span>.<span style=\"color:#a6e22e\">ParseDuration<\/span>(<span style=\"color:#a6e22e\">fmt<\/span>.<span style=\"color:#a6e22e\">Sprintf<\/span>(<span style=\"color:#e6db74\">&#34;%ds&#34;<\/span>, <span style=\"color:#a6e22e\">config<\/span>.<span style=\"color:#a6e22e\">Frequency<\/span>))\n<\/span><\/span><span style=\"display:flex;\"><span><span style=\"color:#a6e22e\">ticker<\/span> <span style=\"color:#f92672\">:=<\/span> <span style=\"color:#a6e22e\">time<\/span>.<span style=\"color:#a6e22e\">NewTicker<\/span>(<span style=\"color:#a6e22e\">duration<\/span>)\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span><span style=\"color:#75715e\">\/\/ timed infinite loop\n<\/span><\/span><\/span><span style=\"display:flex;\"><span><span style=\"color:#75715e\"><\/span><span style=\"color:#66d9ef\">for<\/span> <span style=\"color:#a6e22e\">t<\/span> <span style=\"color:#f92672\">:=<\/span> <span style=\"color:#66d9ef\">range<\/span> <span style=\"color:#a6e22e\">ticker<\/span>.<span style=\"color:#a6e22e\">C<\/span> {\n<\/span><\/span><span style=\"display:flex;\"><span><span style=\"color:#75715e\">\/\/ retrieve the current img and its hash\n<\/span><\/span><\/span><span style=\"display:flex;\"><span><span style=\"color:#75715e\"><\/span>\t<span style=\"color:#a6e22e\">currHash<\/span>, <span style=\"color:#a6e22e\">currImg<\/span>, <span style=\"color:#a6e22e\">err<\/span> = <span style=\"color:#a6e22e\">current<\/span>()\n<\/span><\/span><span style=\"display:flex;\"><span>\t<span style=\"color:#66d9ef\">if<\/span> <span style=\"color:#a6e22e\">err<\/span> <span style=\"color:#f92672\">!=<\/span> <span style=\"color:#66d9ef\">nil<\/span> {\n<\/span><\/span><span style=\"display:flex;\"><span>\t\t<span style=\"color:#a6e22e\">fmt<\/span>.<span style=\"color:#a6e22e\">Println<\/span>(<span style=\"color:#e6db74\">&#34;can&#39;t retrieve hash:&#34;<\/span>, <span style=\"color:#a6e22e\">err<\/span>)\n<\/span><\/span><span style=\"display:flex;\"><span>\t}\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span>\t<span style=\"color:#75715e\">\/\/ compute the distance between previous image and the current one\n<\/span><\/span><\/span><span style=\"display:flex;\"><span><span style=\"color:#75715e\"><\/span>\t<span style=\"color:#a6e22e\">dist<\/span> <span style=\"color:#f92672\">:=<\/span> <span style=\"color:#a6e22e\">imghash<\/span>.<span style=\"color:#a6e22e\">Distance<\/span>(<span style=\"color:#a6e22e\">lastHash<\/span>, <span style=\"color:#a6e22e\">currHash<\/span>)\n<\/span><\/span><span style=\"display:flex;\"><span>\t<span style=\"color:#66d9ef\">if<\/span> <span style=\"color:#a6e22e\">dist<\/span> &gt; <span style=\"color:#a6e22e\">config<\/span>.<span style=\"color:#a6e22e\">Dist<\/span> <span style=\"color:#f92672\">&amp;&amp;<\/span> <span style=\"color:#a6e22e\">lastImg<\/span> <span style=\"color:#f92672\">!=<\/span> <span style=\"color:#66d9ef\">nil<\/span> {\n<\/span><\/span><span style=\"display:flex;\"><span>\t\t<span style=\"color:#a6e22e\">fmt<\/span>.<span style=\"color:#a6e22e\">Println<\/span>(<span style=\"color:#a6e22e\">time<\/span>.<span style=\"color:#a6e22e\">Now<\/span>(), <span style=\"color:#e6db74\">&#34;detected a distance:&#34;<\/span>, <span style=\"color:#a6e22e\">dist<\/span>)\n<\/span><\/span><span style=\"display:flex;\"><span>\t\t<span style=\"color:#66d9ef\">if<\/span> <span style=\"color:#a6e22e\">err<\/span> = <span style=\"color:#a6e22e\">ioutil<\/span>.<span style=\"color:#a6e22e\">WriteFile<\/span>(<span style=\"color:#a6e22e\">config<\/span>.<span style=\"color:#a6e22e\">Out<\/span><span style=\"color:#f92672\">+<\/span><span style=\"color:#a6e22e\">filename<\/span>(<span style=\"color:#a6e22e\">t<\/span>), <span style=\"color:#a6e22e\">currImg<\/span>, <span style=\"color:#ae81ff\">0644<\/span>); <span style=\"color:#a6e22e\">err<\/span> <span style=\"color:#f92672\">!=<\/span> <span style=\"color:#66d9ef\">nil<\/span> {\n<\/span><\/span><span style=\"display:flex;\"><span>\t\t\t<span style=\"color:#a6e22e\">fmt<\/span>.<span style=\"color:#a6e22e\">Println<\/span>(<span style=\"color:#e6db74\">&#34;while writing file:&#34;<\/span>, <span style=\"color:#a6e22e\">err<\/span>)\n<\/span><\/span><span style=\"display:flex;\"><span>\t\t}\n<\/span><\/span><span style=\"display:flex;\"><span>\t}\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span>\t<span style=\"color:#75715e\">\/\/ store for next iteration\n<\/span><\/span><\/span><span style=\"display:flex;\"><span><span style=\"color:#75715e\"><\/span>\t<span style=\"color:#a6e22e\">lastImg<\/span>, <span style=\"color:#a6e22e\">lastHash<\/span> = <span style=\"color:#a6e22e\">currImg<\/span>, <span style=\"color:#a6e22e\">currHash<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span>}\n<\/span><\/span><\/code><\/pre><\/div><p>Golang has an elegant solution to create loop regularly executed using the <code>time.Ticker<\/code> type.<\/p>\n<p>The loop starts by getting the hash and the bytes of the current image (the one currently seen by the camera):<\/p>\n<div class=\"highlight\"><pre tabindex=\"0\" style=\"color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;\"><code class=\"language-go\" data-lang=\"go\"><span style=\"display:flex;\"><span><span style=\"color:#75715e\">\/\/ current snapshots the image currently seen by the cam.\n<\/span><\/span><\/span><span style=\"display:flex;\"><span><span style=\"color:#75715e\">\/\/ It returns the hash and the image (in the original format).\n<\/span><\/span><\/span><span style=\"display:flex;\"><span><span style=\"color:#75715e\"><\/span><span style=\"color:#66d9ef\">func<\/span> <span style=\"color:#a6e22e\">current<\/span>() (<span style=\"color:#66d9ef\">uint64<\/span>, []<span style=\"color:#66d9ef\">byte<\/span>, <span style=\"color:#66d9ef\">error<\/span>) {\n<\/span><\/span><span style=\"display:flex;\"><span>\t<span style=\"color:#a6e22e\">data<\/span>, <span style=\"color:#a6e22e\">err<\/span> <span style=\"color:#f92672\">:=<\/span> <span style=\"color:#a6e22e\">get<\/span>()\n<\/span><\/span><span style=\"display:flex;\"><span>\t<span style=\"color:#66d9ef\">if<\/span> <span style=\"color:#a6e22e\">err<\/span> <span style=\"color:#f92672\">!=<\/span> <span style=\"color:#66d9ef\">nil<\/span> {\n<\/span><\/span><span style=\"display:flex;\"><span>\t\t<span style=\"color:#66d9ef\">return<\/span> <span style=\"color:#ae81ff\">0<\/span>, <span style=\"color:#66d9ef\">nil<\/span>, <span style=\"color:#a6e22e\">fmt<\/span>.<span style=\"color:#a6e22e\">Errorf<\/span>(<span style=\"color:#e6db74\">&#34;error while hashing the image: %s&#34;<\/span>, <span style=\"color:#a6e22e\">err<\/span>.<span style=\"color:#a6e22e\">Error<\/span>())\n<\/span><\/span><span style=\"display:flex;\"><span>\t}\n<\/span><\/span><span style=\"display:flex;\"><span>\n<\/span><\/span><span style=\"display:flex;\"><span>\t<span style=\"color:#66d9ef\">if<\/span> <span style=\"color:#a6e22e\">h<\/span>, <span style=\"color:#a6e22e\">err<\/span> <span style=\"color:#f92672\">:=<\/span> <span style=\"color:#a6e22e\">hash<\/span>(<span style=\"color:#a6e22e\">data<\/span>); <span style=\"color:#a6e22e\">err<\/span> <span style=\"color:#f92672\">!=<\/span> <span style=\"color:#66d9ef\">nil<\/span> {\n<\/span><\/span><span style=\"display:flex;\"><span>\t\t<span style=\"color:#66d9ef\">return<\/span> <span style=\"color:#ae81ff\">0<\/span>, <span style=\"color:#66d9ef\">nil<\/span>, <span style=\"color:#a6e22e\">fmt<\/span>.<span style=\"color:#a6e22e\">Errorf<\/span>(<span style=\"color:#e6db74\">&#34;error while hashing the image: %s&#34;<\/span>, <span style=\"color:#a6e22e\">err<\/span>.<span style=\"color:#a6e22e\">Error<\/span>())\n<\/span><\/span><span style=\"display:flex;\"><span>\t} <span style=\"color:#66d9ef\">else<\/span> {\n<\/span><\/span><span style=\"display:flex;\"><span>\t\t<span style=\"color:#66d9ef\">return<\/span> <span style=\"color:#a6e22e\">h<\/span>, <span style=\"color:#a6e22e\">data<\/span>, <span style=\"color:#a6e22e\">err<\/span>\n<\/span><\/span><span style=\"display:flex;\"><span>\t}\n<\/span><\/span><span style=\"display:flex;\"><span>}\n<\/span><\/span><\/code><\/pre><\/div><p>Then, it uses the <code>imghash.Distance<\/code> method to compute the distance between the previously seen image and the current one. If the distance is considered big, it prints a log and uses the helper method <code>ioutil.WriteFile<\/code> to write the image on disk. Finally it stores the current image and hash as last image and last hash, for the next iteration.<\/p>\n<p>And&hellip; that&rsquo;s it&hellip; ?<\/p>\n<p>I&rsquo;ve got an Intel NUC running on my local network which can access the camera HTTP server. A Raspberry is obviously a perfect match to fulfill this role. I put the camera near the cat food and I let the project run the night after having created it. This is what I found in the morning when I woke up:<\/p>\n<p><img src=\"https:\/\/remy.io\/img\/articles\/cam-part-2\/captures.png\" alt=\"List of image captured by the camera\"><\/p>\n<p>I now have pictures of my cat eating at night, yay!<\/p>\n<p><img src=\"https:\/\/remy.io\/img\/articles\/cam-part-2\/2016-11-05-03-04-05.jpg\" alt=\"Picture of the camera: my cat eating\"><\/p>\n<p>At this point, I have an application running on my Intel NUC which saves the pictures of the camera only when there is motion. Remember at this point that the camera is not connected to Internet because this is something I don&rsquo;t want.<\/p>\n<p>I&rsquo;m actually surprised of the efficiency of this naive implementation of motion detection. It works really well and has close to zero false positives.<\/p>\n<p>That&rsquo;s the end of this <strong>Part 2 - Motion detection<\/strong>. The whole source code is available here: <a href=\"https:\/\/github.com\/remeh\/mehcam\">https:\/\/github.com\/remeh\/mehcam<\/a><\/p>\n<p>In the <a href=\"https:\/\/remy.io\/blog\/how-ive-created-an-intruder-detector-part-3\">Part 3 - Notification<\/a>, we&rsquo;ll see how we can modify the code to send notifications in case of motion.<\/p>\n"}]}}