{"title":"log2","link":[{"@attributes":{"href":"https:\/\/log2.ch\/","rel":"alternate"}},{"@attributes":{"href":"https:\/\/log2.ch\/feeds\/all.atom.xml","rel":"self"}}],"id":"https:\/\/log2.ch\/","updated":"2019-05-17T00:00:00+02:00","entry":[{"title":"Exploring a Pixel-Maze with Evolution\u00a0Strategies","link":{"@attributes":{"href":"https:\/\/log2.ch\/2019\/exploring-a-pixel-maze-with-evolution-strategies","rel":"alternate"}},"published":"2019-05-17T00:00:00+02:00","updated":"2019-05-17T00:00:00+02:00","author":{"name":"Martin Renold"},"id":"tag:log2.ch,2019-05-17:\/2019\/exploring-a-pixel-maze-with-evolution-strategies","summary":"<p>Since a few years it is possible to learn playing Atari games directly from pixels with reinforcement learning. A more recent discovery was that evolution strategies are competitive for training deep neural networks on those tasks. And even a simple genetic algorithm can work.<p>Those methods are fun to play around with. I have created a maze-world&nbsp;\u2026<\/p>","content":"\n\n\n\n<h2>Intro<\/h2>\n\n\n<p>Since a few years it is possible to learn playing Atari games directly from pixels <a href=\"https:\/\/www.semanticscholar.org\/paper\/Playing-Atari-with-Deep-Reinforcement-Learning-Mnih-Kavukcuoglu\/667fb84bfca10bec165f9e2cca3e21f5e4829ca7\">with reinforcement learning<\/a><sup id=\"fnref:Mnih\"><a class=\"footnote-ref\" href=\"#fn:Mnih\">1<\/a><\/sup>. A more recent discovery was that <a href=\"https:\/\/openai.com\/blog\/evolution-strategies\/\">evolution strategies are competitive<\/a><sup id=\"fnref:Salimans\"><a class=\"footnote-ref\" href=\"#fn:Salimans\">2<\/a><\/sup> for training deep neural networks on those tasks. And even a <a href=\"https:\/\/eng.uber.com\/deep-neuroevolution\/\">simple genetic algorithm<\/a><sup id=\"fnref:Such\"><a class=\"footnote-ref\" href=\"#fn:Such\">3<\/a><\/sup> can&nbsp;work.<\/p>\n\n\n\n\n<p>Those methods are fun to play around with. I have created a maze-world and trained a neural network to explore it. This task is much simpler than Atari but still gives interesting&nbsp;results.<\/p>\n<p>I&#8217;m using Covariance-Matrix Adaptation (<span class=\"caps\">CMA<\/span>-<span class=\"caps\">ES<\/span>) and have tried a few simpler methods. The focus is on results but code is&nbsp;available.<\/p>\n\n\n<h2>Maze<\/h2>\n<p>This is my generated 2D maze<sup id=\"fnref:maze\"><a class=\"footnote-ref\" href=\"#fn:maze\">8<\/a><\/sup>,&nbsp;featuring:<\/p>\n<ul>\n<li>Food&nbsp;(red)<\/li>\n<li>Agents&nbsp;(green)<\/li>\n<li>Traces (dark&nbsp;green)<\/li>\n<\/ul>\n<p>The agents cannot die. They all use a copy of the same controller. The goal is to improve this controller so that the agents (together) find as much food as&nbsp;possible.<\/p>\n<p>This video shows a mediocre controller in five random&nbsp;worlds:<\/p>\n\n\n\n\n\n\n<script>\nfunction playNextWorld(id, sequences) {\n  if (sequences === undefined) sequences = 5;\n  let v = document.getElementById(id);\n  v.pause();\n  let pos_rel = Math.ceil(v.currentTime \/ v.duration * sequences) \/ sequences;\n  let seek = (pos_rel % 1.0) * v.duration;\n  if (v.currentTime == seek) {\n    v.play();\n  } else {\n    v.currentTime = seek;\n    setTimeout(function() { v.play(); }, 1000);\n  }\n}\n<\/script>\n\n<video id=\"video-8k\" controls muted class=\"evalvideo\" src=\"https:\/\/log2.ch\/2019\/exploring-a-pixel-maze-with-evolution-strategies\/8kEvals.mp4\" type=\"video\/mp4\">(html5 video)<\/video>\n<p><button class=\"btn btn-small\" onclick=\"playNextWorld('video-8k')\">\u25b6| &nbsp;next world<\/button><\/p>\n<p>Observations: The agents start moving in different directions, but later all drift in a random walk to the bottom right. Sometimes they eat food blobs. (They can actually move through walls with a very low probability<sup id=\"fnref:walls\"><a class=\"footnote-ref\" href=\"#fn:walls\">9<\/a><\/sup>.)<\/p>\n<h2>Agent&nbsp;Controller<\/h2>\n<p>The controller is a neural network with fixed topology<sup id=\"fnref:topology\"><a class=\"footnote-ref\" href=\"#fn:topology\">10<\/a><\/sup> and learned weights. The architecture is pretty standard except for the memory<sup id=\"fnref:memory\"><a class=\"footnote-ref\" href=\"#fn:memory\">11<\/a><\/sup>.<\/p>\n<p><img alt=\"Inputs, Neural Network and Outputs of Controller\" src=\"https:\/\/log2.ch\/2019\/exploring-a-pixel-maze-with-evolution-strategies\/controller-setup.svg\"><\/p>\n<p>Inputs:<\/p>\n<ul>\n<li>The agents are nearly blind. They only see walls and food next to&nbsp;them.<\/li>\n<li>The agents don&#8217;t know their previous action. To keep going into a new direction they must learn how to use the external&nbsp;memory.<\/li>\n<li>The trace count can detect crowded areas. It includes the agent&#8217;s own trace.<sup id=\"fnref:trace\"><a class=\"footnote-ref\" href=\"#fn:trace\">12<\/a><\/sup><\/li>\n<\/ul>\n<p>Outputs:<\/p>\n<ul>\n<li>The softmax makes it simple to output fixed probabilities over actions, like &#8220;60% down, 40%&nbsp;left&#8221;.<\/li>\n<li>An output with high certainty (&#8220;99.9% down&#8221;) requires larger weights, which can only appear later during&nbsp;training.<\/li>\n<\/ul>\n<p>The network has a single hidden layer and about 1&#8216;000 parameters (weights and&nbsp;biases).<\/p>\n<p>The parameters are optimized directly. The setup violates some assumptions<sup id=\"fnref:assumptions\"><a class=\"footnote-ref\" href=\"#fn:assumptions\">13<\/a><\/sup> for standard (value-based) reinforcement learning, but a direct search for the parameters does not rely on those&nbsp;assumptions.<\/p>\n<h2>Training<\/h2>\n\n\n\n\n\n\n<p>Inspired by the <a href=\"http:\/\/blog.otoro.net\/2017\/10\/29\/visual-evolution-strategies\/\">Visual Guide to Evolution Strategies<\/a><sup id=\"fnref:hardmaru\"><a class=\"footnote-ref\" href=\"#fn:hardmaru\">5<\/a><\/sup>\nand by remarks in various papers I have used the Covariance-Matrix Adaptation Evolution Strategy (<span class=\"caps\">CMA<\/span>-<span class=\"caps\">ES<\/span>). It worked with practically no tuning. I have tried other black-box optimization methods (<a href=\"#simpler-methods\">discussed below<\/a>) but was unable to reach the same&nbsp;score.<\/p>\n<p>I dare to compare <span class=\"caps\">CMA<\/span>-<span class=\"caps\">ES<\/span> with the <a href=\"https:\/\/medium.com\/rants-on-machine-learning\/the-unreasonable-effectiveness-of-random-forests-f33c3ce28883\">Random Forest Classifier<\/a><sup id=\"fnref:random-forest\"><a class=\"footnote-ref\" href=\"#fn:random-forest\">6<\/a><\/sup>: a strong baseline that works out-of-the-box and is difficult to beat. Though it only works up to a few thousand&nbsp;parameters.<\/p>\n<p>Here is a summary: <span class=\"caps\">CMA<\/span>-<span class=\"caps\">ES<\/span> tracks a Gaussian distribution (with full covariance) over the parameters. The user provides an initial estimate of the scale of each parameter (variance). \n<span class=\"caps\">CMA<\/span>-<span class=\"caps\">ES<\/span> will then sample a small population, evaluate it and sort by fitness. This rank information is used to update the distribution, and the process&nbsp;repeats.<\/p>\n\n\n\n\n\n\n<p>For a better overview, see <a href=\"http:\/\/blog.otoro.net\/2017\/10\/29\/visual-evolution-strategies\/#what-is-an-evolution-strategy\">&#8220;What is an Evolution Strategy?&#8221;<\/a> from the same guide. The general principle is easy to understand, but the details are a bit involved. With <span class=\"caps\">CMA<\/span>-<span class=\"caps\">ES<\/span> you really should be using a library (<a href=\"https:\/\/github.com\/CMA-ES\/pycma\">pycma<\/a>) and not implement from&nbsp;scratch.<\/p>\n<p>Training&nbsp;plot:<\/p>\n<p><img alt=\"training plot\" src=\"https:\/\/log2.ch\/2019\/exploring-a-pixel-maze-with-evolution-strategies\/cma-es-training-overview.png\"><\/p>\n\n<p>Each generation is evaluated on a different set of random maps. There is a lot of fitness noise because of this. The moving average shows a slight upwards trend at the&nbsp;end.<\/p>\n<p>It takes about an hour to reach an average score of 1&#8216;000 on my low-end <span class=\"caps\">PC<\/span>. It takes about ten hours on an <span class=\"caps\">AWS<\/span> c5.9xlarge instance to reach the score above<sup id=\"fnref:longer\"><a class=\"footnote-ref\" href=\"#fn:longer\">14<\/a><\/sup>.<\/p>\n<p>After 44&#8216;000 evaluations (second&nbsp;video): <\/p>\n<video id=\"video-44k\" controls muted class=\"evalvideo\" src=\"https:\/\/log2.ch\/2019\/exploring-a-pixel-maze-with-evolution-strategies\/44kEvals.mp4\" type=\"video\/mp4\">(html5 video)<\/video>\n<p><button class=\"btn btn-small video-button\" onclick=\"playNextWorld('video-44k')\">\u25b6| &nbsp;next world<\/button><\/p>\n<p>Observations: The agents now keep changing directions. Sometimes they jitter around for a moment, especially when they should be turning. They can escape from dead ends, but it requires many attempts if the exit is&nbsp;narrow.<\/p>\n\n\n<p>And finally, after 2&#8216;000&#8216;000&nbsp;evaluations:<\/p>\n<video id=\"video-2M\" controls muted class=\"evalvideo\" src=\"https:\/\/log2.ch\/2019\/exploring-a-pixel-maze-with-evolution-strategies\/2000kEvals.mp4\" type=\"video\/mp4\">(html5 video)<\/video>\n<p><button class=\"btn btn-small\" onclick=\"playNextWorld('video-2M')\">\u25b6| &nbsp;next world<\/button><\/p>\n<p>Observations: The winning strategy follows the left wall at a short distance. In empty space it prefers straight lines with random left-turns. At the end of a horizontal line of food the agent usually turns 180 degrees and finds back to the remaining food blob.<sup id=\"fnref:find-back\"><a class=\"footnote-ref\" href=\"#fn:find-back\">15<\/a><\/sup><\/p>\n<h2>Discussion<\/h2>\n<p><span class=\"caps\">CMA<\/span>-<span class=\"caps\">ES<\/span> is known to work well with tens to hundreds of parameters. With just 1&#8216;000 parameters we are already approaching its limits. This may be a bit disappointing, because the learned strategy does not look really complex. I expect that the model could be improved to fit the task better, reducing the number of&nbsp;parameters.<\/p>\n<p>To put the network size into context: six parameters are enough to learn the classic cart-pole balancing task; 5&#8216;000 have been used to control a simple 2D-Hopper robot; and 33&#8216;000 to play Atari games from pixels<sup id=\"fnref:Schulman\"><a class=\"footnote-ref\" href=\"#fn:Schulman\">4<\/a><\/sup>, though four million have also been used<sup id=\"fnref2:Such\"><a class=\"footnote-ref\" href=\"#fn:Such\">3<\/a><\/sup>.<\/p>\n\n\n<p>Other evolution strategies can scale up. But <span class=\"caps\">CMA<\/span>-<span class=\"caps\">ES<\/span> can achieve amazing results with smart use of a few hundred parameters, like this <a href=\"https:\/\/www.goatstream.com\/research\/papers\/SA2013\/\">Muscle-Based Locomotion for Bipedal Creatures<\/a><sup id=\"fnref:muscle\"><a class=\"footnote-ref\" href=\"#fn:muscle\">7<\/a><\/sup>.<\/p>\n\n\n\n\n<p><span id=\"simpler-methods\">I have tried a few simpler methods.<\/span> A simple <span class=\"caps\">GA<\/span> (not shown here), the <a href=\"https:\/\/www.semanticscholar.org\/paper\/The-Cross-Entropy-Method-for-Fast-Policy-Search-Mannor-Rubinstein\/4866389d4d29cd31c1c7066a6121fb3984127ad9\">Cross-Entropy Method<\/a> (<span class=\"caps\">CEM<\/span>) and <a href=\"https:\/\/esa.github.io\/pagmo2\/docs\/cpp\/algorithms\/de.html\">differential evolution<\/a>. They all got stuck in local&nbsp;optima.<\/p>\n<p>Here is an impression of my attempts to find good <span class=\"caps\">CEM<\/span>&nbsp;parameters:<\/p>\n<p><img alt=\"plot comparing CMA-ES with CEM and differential evolution\" src=\"https:\/\/log2.ch\/2019\/exploring-a-pixel-maze-with-evolution-strategies\/different-algos.png\"><\/p>\n<p>(The <span class=\"caps\">CMA<\/span>-<span class=\"caps\">ES<\/span> lines have more noise because they use a smaller population size of 23 per generation, while other methods use between 200 and 1000. The dots show the fitness per generation before&nbsp;filtering.)<\/p>\n<p>This is a different (older) variant of the task which is faster to train.\nAs you can see, it is not a trivial optimization&nbsp;problem.<\/p>\n<p>One big challenge is the evaluation noise caused by the random maze generator. One lucky maze can have lots of food exactly where the agents are going. Some algorithms will give this score too much credibility, and use that lucky controller as a starting point for most mutations. Until the score gets beaten by an even more lucky evaluation. I think this is why differential evolution failed&nbsp;here.<\/p>\n\n\n\n\n\n\n\n\n\n<p>I have also tried variations of the neural&nbsp;network:<\/p>\n<ul>\n<li>\n<p>The controller did surprisingly well when I removed the hidden layer. It finds good solutions much faster. Maybe the memory feedback loop helps with that. But the advantage of using a hidden layer becomes clear after 3&#8216;000&nbsp;evaluations.<\/p>\n<\/li>\n<li>\n<p>The performance difference between 20 and 40 hidden neurons is not that large. With 20 neurons it trains faster (only 600 parameters). The defaults of <span class=\"caps\">CMA<\/span>-<span class=\"caps\">ES<\/span> worked great with 20 hidden nodes, but I have found that increasing the population size from 23 (default) to 200 helps with the larger&nbsp;network.<\/p>\n<\/li>\n<li>\n<p>A <a href=\"https:\/\/stats.stackexchange.com\/questions\/373136\/softmax-weights-initialization\/393012#393012\">good scaling of the initial weights<\/a> can speed up the first learning phase a lot. The same is true for input and output normalization. But this advantage often shrinks down to zero long before the final score is reached. Initialization should become more important with additional hidden layers. It&#8217;s known to be critical for deep neural&nbsp;networks.<\/p>\n<\/li>\n<\/ul>\n<h2>Code<\/h2>\n<p>Code is available in my <a href=\"https:\/\/github.com\/martinxyz\/pixelcrawl\">pixelcrawl<\/a> GitHub repository. It is not very polished, however it is reasonably optimized (at the expense of&nbsp;flexibility).<\/p>\n<p>The original version was pure Python. It was about 100x slower than the current Python\/C++ mix. There is a lot access to individual pixels and math involving small arrays, something which Python is really slow&nbsp;at.<\/p>\n<h2>Bonus<\/h2>\n<p>Can it survive in empty space? Or will it always stick to the&nbsp;walls?<\/p>\n<p>Let&#8217;s transfer the trained agent into a new&nbsp;environment:<\/p>\n<video id=\"video-alien\" controls muted class=\"evalvideo\" src=\"https:\/\/log2.ch\/2019\/exploring-a-pixel-maze-with-evolution-strategies\/alien-world.mp4\" type=\"video\/mp4\">(html5 video)<\/video>\n<p><button class=\"btn btn-small\" onclick=\"playNextWorld('video-alien', 4)\">\u25b6| &nbsp;next world<\/button><\/p>\n<p>Observations: The learned strategy is somewhat robust and looks well randomized. It has trouble getting into certain one-pixel cavities. But it keeps&nbsp;exploring.<\/p>\n<div class=\"footnote\">\n<hr>\n<ol>\n<li id=\"fn:Mnih\">\n<p><a href=\"https:\/\/arxiv.org\/abs\/1312.5602\">Playing Atari with Deep Reinforcement Learning<\/a>, Mnih et al., 2013.&#160;<a class=\"footnote-backref\" href=\"#fnref:Mnih\" title=\"Jump back to footnote 1 in the text\">&#8617;<\/a><\/p>\n<\/li>\n<li id=\"fn:Salimans\">\n<p><a href=\"https:\/\/openai.com\/blog\/evolution-strategies\/\">Evolution Strategies as a Scalable Alternative to Reinforcement Learning<\/a>, OpenAI blog, 2017. With accompanying paper <a href=\"https:\/\/arxiv.org\/abs\/1703.03864\">Salimans et al., 2017<\/a>.&#160;<a class=\"footnote-backref\" href=\"#fnref:Salimans\" title=\"Jump back to footnote 2 in the text\">&#8617;<\/a><\/p>\n<\/li>\n<li id=\"fn:Such\">\n<p><a href=\"https:\/\/arxiv.org\/abs\/1712.06567\">Deep Neuroevolution: Genetic Algorithms Are a Competitive&#8230;<\/a>, Such et al., 2017. With related article <a href=\"https:\/\/eng.uber.com\/deep-neuroevolution\/\">Welcoming the Era of Deep Neuroevolution<\/a>, Uber blog, 2017.&#160;<a class=\"footnote-backref\" href=\"#fnref:Such\" title=\"Jump back to footnote 3 in the text\">&#8617;<\/a><a class=\"footnote-backref\" href=\"#fnref2:Such\" title=\"Jump back to footnote 3 in the text\">&#8617;<\/a><\/p>\n<\/li>\n<li id=\"fn:Schulman\">\n<p><a href=\"https:\/\/arxiv.org\/abs\/1502.05477\">Trust Region Policy Optimization<\/a>, Schulman et al., 2015.&#160;<a class=\"footnote-backref\" href=\"#fnref:Schulman\" title=\"Jump back to footnote 4 in the text\">&#8617;<\/a><\/p>\n<\/li>\n<li id=\"fn:hardmaru\">\n<p><a href=\"http:\/\/blog.otoro.net\/2017\/10\/29\/visual-evolution-strategies\/\">A Visual Guide to Evolution Strategies<\/a>, blog post by David Ha (<a href=\"https:\/\/twitter.com\/hardmaru\">hardmaru<\/a>), 2017.&#160;<a class=\"footnote-backref\" href=\"#fnref:hardmaru\" title=\"Jump back to footnote 5 in the text\">&#8617;<\/a><\/p>\n<\/li>\n<li id=\"fn:random-forest\">\n<p><a href=\"https:\/\/medium.com\/rants-on-machine-learning\/the-unreasonable-effectiveness-of-random-forests-f33c3ce28883\">The Unreasonable Effectiveness Of Random Forest<\/a>, blog post by Ahmed El Deeb, 2015.&#160;<a class=\"footnote-backref\" href=\"#fnref:random-forest\" title=\"Jump back to footnote 6 in the text\">&#8617;<\/a><\/p>\n<\/li>\n<li id=\"fn:muscle\">\n<p><a href=\"https:\/\/www.goatstream.com\/research\/papers\/SA2013\/\">Muscle-Based Locomotion for Bipedal Creatures<\/a>, Geijtenbeek et al., 2013. Video and paper.&#160;<a class=\"footnote-backref\" href=\"#fnref:muscle\" title=\"Jump back to footnote 7 in the text\">&#8617;<\/a><\/p>\n<\/li>\n<li id=\"fn:maze\">\n<p>How do you generate a maze? Simple. <strike><a href=\"https:\/\/en.wikipedia.org\/wiki\/Perlin_noise\">Perlin Noise<\/a> and a threshold.<\/strike> Start from a noise image and apply a 3x3 binary filter repeatedly. There are 2<sup>512<\/sup> possible filters to choose from. Most of them transform noise into noise. Pick one that produces a low number of horizontal and vertical edges. Using this as a fitness measure, the Cross-Entropy method can <a href=\"https:\/\/github.com\/martinxyz\/lut2d-search\">find a good filter<\/a>. If you&#8217;re not happy with the result, use novelty search to evolve filters with maximum diversity. To measure the diversity of 2D patterns, use the first few layers of a pre-trained ResNet50 to extract basic visual features. This should <a href=\"https:\/\/log2.ch\/diversity-lut-search\/\">result in different textures<\/a>. Pick one that looks like a maze. Simple as that.&#160;<a class=\"footnote-backref\" href=\"#fnref:maze\" title=\"Jump back to footnote 8 in the text\">&#8617;<\/a><\/p>\n<\/li>\n<li id=\"fn:walls\">\n<p>In <a href=\"https:\/\/log2.ch\/misc\/cmaes\/rerun-score-360.mp4\">early experiments<\/a> I used a higher probability to move through walls because it was possible to start inside of a wall. The agents liked this a bit too much, especially when I gave them more time to explore. Drifting through walls in a fixed direction is a sure strategy to explore the whole maze, including disconnected parts. Whenever they got stuck they moved into the next wall instead of searching a way around.&#160;<a class=\"footnote-backref\" href=\"#fnref:walls\" title=\"Jump back to footnote 9 in the text\">&#8617;<\/a><\/p>\n<\/li>\n<li id=\"fn:topology\">\n<p>A well-known alternative would be to evolve the topology together with the weights using <a href=\"https:\/\/www.cs.ucf.edu\/~kstanley\/neat.html\"><span class=\"caps\">NEAT<\/span><\/a>. I have not tried this, but it&#8217;s known to work well for small networks. During the deep learning hype this approach was not getting much attention. Now the evolution of topology seems to be coming back as architecture search for deep vision networks. There is also recent work happening on indirect encoding (e.g. evolving regular patterns). See <a href=\"https:\/\/www.nature.com\/articles\/s42256-018-0006-z\">Designing neural networks through neuroevolution<\/a> (Stanley et al., 2019) for a recent overview.&#160;<a class=\"footnote-backref\" href=\"#fnref:topology\" title=\"Jump back to footnote 10 in the text\">&#8617;<\/a><\/p>\n<\/li>\n<li id=\"fn:memory\">\n<p>Using <code>memory[t+1] = 0.99 * memory[t] + 0.01 * outputs[t]<\/code>, so the memory is just a low-pass filtered version of the outputs. A standard recurrent neuron could have learnt this rule. The memory half-time of about 70 steps (<code>1\/-log2(0.99)<\/code>) was found to work well. I have also attempted to learn individual half-times for all 6 memory cells, so far without success.  I have not tested other structures, but it seems to be working okay. At first I initialized the memory (the filter states) with zeros. But <a href=\"https:\/\/log2.ch\/misc\/cmaes\/rerun-score-1121.mp4\">a swarm of 200 agents<\/a> performed <em>much<\/em> better with random initialization, because it gives each agent access to a different (somewhat constant) input. This helps to spread out early.&#160;<a class=\"footnote-backref\" href=\"#fnref:memory\" title=\"Jump back to footnote 11 in the text\">&#8617;<\/a><\/p>\n<\/li>\n<li id=\"fn:trace\">\n<p>The trace input is a left-over from <a href=\"https:\/\/log2.ch\/misc\/cmaes\/rerun-score-1121.mp4\">early experiments with 200 agents<\/a> swarming the maze. It was meant to detect and avoid crowded areas. But 200 agents were very confusing to watch and the strategy mostly resembled a flood-fill algorithm. I&#8217;m not sure if or how the current agents use this input.&#160;<a class=\"footnote-backref\" href=\"#fnref:trace\" title=\"Jump back to footnote 12 in the text\">&#8617;<\/a><\/p>\n<\/li>\n<li id=\"fn:assumptions\">\n<p>Standard <span class=\"caps\">RL<\/span> methods like Q-learning expect to be solving a <a href=\"https:\/\/danieltakeshi.github.io\/2015-08-02-markov-decision-processes-and-reinforcement-learning\/\">Markov Decision Process<\/a> (<span class=\"caps\">MDP<\/span>) that is fully observable. However the nearest four pixels are lacking a lot of information about the state. A memory is evolved to extract relevant information from the past. For the evolution strategy this is just some additional parameters to optimize, and never mind the feedback loop this creates. When learning Atari games the inputs are usually extended to include some history, e.g. the last four video frames, in the hope that all relevant state information can be extracted. A different issue is that the network controls only a single agent. It would be straight-forward to maximize the reward of this agent with standard <span class=\"caps\">RL<\/span>. But it is not clear how prevent competition and maximize the collective reward of all agents instead.&#160;<a class=\"footnote-backref\" href=\"#fnref:assumptions\" title=\"Jump back to footnote 13 in the text\">&#8617;<\/a><\/p>\n<\/li>\n<li id=\"fn:longer\">\n<p>The run presented here actually took a few days, because I was evaluating each agent on 100 maps instead of 10. This was complete overkill. <a href=\"https:\/\/log2.ch\/misc\/cmaes\/long-hunt-10vs100worlds.png\">It helps<\/a>, but not enough to justify 10x training time. Training was done on <span class=\"caps\">AWS<\/span> spot instances because they are cheap. They can be interrupted at any time, and I&#8217;m just counting on my luck; so far I&#8217;ve seen only two interruptions. Previously I used <a href=\"https:\/\/distributed.dask.org\/en\/latest\/\">dask distributed<\/a> with a central scheduler and spot instances as workers for a different experiment. This was quite involved to manage. For now I&#8217;m back to plain <a href=\"https:\/\/docs.dask.org\/en\/latest\/\">dask<\/a> to use multiple cores and good old rsync and ssh.<br>There is also a scaling issue that I didn&#8217;t get around to debug yet: the 36 vCPUs of the c5.9xlarge instance are utilized only at 40%. (Update: it&#8217;s mostly caused by the non-parallel <span class=\"caps\">CMA<\/span>-<span class=\"caps\">ES<\/span> update step, which must wait for all parallel evaluations to complete before starting new ones.) There is no point going distributed if the largest <span class=\"caps\">AWS<\/span> instance (72 vCPUs) cannot be saturated. The price per vCPU is nearly the same for small and large instances.&#160;<a class=\"footnote-backref\" href=\"#fnref:longer\" title=\"Jump back to footnote 14 in the text\">&#8617;<\/a><\/p>\n<\/li>\n<li id=\"fn:find-back\">\n<p>Remember that the agent cannot see when a food blob has ended, only that all four directions are empty. It must therefore be using its memory to prepare the correct turn-around. Also, the agents sometimes leave crumbs of food that they should have noticed. Probably a fast exploration of the maze has higher priority than collecting every bit of food. It&#8217;s also likely that it lacks the memory capacity to remember long enough. The videos actually run twice as long as an episode. The fitness is the amount of food collected in the first half of each video-episode. At this point there are usually big blobs left, which are more important than the crumbs.&#160;<a class=\"footnote-backref\" href=\"#fnref:find-back\" title=\"Jump back to footnote 15 in the text\">&#8617;<\/a><\/p>\n<\/li>\n<\/ol>\n<\/div>","category":[{"@attributes":{"term":"machine-learning"}},{"@attributes":{"term":"machine-learning"}}]},{"title":"STM32 and JTAG via\u00a0Raspberry","link":{"@attributes":{"href":"https:\/\/log2.ch\/2014\/stm32-and-jtag-via-raspberry","rel":"alternate"}},"published":"2014-10-28T00:00:00+01:00","updated":"2014-10-28T00:00:00+01:00","author":{"name":"Martin Renold"},"id":"tag:log2.ch,2014-10-28:\/2014\/stm32-and-jtag-via-raspberry","summary":"<p>Those are my notes about using the <a\nhref=\"http:\/\/www.coocox.org\/epi.html\">Embedded Pi<\/a> board with an\nopen source toolchain. Some apply to <span class=\"caps\">STM32<\/span> in general.  The Embedded\nPi board has a <span class=\"caps\">STM32<\/span> microcontroller with Arduino-compatible&nbsp;headers.<\/p>\n<p><small>&lt;rant&gt; It was obviously designed by people who don&#8217;t know\nthe open source toolchain for that chip \u2026<\/small><\/p>","content":"<p>Those are my notes about using the <a\nhref=\"http:\/\/www.coocox.org\/epi.html\">Embedded Pi<\/a> board with an\nopen source toolchain. Some apply to <span class=\"caps\">STM32<\/span> in general.  The Embedded\nPi board has a <span class=\"caps\">STM32<\/span> microcontroller with Arduino-compatible&nbsp;headers.<\/p>\n<p><small>&lt;rant&gt; It was obviously designed by people who don&#8217;t know\nthe open source toolchain for that chip.  If you don&#8217;t already have an\nEmbedded Pi, consider buying something else like the <a\nhref=\"http:\/\/www.raspberrypi.org\/forums\/viewtopic.php?f=45&t=80726\">ARMinARM<\/a>\nor a <span class=\"caps\">STM32<\/span> discovery&nbsp;board.<\/p>\n<p>Most likely CooCox just wanted to ride\nthe hype around Arduino and Raspberry to push their Windows-based\n<span class=\"caps\">IDE<\/span> and their &#8220;<span class=\"caps\">BSD<\/span> licensed free and open source&#8221; operating system\nfor which &#8220;No body has the right to sell the software or related\nproducts without permission&#8221;. (Both quotes are from the CoOS terms and\ncondition.)  I&#8217;m not going to bother with this stuff in my\nspare time.  This is a board with a <span class=\"caps\">STM32<\/span> chip on it, and there is an\nopen source toolchain for that chip.\n&lt;\/rant&gt;<\/small><\/p>\n<p>I&#8217;m using the Embedded Pi in standalone mode, with the <a\nhref=\"http:\/\/www.raspberrypi.org\/\">Raspberry Pi<\/a> acting as a\nprogrammer. The provided adapter cable connects only the\nArduino-compatible pins, not the pins used for programming. It&#8217;s\neasier to ignore the &#8220;<span class=\"caps\">RAS<\/span>-<span class=\"caps\">PI<\/span>&#8221; header altogether and just wire up\nthings&nbsp;directly.<\/p>\n<p>There are two programming&nbsp;options:<\/p>\n<ul>\n<li><span class=\"caps\">JTAG<\/span><\/li>\n<li><span class=\"caps\">STM32<\/span> built-in&nbsp;bootloader<\/li>\n<\/ul>\n<p><span class=\"caps\">JTAG<\/span> is prefered because it allows debugging.  The <span class=\"caps\">STM32<\/span> built-in\nbootloader allows to flash over a serial interface, which is a simpler\nprotocol. I will describe both&nbsp;ways.<\/p>\n<h1>Fancyblink<\/h1>\n<p>Here is my <a\nhref=\"https:\/\/github.com\/martinxyz\/fancyblink\">fancyblink<\/a> demo,\nwhich blinks the board <span class=\"caps\">LED<\/span> in a sine wave.  If you just want a binary\nfor testing, use <a href=\"\/images\/fancyblink.elf\">fancyblink.elf<\/a> or <a\nhref=\"\/images\/fancyblink.bin\">fancyblink.bin<\/a>.<\/p>\n<p>I use <a href=\"http:\/\/libopencm3.org\">libopencm3<\/a> with the <a\nhref=\"http:\/\/www.st.com\/web\/en\/resource\/technical\/document\/reference_manual\/CD00171190.pdf\">\n<span class=\"caps\">STM32<\/span> reference manual<\/a> close at hand. An alternative may be the <a\nhref=\"https:\/\/github.com\/mbedmicro\/mbed\">mbed <span class=\"caps\">SDK<\/span><\/a> based on <span class=\"caps\">CMSIS<\/span>.\nI compile on Linux (Debian sid). The compiler was installed&nbsp;with:<\/p>\n<div class=\"highlight\"><pre><span><\/span><code>apt-get install gcc-arm-none-eabi gdb-arm-none-eabi\n<\/code><\/pre><\/div>\n\n<p>You could also compile on the Raspberry Pi, at least the <a\nhref=\"http:\/\/www.raspberrypi.org\/forums\/viewtopic.php?f=45&t=80726\">ARMinARM<\/a>\npeople seem to have that&nbsp;working.<\/p>\n<h1><span class=\"caps\">JTAG<\/span><\/h1>\n<h2>Hardware&nbsp;setup:<\/h2>\n<p>By default the <span class=\"caps\">JTAG<\/span> connector can only be used for <span class=\"caps\">SW<\/span> (serial wire).\nCurrently support for <span class=\"caps\">SW<\/span> in OpenOCD is experimental, but\nsupport for <span class=\"caps\">JTAG<\/span> is stable.  Solder the bridge to connect the <span class=\"caps\">TDI<\/span> <span class=\"caps\">JTAG<\/span>\npin, as described in the <a\nhref=\"http:\/\/www.coocox.org\/Embedded_Pi\/Embedded_Pi_User_Manual.pdf\">Embedded\nPi User Manual<\/a>. Keep in mind that Arduino pin 8 will not be accessible\nfrom the <span class=\"caps\">STM32<\/span>.<\/p>\n<table>\n<thead>\n<tr>\n<th><span class=\"caps\">JTAG<\/span> Header<\/th>\n<th>Pi Header<\/th>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td>3 (<span class=\"caps\">GND<\/span>)<\/td>\n<td>25 (<span class=\"caps\">GND<\/span>)<\/td>\n<\/tr>\n<tr>\n<td>4 (<span class=\"caps\">TCK<\/span>)<\/td>\n<td>23 (<span class=\"caps\">GPIO11<\/span>\/<span class=\"caps\">SCLK<\/span>)<\/td>\n<\/tr>\n<tr>\n<td>6 (<span class=\"caps\">TDO<\/span>)<\/td>\n<td>21 (<span class=\"caps\">GPIO9<\/span>\/<span class=\"caps\">MISO<\/span>)<\/td>\n<\/tr>\n<tr>\n<td>8 (<span class=\"caps\">TDI<\/span>)<\/td>\n<td>19 (<span class=\"caps\">GPIO10<\/span>\/<span class=\"caps\">MOSI<\/span>)<\/td>\n<\/tr>\n<tr>\n<td>2 (<span class=\"caps\">TMS<\/span>)<\/td>\n<td>22 (<span class=\"caps\">GPIO25<\/span>)<\/td>\n<\/tr>\n<tr>\n<td>10 (<span class=\"caps\">RESET<\/span>)<\/td>\n<td>18 (<span class=\"caps\">GPIO24<\/span>)<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<p>Note that <span class=\"caps\">JTAG<\/span> pin 10 is the system reset (<span class=\"caps\">SRST<\/span>, not <span class=\"caps\">TRST<\/span>) which is\nconnected to the reset&nbsp;button.<\/p>\n<p><a href=\"\/images\/jtag1.jpg\"> <img src=\"\/images\/jtag1.jpg\" width=\"33%\" align=\"left\"\/> <\/a><\/p>\n<p><a href=\"\/images\/jtag2.jpg\"> <img src=\"\/images\/jtag2.jpg\" width=\"33%\" align=\"right\"\/> <\/a><\/p>\n<p><br clear=\"all\"\/><\/p>\n<h2>Software&nbsp;setup:<\/h2>\n<p>I&#8217;m using <a href=\"http:\/\/www.raspbian.org\/\">raspbian<\/a> on the Pi,\nwhich has an <a href=\"http:\/\/openocd.sourceforge.net\/\">OpenOCD<\/a>\npackage. But it doesn&#8217;t have the fast <span class=\"caps\">GPIO<\/span> driver&nbsp;(bcm2835gpio).<\/p>\n<p>Installing:<\/p>\n<div class=\"highlight\"><pre><span><\/span><code>sudo apt-get build-dep openocd\nsudo apt-get install git\ngit clone git:\/\/git.code.sf.net\/p\/openocd\/code openocd\ncd openocd\n.\/bootstrap &amp;&amp; .\/configure --enable-bcm2835gpio &amp;&amp; make\nsudo make install\n<\/code><\/pre><\/div>\n\n<p>Running:<\/p>\n<div class=\"highlight\"><pre><span><\/span><code>sudo openocd -f interface\/raspberrypi-native.cfg \\\n -c &#39;bcm2835gpio_srst_num 24&#39; \\\n -c &#39;reset_config srst_only srst_push_pull&#39; \\\n -f target\/stm32f1x.cfg\n<\/code><\/pre><\/div>\n\n<p>You may get messages&nbsp;like:<\/p>\n<blockquote>\n<p>Error: <span class=\"caps\">JTAG<\/span> scan chain interrogation failed: all ones<br>\nWarn : Invalid <span class=\"caps\">ACK<\/span> 0x7 in <span class=\"caps\">JTAG<\/span>-<span class=\"caps\">DP<\/span>&nbsp;transaction<\/p>\n<\/blockquote>\n<p>which means you got the wires wrong.  OpenOCD will run anyway, but\ndon&#8217;t be tempted to continue. Go fix your&nbsp;wiring.<\/p>\n<p>OpenOCD is now in server mode, waiting for a debugger connection. \nYou only wanted to flash? No problem. Add yet another&nbsp;argument:<\/p>\n<div class=\"highlight\"><pre><span><\/span><code> -c &#39;program fancyblink.elf verify reset&#39;\n<\/code><\/pre><\/div>\n\n<p>or<\/p>\n<div class=\"highlight\"><pre><span><\/span><code> -c &#39;program fancyblink.bin 0x08000000 verify reset&#39;\n<\/code><\/pre><\/div>\n\n<p>For debugging, I&#8217;m running gdb on my Debian <span class=\"caps\">PC<\/span>:<\/p>\n<div class=\"highlight\"><pre><span><\/span><code>arm-none-eabi-gdb fancyblink.elf\n<\/code><\/pre><\/div>\n\n<p>on the gdb&nbsp;prompt:<\/p>\n<div class=\"highlight\"><pre><span><\/span><code><span class=\"n\">target<\/span><span class=\"w\"> <\/span><span class=\"k\">remote<\/span><span class=\"w\"> <\/span><span class=\"n\">raspberrypi<\/span><span class=\"p\">:<\/span><span class=\"mi\">3333<\/span>\n<span class=\"nb\">load<\/span>\n<span class=\"k\">continue<\/span>\n<\/code><\/pre><\/div>\n\n<p>You can put some commands into a .gdbinit file for convenience. And,\nof course, create a shell script for the lengthy openocd\ninvocation. Also, if you connect via ssh, use &#8216;ssh-copy-id&#8217; instead of\ntyping a password every&nbsp;time.<\/p>\n<h1><span class=\"caps\">STM32<\/span> built-in&nbsp;bootloader<\/h1>\n<ul>\n<li>Disable the Linux system console on the Raspberry Pi.  Edit\n  \/boot\/cmdline.txt and \/etc\/inittab to get rid of all usage of&nbsp;ttyAMA0.<\/li>\n<li>Compile <a href=\"https:\/\/code.google.com\/p\/stm32flash\/\">stm32flash<\/a> on the Raspberry Pi.<ul>\n<li>Do not use stm32load.py.  It works, but its error handling is\n  broken. You get messages like &#8220;None&#8221; and &#8220;<span class=\"caps\">NONE<\/span>&#8221;.  Stm32flash is\n  more mature, it prints chip information and can drive <span class=\"caps\">GPIO<\/span> pins\n  so you don&#8217;t have to push <span class=\"caps\">RESET<\/span>.<\/li>\n<\/ul>\n<\/li>\n<li>Wire it&nbsp;up.<\/li>\n<\/ul>\n<table>\n<thead>\n<tr>\n<th>Label (Embedded Pi)<\/th>\n<th>Pi Header<\/th>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td><span class=\"caps\">GND<\/span><\/td>\n<td>25 (<span class=\"caps\">GND<\/span>)<\/td>\n<\/tr>\n<tr>\n<td><span class=\"caps\">RESET<\/span><\/td>\n<td>18 (<span class=\"caps\">GPIO24<\/span>)<\/td>\n<\/tr>\n<tr>\n<td>28<\/td>\n<td>8 (<span class=\"caps\">GPIO14<\/span>\/<span class=\"caps\">TXD<\/span>)<\/td>\n<\/tr>\n<tr>\n<td>26<\/td>\n<td>10 (<span class=\"caps\">GPIO15<\/span>\/<span class=\"caps\">RXD<\/span>)<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<p><a href=\"\/images\/serial.jpg\"> <img src=\"\/images\/serial.jpg\" width=\"55%\"\/> <\/a><\/p>\n<p>Also connect the <span class=\"caps\">BOOT0<\/span> to 3V3 on the Embedded Pi.  Theoretically you\ncan push the <span class=\"caps\">RESET<\/span> and <span class=\"caps\">BOOT0<\/span> buttons. But often the bootloader seems\nto be stuck answering at a low baud rate. It&#8217;s more reliable (and easier)\nto let stm32flash drive <span class=\"caps\">RESET<\/span>.<\/p>\n<p>Flash, Verify,&nbsp;Run:<\/p>\n<div class=\"highlight\"><pre><span><\/span><code>stm32flash -b 115200 \/dev\/ttyAMA0 -i -24,24 -w fancyblink.bin -v -R -g 0\n<\/code><\/pre><\/div>\n\n<p><em>Warning:<\/em> when you execute the flashed program with the &#8220;-g&#8221; command,\ninterrupt handlers will not work (if you use them). You have\nto&nbsp;add<\/p>\n<div class=\"highlight\"><pre><span><\/span><code>SCB_VTOR = 0x08000000; \/\/ point irq vector to flash\n<\/code><\/pre><\/div>\n\n<p>to your startup code. Otherwise execution will jump into the\nbootloader&#8217;s interrupt handler at the first <span class=\"caps\">IRQ<\/span> and most likely never\nreturn. You could also release <span class=\"caps\">BOOT0<\/span> and then reset to solve this\nproblem by starting directly from&nbsp;flash.<\/p>","category":[{"@attributes":{"term":"embedded"}},{"@attributes":{"term":"stm32"}}]},{"title":"Developer Monologue: Short History of\u00a0MyPaint","link":{"@attributes":{"href":"https:\/\/log2.ch\/2009\/developer-monologue-short-history-of-mypaint","rel":"alternate"}},"published":"2009-06-18T00:00:00+02:00","updated":"2009-06-18T00:00:00+02:00","author":{"name":"Martin Renold"},"id":"tag:log2.ch,2009-06-18:\/2009\/developer-monologue-short-history-of-mypaint","summary":"<p>Okay, this is so that everyone can just stop asking why <a href=\"https:\/\/github.com\/mypaint\/mypaint\">MyPaint<\/a>,&nbsp;etc.<\/p>\n<p>Around 2004 I bought myself a wacom tablet, having checked that it should work with <span class=\"caps\">GIMP<\/span> on Linux. It did work. With minor bugs and glitches. For example, <span class=\"caps\">GIMP<\/span> would sometimes drop a stroke when scribbling too \u2026<\/p>","content":"<p>Okay, this is so that everyone can just stop asking why <a href=\"https:\/\/github.com\/mypaint\/mypaint\">MyPaint<\/a>,&nbsp;etc.<\/p>\n<p>Around 2004 I bought myself a wacom tablet, having checked that it should work with <span class=\"caps\">GIMP<\/span> on Linux. It did work. With minor bugs and glitches. For example, <span class=\"caps\">GIMP<\/span> would sometimes drop a stroke when scribbling too fast. I also thought I could paint more freely if the brush would react in different ways to pressure and&nbsp;velocity.<\/p>\n<p>I created a small prototype to try this. It did render circular dabs with radius and opacity depending on pressure and velocity. I was also eager to use a lowpass filter when calculating the velocity, since this topic was covered in my engineering studies&nbsp;recently.<\/p>\n<p>It did work - I could paint with the program, and it was fun! I saved by taking <a href=\"\/images\/mypaint-prerelease.jpg\">screenshots<\/a>. I could have done the same art inside <span class=\"caps\">GIMP<\/span>, or with some freeware tool on Windows, but&#8230; A co-worker once joked that, should I ever want to drive, I would start by building my own&nbsp;car.<\/p>\n<p>I had some fun experimenting with other brush ideas. My simple XInput test program slowly turned into a <a href=\"\/images\/mypaint_sshot_v0.4.jpg\">brush editor<\/a>. Even today, the core of MyPaint is still the same experimental brush editor that I wanted to create at this time. It allows me to try my brush ideas quickly. Since I already prefered it to <span class=\"caps\">GIMP<\/span> for creating simple art, I released the code without much&nbsp;noise.<\/p>\n<p>The program slowly grew after that. Around 2006, with the release of version 0.4, the program was everything I wanted it to be. There was no undo or layers, but I had a nice brush collection now. It worked well with my tablet, and it was fun to use. I had reached my&nbsp;goal.<\/p>\n<p>Something must have gone wrong, since I continued development anyway. Users started to appear, some of them requesting features that I also wanted for myself anyway. When I added layer support, the project turned into something more serious than a brushtest. Well, as serious as a project can be when it ships with a debug menu and user-visible <span class=\"caps\">FIXME<\/span> comments in the tool&nbsp;settings.<\/p>\n<p>In the meantime, several people have contributed small enhancements or have just been enthusiastic and spread the word. And in general it looks like MyPaint is here to stay, since for some reason I can&#8217;t seem to stop working on it. Maybe for the lack of a better&nbsp;challenge?<\/p>\n<p>And MyPaint even <a href=\"http:\/\/web.archive.org\/web\/20150726015005\/http:\/\/mypaint.intilinux.com\/?page_id=56\">has some goals<\/a> now. (Update: links to web archive now. Try <a href=\"https:\/\/github.com\/mypaint\/mypaint\/wiki\/Documentation\">MyPaint documentation<\/a> for up-to-date&nbsp;information.)<\/p>\n<p><small>This post originally appeared in the <a href=\"http:\/\/web.archive.org\/web\/20150910154855\/http:\/\/mypaint.intilinux.com\/?p=57\">MyPaint blog<\/a>.<\/small><\/p>","category":[{"@attributes":{"term":"MyPaint"}},{"@attributes":{"term":"mypaint"}}]},{"title":"New pixel\u00a0format","link":{"@attributes":{"href":"https:\/\/log2.ch\/2008\/new-pixel-format","rel":"alternate"}},"published":"2008-11-21T00:00:00+01:00","updated":"2008-11-21T00:00:00+01:00","author":{"name":"Martin Renold"},"id":"tag:log2.ch,2008-11-21:\/2008\/new-pixel-format","summary":"<p><span class=\"caps\">GEGL<\/span> (the upcoming <span class=\"caps\">GIMP<\/span> backend) uses linear-light floating point as\nthe main pixel format. Krita allows several colorspaces and color\nmanagement through LittleCMS. What about MyPaint? Version 0.5 uses\n8bit sRGB without any alpha channel. Time for a&nbsp;change.<\/p>\n<p>I have settled now with a 16bit integer for each \u2026<\/p>","content":"<p><span class=\"caps\">GEGL<\/span> (the upcoming <span class=\"caps\">GIMP<\/span> backend) uses linear-light floating point as\nthe main pixel format. Krita allows several colorspaces and color\nmanagement through LittleCMS. What about MyPaint? Version 0.5 uses\n8bit sRGB without any alpha channel. Time for a&nbsp;change.<\/p>\n<p>I have settled now with a 16bit integer for each channel,\npermultiplied alpha, no linear light (gamma as in sRGB). The extra\nprecision helps against the noise when blitting many faint dabs on top\nof each other. Here you can see the difference (click to&nbsp;enlarge):<\/p>\n<p><a href=\"\/images\/8bit_vs_16bit.png\"> <img src=\"\/images\/8bit_vs_16bit.png\" width=\"66%\"\/> <\/a><\/p>\n<p>A more subtle point is that white (or opaque alpha) is not stored as\n2^16-1 but as 2^15. So, after multiplying a component with the alpha\nchannel, the neccessary division by 2^15 can be done with a simple bit\nshift (fixed-point arithmetic). So effectively only 15 bits are\nused. Nobody will miss that final bit of&nbsp;precision.<\/p>\n<p>I also have read quite a bit about linear light and even did a test\nimplementation. But after a short painting session I came to the\nconclusion that, while the effect for painting is subtle, it is mostly\ncounter-intuitive. Especially the fact that white is &#8220;stronger&#8221; than\nblack. I&#8217;m sure there are some special situations where it works\nbetter (eg zooming, and maybe layer compositing in some special\ncases), but right now, linear light is no longer a topic for&nbsp;me.<\/p>\n<p>The advantages of premultiplied alpha have been discussed more than\nenough elsewhere. I just want to share one little pleasant surprise:\nmy almost unoptimized C implementation blitting 16bit premultiplied\n<span class=\"caps\">RGBA<\/span> onto 8bit <span class=\"caps\">RGB<\/span> is three times faster than gdk-pixbuf blitting 8bit\n<span class=\"caps\">RGBA<\/span> onto 8bit <span class=\"caps\">RGB<\/span>.<\/p>\n<p><small>This post originally appeared in the <a href=\"http:\/\/mypaint.intilinux.com\/?p=19\">MyPaint blog<\/a>.<\/small><\/p>","category":[{"@attributes":{"term":"MyPaint"}},{"@attributes":{"term":"mypaint"}},{"@attributes":{"term":"fixed-point"}}]}]}