{"title":"Random stuff","generator":"Jekyll","link":[{"@attributes":{"rel":"self","type":"application\/atom+xml","href":"https:\/\/leucos.github.io\/feed.xml"}},{"@attributes":{"rel":"alternate","type":"text\/html","href":"https:\/\/leucos.github.io"}}],"updated":"2018-11-10T11:16:43+00:00","id":"https:\/\/leucos.github.io\/","author":{"name":"Michel Blanc","uri":"https:\/\/leucos.github.io\/","email":"mb@mbnet.fr"},"entry":[{"title":{"@attributes":{"type":"html"}},"link":{"@attributes":{"rel":"alternate","type":"text\/html","href":"https:\/\/leucos.github.io\/ansible-files-layout"}},"id":"https:\/\/leucos.github.io\/ansible-layout","published":"2015-07-02T00:00:00+00:00","updated":"2015-07-02T00:00:00+00:00","author":{"name":"Michel Blanc","uri":"https:\/\/leucos.github.io","email":"mb@mbnet.fr"},"content":"\n    <p><em>(revised 20181110 per @theenglishway suggestions)<\/em><\/p>\n\n<p>I have been writing playbooks for quite a while now. Along the way, I\nwent through various stages, and used different ways to layout Ansible\nfiles. I guess that after going down this trial and error path, I\nfinally came up with something I will stick to.<\/p>\n\n<p>I am not saying that this is the be-all and end-all of Ansible files\nlayout but may be it will fast forward you to a saner file layout, and\nyou\u2019ll be able to move on from there. This post will probably help you\nif you are new to Ansible, trying to figure out what to put and where.\nI hope it will prove usefull if you have some Ansible experience too.<\/p>\n\n<h2 id=\"some-terminology\">Some terminology<\/h2>\n\n<p>In this post, I will mostly talk about 3 things: roles, inventories and\nplaybooks. Other items do exist (plays, tasks, \u2026) but those 3 elements\nshape the big picture of the layout.<\/p>\n\n<h3 id=\"roles\">Roles<\/h3>\n\n<p>A role is a collection of tasks and templates (among other things, but\nthose are the most common) focused on one very specific goal. For\ninstance, you can have a role that installs nginx, another that deploys\nssh keys for admins, etc\u2026<\/p>\n\n<p>Nginx role will install and configure nginx. Nothing else. It won\u2019t\ncreate DNS entries, trim logs, add a ftp server or anything. It just\ninstalls nginx. Period.<\/p>\n\n<h3 id=\"inventories\">Inventories<\/h3>\n\n<p>An inventory is a list of hosts, eventually assembled into groups, on\nwhich you will run ansible playbooks. Ansible automatically puts all defined\nhosts in the aptly named group <code class=\"highlighter-rouge\">all<\/code>.<\/p>\n\n<p>For instance, you could have hosts <code class=\"highlighter-rouge\">www1<\/code> and <code class=\"highlighter-rouge\">www2<\/code>, assembled in group\n<code class=\"highlighter-rouge\">webservers<\/code>, and later reference the group or individual hosts,\ndepending on your needs.<\/p>\n\n<p>Inventories can also come with variables applied to hosts or groups\n(including <code class=\"highlighter-rouge\">all<\/code>).<\/p>\n\n<p>Inventories can be dynamic. If the inventory file is executable, Ansible\nwill run it and use its output as the inventory (note that, in this\ncase, the format is not the same as static inventory).<\/p>\n\n<p>You can of course have multiple inventories, segregated from each other.\nWe will take advantage from this later on.<\/p>\n\n<h3 id=\"playbooks\">Playbooks<\/h3>\n\n<p>The last piece of the puzzle is the playbook. The playbook is the pivot\nbetween and inventory and roles. This is where you basically tell\nAnsible: <em>please install roles foo, bar and baz on machines alice, bob\nand charlie<\/em>.<\/p>\n\n<h2 id=\"role-layout\">Role layout<\/h2>\n\n<p>Role layout is pretty well documented at Ansible website. A role contains\nseveral directories. All directories are optional besides <code class=\"highlighter-rouge\">tasks<\/code>. For each\ndirectory, the entry point is <code class=\"highlighter-rouge\">main.yml<\/code>. Thus, the only compulsory file in a\nrole is <code class=\"highlighter-rouge\">tasks\/main.yml<\/code>.<\/p>\n\n<div class=\"highlighter-rouge\"><div class=\"highlight\"><pre class=\"highlight\"><code>ansible-foobar\/\n\u251c\u2500\u2500 defaults\n\u2502\u00a0\u00a0 \u2514\u2500\u2500 main.yml\n\u251c\u2500\u2500 files\n\u251c\u2500\u2500 handlers\n\u2502\u00a0\u00a0 \u2514\u2500\u2500 main.yml\n\u251c\u2500\u2500 meta\n\u2502\u00a0\u00a0 \u2514\u2500\u2500 main.yml\n\u251c\u2500\u2500 tasks\n\u2502\u00a0\u00a0 \u251c\u2500\u2500 check_vars.yml\n\u2502\u00a0\u00a0 \u251c\u2500\u2500 foobar.yml\n\u2502\u00a0\u00a0 \u2514\u2500\u2500 main.yml\n\u2514\u2500\u2500 templates\n \u00a0\u00a0 \u2514\u2500\u2500 foobar.conf.j2\n<\/code><\/pre><\/div><\/div>\n\n<p>Let\u2019s cover briefly the layout an see the function of each file and\ndirectory.<\/p>\n\n<h3 id=\"defaultsmainyml\"><code class=\"highlighter-rouge\">defaults\/main.yml<\/code><\/h3>\n\n<p>This directory contains defaults for variables used in roles. I\nencourage you to define every variable used in your role, for several\nreasons:<\/p>\n\n<ul>\n  <li>this file will be a nice and always up to date reference list of\nsettings configuration in your roles<\/li>\n  <li>having configured variables will prevent your role failing in an\nuncontrolled way (more on this later).<\/li>\n<\/ul>\n\n<p>If some of these variables are used in templates to generate config\nfiles, I highly encourage you to use your target OS defaults. The principle of\nleast surprise should apply here.<\/p>\n\n<p>Best practices assumes that you are using <em>pseudo-namespacing<\/em> for your\nrole\u2019s variables (e.g. for role <code class=\"highlighter-rouge\">foobar<\/code>, all variables should begin\nwith <code class=\"highlighter-rouge\">foobar_<\/code>) to avoid collisions with other roles.<\/p>\n\n<h3 id=\"files\"><code class=\"highlighter-rouge\">files\/<\/code><\/h3>\n\n<p>This directory holds files that do not require Jinja interpolation, and can be copied as-is on the remote nodes.<\/p>\n\n<h3 id=\"handlersmainyml\"><code class=\"highlighter-rouge\">handlers\/main.yml<\/code><\/h3>\n\n<p>This is where you define handlers that get notified by tasks. Handlers\nare just standard tasks. You can use <code class=\"highlighter-rouge\">include<\/code> in this file if you want\nto separate handlers (for different OSes versions for instances), but\ntry to keep the file number as low as possible so you don\u2019t end up\nhunting down stuff everywhere.<\/p>\n\n<p>If your handler restarts any service, you have to make sure that the\nservice config file is valid before attempting to restart it. Some\ndaemons allow this (e.g. nginx, haproxy, apache). If your service does\nnot, provide some fallback mechanism. You don\u2019t want your playbook to\nscrew up your running system because you typoed a configuration\nvariable. See the <code class=\"highlighter-rouge\">validate<\/code> option in the \n<a href=\"http:\/\/docs.ansible.com\/template_module.html\">template module<\/a>.<\/p>\n\n<p>Note that handlers are just standard tasks.<\/p>\n\n<h3 id=\"metamainyml\"><code class=\"highlighter-rouge\">meta\/main.yml<\/code><\/h3>\n\n<p>This metadata has (AFAIK) only two variables:<\/p>\n\n<ul>\n  <li><code class=\"highlighter-rouge\">galaxy_info<\/code>: meta information for galaxy about your role. You just\ndon\u2019t need this if you don intend to push your role to Galaxy. For\ndetails on the format, see TODO: find ref<\/li>\n  <li><code class=\"highlighter-rouge\">dependencies<\/code>: what roles this role depends on.<\/li>\n<\/ul>\n\n<p>The latter is of utmost importance, and setting it right deserves a blog\npost on it\u2019s own. Until then, the rule of thumb to remember is to <strong>only\ninclude compulsory role dependencies for the target host<\/strong>.<\/p>\n\n<p>This means that adding <code class=\"highlighter-rouge\">nginx<\/code> dependency in a <code class=\"highlighter-rouge\">php-fpm<\/code> role sounds\nperfectly reasonable<sup id=\"fnref:1\"><a href=\"#fn:1\" class=\"footnote\">1<\/a><\/sup>. However, adding a <code class=\"highlighter-rouge\">mysql<\/code> dependency to your\nweb application role is not, because <code class=\"highlighter-rouge\">mysql<\/code> can be deployed on another\nserver.<\/p>\n\n<p><strong>A note 3 years later<\/strong>: I do not use dependencies anymore. I had issues\nregarding role\u2019s defaults variables behavior. Also, the playbook is the\nmain focus area when building infrastructure code. Having explicit\ndependencies in the playbook is the way to go. No weird or hard to track\nmagic.<\/p>\n\n<h3 id=\"tasksmainyml\"><code class=\"highlighter-rouge\">tasks\/main.yml<\/code><\/h3>\n\n<p>This file is the tasks entry point. However, it should be mostly empty.\nWhy ? Because you want to use Ansible tags. Tags are a great way to\nlimit task execution for an Ansible run, where only tagged tasks are\nrun.<\/p>\n\n<p>For instance, in a playbook that deploys your application, you could\nchoose to run only tasks regarding nginx.<\/p>\n\n<p>The problem is that tagging every task in <code class=\"highlighter-rouge\">main.yml<\/code> would be\ncumbersome, error prone, and clutter the code unnecessarily.<\/p>\n\n<p>The best way to tag all your tasks is to include your real task file\nfrom <code class=\"highlighter-rouge\">tasks\/main.yml<\/code> and tag the whole file:<\/p>\n\n<figure class=\"highlight\"><pre><code class=\"language-yaml\" data-lang=\"yaml\"><span class=\"pi\">-<\/span> <span class=\"na\">import_tasks<\/span><span class=\"pi\">:<\/span> <span class=\"s\">foobar.yml<\/span>\n  <span class=\"na\">tags<\/span><span class=\"pi\">:<\/span>\n    <span class=\"pi\">-<\/span> <span class=\"s\">foobar<\/span><\/code><\/pre><\/figure>\n\n<p>Here, I name the real task file <code class=\"highlighter-rouge\">foobar.yml<\/code> with the same name as the role\n(quite handy with <code class=\"highlighter-rouge\">find<\/code> or <code class=\"highlighter-rouge\">locate<\/code>; no need to guess which <code class=\"highlighter-rouge\">main.yml<\/code> you are\nlooking for) and apply the tag <code class=\"highlighter-rouge\">foobar<\/code> to all tasks in the role.<\/p>\n\n<p>You can repeat this if you have a big list of tasks and want to split\nthem in several files. You could, for instance, separate configuration\nand installation matters, and add another specific tag for each of them:<\/p>\n\n<figure class=\"highlight\"><pre><code class=\"language-yaml\" data-lang=\"yaml\"><span class=\"pi\">-<\/span> <span class=\"na\">import_tasks<\/span><span class=\"pi\">:<\/span> <span class=\"s\">foobar-install.yml<\/span>\n  <span class=\"na\">tags<\/span><span class=\"pi\">:<\/span>\n    <span class=\"pi\">-<\/span> <span class=\"s\">foobar<\/span>\n    <span class=\"pi\">-<\/span> <span class=\"s\">foobar:install<\/span>\n\n<span class=\"pi\">-<\/span> <span class=\"na\">import_tasks<\/span><span class=\"pi\">:<\/span> <span class=\"s\">foobar-config.yml<\/span>\n  <span class=\"na\">tags<\/span><span class=\"pi\">:<\/span>\n    <span class=\"pi\">-<\/span> <span class=\"s\">foobar<\/span>\n    <span class=\"pi\">-<\/span> <span class=\"s\">foobar:config<\/span><\/code><\/pre><\/figure>\n\n<p>Here I added two tags to the installation part (<code class=\"highlighter-rouge\">foobar<\/code> and\n<code class=\"highlighter-rouge\">foobar:install<\/code>), and two for the configuration part (<code class=\"highlighter-rouge\">foobar<\/code> and\n<code class=\"highlighter-rouge\">foobar:config<\/code>).<\/p>\n\n<p>Note that the <code class=\"highlighter-rouge\">:<\/code> between, for instance, <code class=\"highlighter-rouge\">foobar<\/code> and <code class=\"highlighter-rouge\">config<\/code> has no\nmeaning. Ansible treats tags as dumb strings. It is just a personnal\nconvention (Redis like) for refining tags.<\/p>\n\n<p>With this setup, you could run only the configuration part of your role\nby issuing:<\/p>\n\n<p><code class=\"highlighter-rouge\">ansible-playbook playbook.yml -t foobar:config<\/code><\/p>\n\n<p>The <code class=\"highlighter-rouge\">-t<\/code> and <code class=\"highlighter-rouge\">-l<\/code> combination is a very powerful weapon to target\na specific host with a precise change (think of this as pointing to a\nmatrix cell targetting host (i.e. row) and tag (i.e. column)).<\/p>\n\n<h4 id=\"a-word-of-caution\">A word of caution<\/h4>\n\n<p>Do not overdo tags: most of the time, this is YAGNI (You Ain\u2019t Gonna\nNeed It). Create a tag if you\u2019re gonna need it. It can be hard to\nmentally predict what will happen if you do too much. Beware of the\n<code class=\"highlighter-rouge\">never<\/code> tag, that will skip tasks <strong>unless<\/strong> you explicitely use another\ntag.<\/p>\n\n<p>For instance:<\/p>\n\n<figure class=\"highlight\"><pre><code class=\"language-yaml\" data-lang=\"yaml\"><span class=\"pi\">-<\/span> <span class=\"na\">import_tasks<\/span><span class=\"pi\">:<\/span> <span class=\"s\">foobar-uninstall.yml<\/span>\n  <span class=\"s\">tags<\/span>\n    <span class=\"s\">- never<\/span>\n    <span class=\"s\">- foobar<\/span><\/code><\/pre><\/figure>\n\n<p>will execute tasks in <code class=\"highlighter-rouge\">foobar-uninstall.yml<\/code> if tag <code class=\"highlighter-rouge\">foobar<\/code> is\nspecified at the command line.<\/p>\n\n<h3 id=\"taskscheck_varsyml\"><code class=\"highlighter-rouge\">tasks\/check_vars.yml<\/code><\/h3>\n\n<p>I use this file to ensure that required variables are defined.<\/p>\n\n<figure class=\"highlight\"><pre><code class=\"language-yaml\" data-lang=\"yaml\"><span class=\"c1\">#<\/span>\n<span class=\"c1\"># Checking that required variables are set<\/span>\n<span class=\"c1\">#<\/span>\n<span class=\"pi\">-<\/span> <span class=\"na\">name<\/span><span class=\"pi\">:<\/span> <span class=\"s\">Checking that required variables are set<\/span>\n  <span class=\"na\">fail<\/span><span class=\"pi\">:<\/span> <span class=\"s\">msg=\"{{ item }} is not defined\"<\/span>\n  <span class=\"na\">when<\/span><span class=\"pi\">:<\/span> <span class=\"s\">item not in vars<\/span>\n  <span class=\"na\">loop<\/span><span class=\"pi\">:<\/span>\n    <span class=\"pi\">-<\/span> <span class=\"s\">foobar_database<\/span>\n    <span class=\"pi\">-<\/span> <span class=\"s\">foobar_deploy_user<\/span><\/code><\/pre><\/figure>\n\n<p>Then, include this file in <code class=\"highlighter-rouge\">tasks\/main.yml<\/code>:<\/p>\n\n<figure class=\"highlight\"><pre><code class=\"language-yaml\" data-lang=\"yaml\"><span class=\"pi\">-<\/span> <span class=\"na\">import_tasks<\/span><span class=\"pi\">:<\/span> <span class=\"s\">check_vars.yml<\/span>\n  <span class=\"na\">tags<\/span><span class=\"pi\">:<\/span>\n    <span class=\"pi\">-<\/span> <span class=\"s\">foobar<\/span>\n    <span class=\"pi\">-<\/span> <span class=\"s\">foobar:check<\/span>\n    <span class=\"pi\">-<\/span> <span class=\"s\">check<\/span>\n\n<span class=\"pi\">-<\/span> <span class=\"na\">import_tasks<\/span><span class=\"pi\">:<\/span> <span class=\"s\">foobar.yml<\/span>\n  <span class=\"na\">tags<\/span><span class=\"pi\">:<\/span>\n    <span class=\"pi\">-<\/span> <span class=\"s\">foobar<\/span><\/code><\/pre><\/figure>\n\n<h3 id=\"templates\"><code class=\"highlighter-rouge\">templates\/*<\/code><\/h3>\n\n<p>This is the place where templates (i.e. files with interpolated\nvariables goes). While this is not necessary, I often reference them\nusing a relative path like so:<\/p>\n\n<figure class=\"highlight\"><pre><code class=\"language-yaml\" data-lang=\"yaml\"><span class=\"pi\">-<\/span> <span class=\"na\">name<\/span><span class=\"pi\">:<\/span> <span class=\"s\">Template foo<\/span>\n  <span class=\"na\">template<\/span><span class=\"pi\">:<\/span>\n    <span class=\"na\">src<\/span><span class=\"pi\">:<\/span> <span class=\"s\">..\/templates\/foo.conf.j2<\/span>\n    <span class=\"na\">dest<\/span><span class=\"pi\">:<\/span> <span class=\"s\">\/some\/place\/in\/the\/node\/filesystem\/foo.conf<\/span><\/code><\/pre><\/figure>\n\n<p>The goal of using relative path is to be able to hit <code class=\"highlighter-rouge\">gf<\/code> in Vim and\nopen the file directly. You can get rid of that and just use <code class=\"highlighter-rouge\">src:\nfoo.conf.j2<\/code>. It is just a readability\/convenience tradeoff.<\/p>\n\n<p>The file name I use is the <strong>intended filename at the destination<\/strong>,\nappended with <code class=\"highlighter-rouge\">.j2<\/code> so it is clear that it is a Jinja2 template, and\neasier to search (<code class=\"highlighter-rouge\">find<\/code> or <code class=\"highlighter-rouge\">locate<\/code>).<\/p>\n\n<p>Some folks like to replicate the destination hierarchy (e.g. <code class=\"highlighter-rouge\">src: etc\/sysconfig\n\/network-scripts\/ifcfg-ethx.cfg.j2<\/code>). This is a matter of taste, but personaly I\ndon\u2019t see the point of having those deep hierarchies in the role if the naming is correct.<\/p>\n\n<h3 id=\"varsmainyml\"><code class=\"highlighter-rouge\">vars\/main.yml<\/code><\/h3>\n\n<p>It is sometimes difficult to grok the difference between\n<code class=\"highlighter-rouge\">vars\/main.yml<\/code> and <code class=\"highlighter-rouge\">defaults\/main.yml<\/code>. After all, they both contain\nvariable assignements.<\/p>\n\n<p>I do not always use a <code class=\"highlighter-rouge\">vars\/main.yml<\/code>, but when I do, I put \u201cconstants\nlike\u201d variables in it. These are variables that are not intended to be\noverriden.<\/p>\n\n<p>For instance the github repository for a particular piece of code (e.g.\nyour web application) will certainly go there. However, the version you\nwant to deploy won\u2019t.<\/p>\n\n<p>All in all, it is just a mechanism to take those values out of tasks files\nreadability and role life cycle.<\/p>\n\n<h2 id=\"inventories-and-playbook-layout\">Inventories and playbook layout<\/h2>\n\n<p>A playbook glues together roles and inventories. Thus playbooks depend on roles\nand inventories. But while you have mechanisms to list roles requirements in a\nplaybook, you don\u2019t have any for inventories.<\/p>\n\n<p>Since the playbook can not live without the targeted inventories I include my\ninventories in my playbooks.<\/p>\n\n<div class=\"highlighter-rouge\"><div class=\"highlight\"><pre class=\"highlight\"><code>playbook-foobar\/\n\u251c\u2500\u2500 ansible.cfg\n\u251c\u2500\u2500 requirements.txt\n\u251c\u2500\u2500 roles\/\n\u2502\u00a0\u00a0 \u2514\u2500\u2500 requirements.yml\n\u251c\u2500\u2500 inventories\n\u2502\u00a0\u00a0 \u251c\u2500\u2500 development\n\u2502\u00a0\u00a0 |   \u251c\u2500\u2500 group_vars\n\u2502\u00a0\u00a0 |   \u2502\u00a0\u00a0 \u2514\u2500\u2500 all\n\u2502\u00a0\u00a0 |   \u2514\u2500\u2500 hosts\n\u2502\u00a0\u00a0 \u251c\u2500\u2500 integration\n\u2502\u00a0\u00a0 |   \u251c\u2500\u2500 group_vars\n\u2502\u00a0\u00a0 |   \u2502\u00a0\u00a0 \u2514\u2500\u2500 all\n\u2502\u00a0\u00a0 |   \u2514\u2500\u2500 hosts\n\u2502\u00a0\u00a0 \u2514\u2500\u2500 production\n\u2502\u00a0\u00a0     \u251c\u2500\u2500 group_vars\n\u2502\u00a0\u00a0     \u2502\u00a0\u00a0 \u2514\u2500\u2500 all\n\u2502\u00a0\u00a0     \u2514\u2500\u2500 hosts\n\u251c\u2500\u2500 site.yml\n\u2514\u2500\u2500 playbooks\n \u00a0\u00a0 \u251c\u2500\u2500 10_database.yml\n \u00a0\u00a0 \u2514\u2500\u2500 20_stuff.yml\n<\/code><\/pre><\/div><\/div>\n\n<h3 id=\"ansiblecfg-roles-and-rolesrequirementsyml\"><code class=\"highlighter-rouge\">ansible.cfg<\/code>, <code class=\"highlighter-rouge\">roles\/<\/code> and <code class=\"highlighter-rouge\">roles\/requirements.yml<\/code><\/h3>\n\n<p>This file controles ansible behaviour. You can have one in <code class=\"highlighter-rouge\">\/etc\/ansible<\/code> or as\na personal dotfile (<code class=\"highlighter-rouge\">~\/.ansible.cfg<\/code>). Adding an <code class=\"highlighter-rouge\">ansible.cfg<\/code> file in the\nplaybook root will ensure that the required settings for the playbook to run are\nreally there. The precedence order for Ansible config files is<sup id=\"fnref:2\"><a href=\"#fn:2\" class=\"footnote\">2<\/a><\/sup>:<\/p>\n\n<ol>\n  <li><code class=\"highlighter-rouge\">ANSIBLE_CONFIG<\/code> (an environment variable pointing to a file)<\/li>\n  <li><code class=\"highlighter-rouge\">ansible.cfg<\/code> (in the current directory)<\/li>\n  <li><code class=\"highlighter-rouge\">.ansible.cfg<\/code> (in the home directory)<\/li>\n  <li><code class=\"highlighter-rouge\">\/etc\/ansible\/ansible.cfg<\/code><\/li>\n<\/ol>\n\n<p>Ansible will use the first config file found.<\/p>\n\n<p>In this config file, I always set at least two options:<\/p>\n\n<figure class=\"highlight\"><pre><code class=\"language-ini\" data-lang=\"ini\"><span class=\"py\">hostfile<\/span> <span class=\"p\">=<\/span> <span class=\"s\">.\/inventories\/dev<\/span>\n<span class=\"py\">roles_path<\/span> <span class=\"p\">=<\/span> <span class=\"s\">.\/roles:\/some\/path\/to\/roles\/repos<\/span><\/code><\/pre><\/figure>\n\n<p>The first one (<code class=\"highlighter-rouge\">hostfile<\/code>) sets which inventory Ansible will use. More\nexplanations will come below.<\/p>\n\n<p>The second one set the path where Ansible will look for roles. I typically set\ntwo directories here (separated by <code class=\"highlighter-rouge\">:<\/code>, like shell\u2019s <code class=\"highlighter-rouge\">PATH<\/code>):<\/p>\n\n<ul>\n  <li>the first directory will be used by ansible galaxy to install imported\nroles. I set it to <code class=\"highlighter-rouge\">.\/roles<\/code> but the name doesn\u2019t matter. Don\u2019t forget\nto add the directory content (except <code class=\"highlighter-rouge\">requirements.yml<\/code>) in your\nplaybook\u2019s <code class=\"highlighter-rouge\">.gitignore<\/code> like so:<\/li>\n<\/ul>\n\n<div class=\"highlighter-rouge\"><div class=\"highlight\"><pre class=\"highlight\"><code>!\/roles\n\/roles\/*\n!\/roles\/requirements.yml\n<\/code><\/pre><\/div><\/div>\n\n<ul>\n  <li>sometimes I add a second directory that points to my roles\ndevelopmenent directory path<\/li>\n<\/ul>\n\n<p>The advantages for this setup are two fold: first, you have a dedicated path,\nignored by your SCM, where you will download roles. The roles will be searched\nthere first. Secondly, if a role is not found, it will be searched in your role\ndevelopment directory. This let you hack on your roles while writing a playbook.\nYou don\u2019t need to go through a <em>commit\/push\/install<\/em> cycle when you are coding\nyour roles for this playbook.<\/p>\n\n<p>Roles dependencies for your playbook are listed in <code class=\"highlighter-rouge\">requirements.yml<\/code>\nand can be installed with <code class=\"highlighter-rouge\">ansible-galaxy install -r requirements.yml<\/code>:<\/p>\n\n<figure class=\"highlight\"><pre><code class=\"language-yaml\" data-lang=\"yaml\"><span class=\"c1\"># Role on galaxy<\/span>\n<span class=\"pi\">-<\/span> <span class=\"s\">you.rolename<\/span>\n<span class=\"c1\"># Public role on github<\/span>\n<span class=\"pi\">-<\/span> <span class=\"na\">name<\/span><span class=\"pi\">:<\/span> <span class=\"s\">role-public<\/span>\n  <span class=\"na\">src<\/span><span class=\"pi\">:<\/span> <span class=\"s\">https:\/\/github.com\/erasme\/role-public.git<\/span>\n<span class=\"c1\"># Private role on github<\/span>\n<span class=\"pi\">-<\/span> <span class=\"na\">name<\/span><span class=\"pi\">:<\/span> <span class=\"s\">role-private<\/span>\n  <span class=\"na\">src<\/span><span class=\"pi\">:<\/span> <span class=\"s\">git+ssh:\/\/git@github.com\/you\/role-private.git<\/span><\/code><\/pre><\/figure>\n\n<h3 id=\"inventories-1\"><code class=\"highlighter-rouge\">inventories\/<\/code><\/h3>\n\n<p>This directory holds all inventories you want to apply your playbook too. The\nmost common pattern is to use per-environment inventories: one for\n<code class=\"highlighter-rouge\">development<\/code>, one for <code class=\"highlighter-rouge\">integration<\/code>, another for <code class=\"highlighter-rouge\">production<\/code>, etc\u2026<\/p>\n\n<p>Of course, the <code class=\"highlighter-rouge\">hostfile<\/code> variable in <code class=\"highlighter-rouge\">ansible.cfg<\/code> should point to\n<code class=\"highlighter-rouge\">development<\/code> to avoid accidentaly messing with production. Executing the\nplaybook on non-development inventories will force you tu use the <code class=\"highlighter-rouge\">-i<\/code>, which is\na good safety measure.<\/p>\n\n<p>While you can define variables in groups (in <code class=\"highlighter-rouge\">group_vars<\/code>) and hosts\n(<code class=\"highlighter-rouge\">host_vars<\/code>), you should stuff as much variables as possible in\n<code class=\"highlighter-rouge\">group_vars\/all<\/code>. The rationale is that it is much easier to find a\nvariable when a single file is involved. Variables scattered in a dozen\nof files are <em>not<\/em> manageable.<\/p>\n\n<p>And when you\u2019ll want to create an additional inventory (e.g. create <code class=\"highlighter-rouge\">production<\/code>\nfrom <code class=\"highlighter-rouge\">development<\/code>), it will be much easier to change a single file and set the\nvariables to proper values than to do the same in several files.<\/p>\n\n<p>Note that <code class=\"highlighter-rouge\">group_vars\/all<\/code> can be a directory containing several files. I\nusually split variables in a clear text file (<code class=\"highlighter-rouge\">group_vars\/all\/all<\/code>) and a\nciphered one (<code class=\"highlighter-rouge\">group_vars\/all_secret<\/code>) using the transparent vaulting techniques\ndescribed in \n<a href=\"\/articles\/transparent-vault-revisited\/\">this post<\/a>.<\/p>\n\n<p><strong>Note 3 years later<\/strong>: now ansible allow you to vault a single variable in\nan inventory. Use it !<\/p>\n\n<p>Here is a handy bash alias that crypts text selected with a mouse:<\/p>\n\n<p><code class=\"highlighter-rouge\">alias vault_clip_crypt='echo Passing $(xclip -rmlastnl -o) to ansible-vault &amp;&amp; echo -n \"$(xclip -rmlastnl -o)\" | ansible-vault encrypt_string'<\/code><\/p>\n\n<h3 id=\"siteyml-and-playbooks\"><code class=\"highlighter-rouge\">site.yml<\/code> and <code class=\"highlighter-rouge\">playbooks\/<\/code><\/h3>\n\n<p>This directory contains the playbooks themselves. I always create a\n\u201cmaster\u201d playbook called <code class=\"highlighter-rouge\">site.yml<\/code> in the playbook root directory,\nwhich includes all other playbooks located in <code class=\"highlighter-rouge\">playbooks\/<\/code>.<\/p>\n\n<p>I prefix playbooks with a number (Basic style) so I get a sense of the\norder playbook will be executed just by looking at <code class=\"highlighter-rouge\">playbooks\/<\/code> content.<\/p>\n\n<p>For instance:<\/p>\n\n<figure class=\"highlight\"><pre><code class=\"language-yaml\" data-lang=\"yaml\"><span class=\"c1\">#!\/usr\/bin\/env ansible-playbook<\/span>\n\n<span class=\"pi\">-<\/span> <span class=\"na\">import_playbook<\/span><span class=\"pi\">:<\/span> <span class=\"s\">playbooks\/10_database.yml<\/span>\n<span class=\"pi\">-<\/span> <span class=\"na\">import_playbook<\/span><span class=\"pi\">:<\/span> <span class=\"s\">playbooks\/20_stuff.yml<\/span><\/code><\/pre><\/figure>\n\n<p>The rationale is to be able to use <code class=\"highlighter-rouge\">ansible-pull<\/code> easily if needed (<code class=\"highlighter-rouge\">ansible-\npull<\/code>, by default, tries to execute a playbook called<code class=\"highlighter-rouge\">site.yml<\/code>). The other\npoint is to split playbook in related parts.<\/p>\n\n<p>For instance, you could have a playbook the takes care of setting up the\ndatabase, another that will set the OS level stuff (e.g. ssh keys, firewalling,\n\u2026), another one that takes care of deploying your web application, etc\u2026 When\nneeded, You can use all the playbooks at once with <code class=\"highlighter-rouge\">site.yml<\/code>, or just focus on\na specific problem running the appropriate playbook (no need to run the ssh-key\nsetup if you\u2019re just deploying the latest version of your web application).<\/p>\n\n<p>The shebang line at the top of the file (<code class=\"highlighter-rouge\">#!\/usr\/bin\/env ansible-playbook<\/code>) will\nmake the playbook directly executable (adjust <code class=\"highlighter-rouge\">ansible-playbook<\/code> path and <code class=\"highlighter-rouge\">chmod\n+x<\/code> the playbook file). You can still pass additional\n<code class=\"highlighter-rouge\">ansible-playbook<\/code> parameters if required.<\/p>\n\n<h3 id=\"requirementstxt\"><code class=\"highlighter-rouge\">requirements.txt<\/code><\/h3>\n\n<p>This file contains the result of a <code class=\"highlighter-rouge\">pip freeze<\/code>. I now only use <code class=\"highlighter-rouge\">pip<\/code>\nunder <code class=\"highlighter-rouge\">virtualenv<\/code> to install ansible and required modules. It makes it\nreally easy to switch ansible (and even python) version between\nprojects.<\/p>\n\n<p>So when someone needs to work on this project, the workflow is simple:<\/p>\n\n<div class=\"language-bash highlighter-rouge\"><div class=\"highlight\"><pre class=\"highlight\"><code>git clone http:\/\/github.com\/some\/playbook-repos\n<span class=\"nb\">cd <\/span>playbook-repos\nmkvirtualenv playbook-repos <span class=\"nt\">--no-site-packages<\/span>\npip install <span class=\"nt\">-r<\/span> requirements.txt\nansible-galaxy install <span class=\"nt\">-r<\/span> roles\/requirements.yml\n<\/code><\/pre><\/div><\/div>\n\n<p>and you\u2019re good to go.<\/p>\n\n<h2 id=\"layout-antipatterns\">Layout Antipatterns<\/h2>\n\n<p>When I started using Ansible, I cumulated several antipatterns at the\nsame time: trying to emcompass all my infrastructure in a single\ninventory containing per-host fine grained variables, used in a single\nplaybook, without using any role.<\/p>\n\n<p>While this sounds feasible, it is doomed to failure unless you manage a\nvery small infrastructure. Let\u2019s zoom in briefly on each mistake.<\/p>\n\n<h3 id=\"trying-to-encompass-all-your-infrastructure-in-one-playbook\">Trying to encompass all your infrastructure in one playbook<\/h3>\n\n<p>Is is tempting to aim for a one-liner that will magically deploy all\nyour infrastructure in one shot. This gives you some bragging rights at\nyour next meetup, and feels like the ultimate sysadmin masterpiece.<\/p>\n\n<p>However, it has many drawbacks:<\/p>\n\n<ul>\n  <li>\n    <p>it will be slow: do you really want to run a playbook over dozens of\nmore tasks or roles, just to change an entry in <code class=\"highlighter-rouge\">\/etc\/hosts<\/code> ? Yes,\nthere are workaround for this, but it will require some command line\nmagic, a lot of thinking.<\/p>\n  <\/li>\n  <li>\n    <p>it mixes bananas and apples: you should strive for separation of\nconcerns in your playbooks if you want be able to read them (and, as a\nconsequence, maintain them).<\/p>\n  <\/li>\n<\/ul>\n\n<p>As a consequence, your infrastructure code will be unnecessary hard to\ntest and maintain.<\/p>\n\n<h3 id=\"per-host-fine-grained-variables\">Per-host fine grained variables<\/h3>\n\n<p>This is a corolary of the previous antipattern: when you try to\nencompass your whole infrastructure, you start to think, inheritance,\nvariables overriding and refining.<\/p>\n\n<p>And while doing this, you add considerable complexity to your\ninventories. It is very hard to track down variables definitions when\nyou overrides them in <code class=\"highlighter-rouge\">group_vars\/some_group<\/code>, <code class=\"highlighter-rouge\">group_vars\/all<\/code>,\n<code class=\"highlighter-rouge\">hosts_vars\/machine<\/code>, role defaults, \u2026<\/p>\n\n<p>Now this can get even worse when you use the <code class=\"highlighter-rouge\">hash_behavior: merge<\/code>\nAnsible configuration setting: it introduces more confusion, and makes\nyour Ansible work potentially unshareable with people using\n<code class=\"highlighter-rouge\">hash_behaviour: replace<\/code>. Since I am\n<a href=\"https:\/\/github.com\/ansible\/ansible\/commit\/e28e538c6ed7520ecef305c776eb6036aff42d06\">guilty<\/a>\non this one, it is time to make some apologies. Sorry folks. Michael\nDeHaan did not like it, and he was right.<\/p>\n\n<h3 id=\"single-playbook\">Single playbook<\/h3>\n\n<p>A single playbook relates to the first Sin again, but also applies to\nmore focused playbooks where you only deploy one thing. Splitting your\nplaybooks between various logically related roles will fasten your\ndeployments. Again, why running ssh key distribution, storage cluster\ndeployment, web stack, middlewares and application when you just change\nthe color of a button in your web app ?<\/p>\n\n<p>Split your playbook in related parts that reflects your stack\narchitecture. They will be faster and easier to use.<\/p>\n\n<h3 id=\"no-roles-tasks-only\">No roles (tasks only)<\/h3>\n\n<p>Well, this is obvious. Even if you don\u2019t want to share, make roles and\nstrive for code reuse. Reused code will save you time of course, but it\nis also battle tested since it is used more frequently.<\/p>\n\n<p>Tasks-only playbook can be used for a quick hit and run, solving a\ntransient problem that doesn\u2019t offer any code reuse opportunities.<\/p>\n\n<p>I also try to avoid tasks along roles in playbooks: this hurts the\nabstraction level you manage to build using roles. When thinking in\nterms of roles, you don\u2019t need to think about the nitty gritty details\nof the roles when reading your playbooks. If your roles are thouroughly\ntested, you can read your infrastructure in seconds. Add tasks to the\nmix, and you loose this superpower.<\/p>\n\n<div class=\"footnotes\">\n  <ol>\n    <li id=\"fn:1\">\n      <p>Yes, you could separate your application server (e.g. php-fpm) and put it on a different machine than your webserver, it ll depends on your local context.\u00a0<a href=\"#fnref:1\" class=\"reversefootnote\">&#8617;<\/a><\/p>\n    <\/li>\n    <li id=\"fn:2\">\n      <p>http:\/\/docs.ansible.com\/intro_configuration.html\u00a0<a href=\"#fnref:2\" class=\"reversefootnote\">&#8617;<\/a><\/p>\n    <\/li>\n  <\/ol>\n<\/div>\n\n    <p><a href=\"https:\/\/leucos.github.io\/ansible-files-layout\">Laying out roles, inventories and playbooks<\/a> was originally published by Michel Blanc at <a href=\"https:\/\/leucos.github.io\">Random stuff<\/a> on July 02, 2015.<\/p>\n  "},{"title":{"@attributes":{"type":"html"}},"link":{"@attributes":{"rel":"alternate","type":"text\/html","href":"https:\/\/leucos.github.io\/decoupling-your-ansible-roles"}},"id":"https:\/\/leucos.github.io\/decoupling-your-ansible-roles","published":"2015-06-27T00:00:00+00:00","updated":"2015-06-27T00:00:00+00:00","author":{"name":"Michel Blanc","uri":"https:\/\/leucos.github.io","email":"mb@mbnet.fr"},"content":"\n    <p>Having tightly coupled role is the best way to have a hard time\nmaintaining roles and playbooks, and live in fear of changing anything\nin them.<\/p>\n\n<p>Here is a journey into role decoupling.<\/p>\n\n<h2 id=\"the-problem\">The problem<\/h2>\n\n<p>Let say we have a role (<em>my_app<\/em>) that depend on <em>php-fpm<\/em> role. In the php-fpm\nrole, we want to display errors in HTML output depending on the application\nrunning environment (e.g. always display unless we\u2019re running in production\nenvironment).<\/p>\n\n<p>The application running environment is available in <code class=\"highlighter-rouge\">myapp_environment<\/code>.<\/p>\n\n<h2 id=\"first-idea\">First idea<\/h2>\n\n<p>The first idea that comes to mind is to change the php.ini according to\n<code class=\"highlighter-rouge\">myapp_environment<\/code>, like so:<\/p>\n\n<figure class=\"highlight\"><pre><code class=\"language-jinja\" data-lang=\"jinja\"><span class=\"cp\">{%<\/span> <span class=\"k\">if<\/span> <span class=\"nv\">myapp_environment<\/span> <span class=\"o\">==<\/span> <span class=\"s2\">\"production\"<\/span> <span class=\"cp\">%}<\/span>\ndisplay_errors = Off\n<span class=\"cp\">{%<\/span> <span class=\"k\">else<\/span> <span class=\"cp\">%}<\/span>\ndisplay_errors = On\n<span class=\"cp\">{%<\/span> <span class=\"k\">endif<\/span> <span class=\"cp\">%}<\/span><\/code><\/pre><\/figure>\n\n<p>The problem with this approach if that php-fpm role now needs\n<code class=\"highlighter-rouge\">myapp_environment<\/code> to be defined, which is quite absurd.<\/p>\n\n<p>So instead, you could rename the variable <code class=\"highlighter-rouge\">environment<\/code>, and use this in\nboth roles (<code class=\"highlighter-rouge\">myapp<\/code> and <code class=\"highlighter-rouge\">php-fpm<\/code>). This is better, but not much. The\nproblem with this approach is that a plain <code class=\"highlighter-rouge\">environment<\/code> variable is not\nlinked (by it\u2019s name) to any role, and this can lead to great confusion\nis it is used and set in many roles or in different places in the\ninventory.<\/p>\n\n<h2 id=\"another-try\">Another try<\/h2>\n\n<p>So the best way is to have two variables, <code class=\"highlighter-rouge\">php_fpm_environment<\/code> and <code class=\"highlighter-rouge\">myapp_environment<\/code> which makes is meaningful. But now how can I sync\nthem together ?<\/p>\n\n<p>One ways is to match them in your inventory, like so :<\/p>\n\n<figure class=\"highlight\"><pre><code class=\"language-yaml\" data-lang=\"yaml\"><span class=\"c1\"># Somewhere in inventory<\/span>\n<span class=\"na\">myapp_environment<\/span><span class=\"pi\">:<\/span> <span class=\"s2\">\"<\/span><span class=\"s\">production\"<\/span>\n<span class=\"na\">php_fpm_environment<\/span><span class=\"pi\">:<\/span> <span class=\"s2\">\"<\/span><span class=\"s\">{{<\/span><span class=\"nv\"> <\/span><span class=\"s\">myapp_environment<\/span><span class=\"nv\"> <\/span><span class=\"s\">}}\"<\/span><\/code><\/pre><\/figure>\n\n<p>However, this has some drawbacks. For instance, we are still talking about\n<code class=\"highlighter-rouge\">php_fpm_environment<\/code> and, while not a big deal, it has no php-fpm\nmeaning per se and it is not obvious what this variable does.<\/p>\n\n<p>Also, in the <code class=\"highlighter-rouge\">php.ini<\/code> template, we will still have to test against the string\n\u201cproduction\u201d to set <code class=\"highlighter-rouge\">display_errors<\/code>. Testing against a string set somewhere\nelse is quite dangerous. What is the production name for the app is \u201clive\u201d\ninstead ? Our php-fpm role is broken now.<\/p>\n\n<h2 id=\"some-progress\">Some progress<\/h2>\n\n<p>We could go a better way: let\u2019s call the variable <code class=\"highlighter-rouge\">php_fpm_display_error<\/code> (more\nmeaningful) and make it a boolean. We now can do this :<\/p>\n\n<figure class=\"highlight\"><pre><code class=\"language-jinja\" data-lang=\"jinja\"><span class=\"cp\">{%<\/span> <span class=\"k\">if<\/span> <span class=\"nv\">php_fpm_display_errors<\/span> <span class=\"cp\">%}<\/span>\ndisplay_errors = On\n<span class=\"cp\">{%<\/span> <span class=\"k\">else<\/span> <span class=\"cp\">%}<\/span>\ndisplay_errors = Off\n<span class=\"cp\">{%<\/span> <span class=\"k\">endif<\/span> <span class=\"cp\">%}<\/span><\/code><\/pre><\/figure>\n\n<p>and somewhere in inventory:<\/p>\n\n<figure class=\"highlight\"><pre><code class=\"language-yaml\" data-lang=\"yaml\"><span class=\"na\">myapp_environment<\/span><span class=\"pi\">:<\/span> <span class=\"s2\">\"<\/span><span class=\"s\">production\"<\/span>\n<span class=\"na\">php_fpm_display_errors<\/span><span class=\"pi\">:<\/span> <span class=\"s2\">\"<\/span><span class=\"s\">{{<\/span><span class=\"nv\"> <\/span><span class=\"s\">myapp_environment<\/span><span class=\"nv\"> <\/span><span class=\"s\">==<\/span><span class=\"nv\"> <\/span><span class=\"s\">'production'<\/span><span class=\"nv\"> <\/span><span class=\"s\">}}\"<\/span><\/code><\/pre><\/figure>\n\n<h2 id=\"streamlining-our-solution\">Streamlining our solution<\/h2>\n\n<p>Well, this is better now. But it is not perfect. The inventory is more verbose\nthan required and handles something that it shouldn\u2019t have to take care\nof. It is also quite easy to forget to add it to the inventory and end up\nwith errors showing in production.<\/p>\n\n<p>By moving this logic away from the inventory, and directly in the role\ndependencies, this configuration setting becomes completely transparent, and we get rid of redundancy. We just have to add the following lines in <code class=\"highlighter-rouge\">myapp\/meta\/main.yml<\/code>:<\/p>\n\n<figure class=\"highlight\"><pre><code class=\"language-yaml\" data-lang=\"yaml\"><span class=\"na\">dependencies<\/span><span class=\"pi\">:<\/span>\n  <span class=\"pi\">-<\/span> <span class=\"na\">role<\/span><span class=\"pi\">:<\/span> <span class=\"s\">role-php-fpm<\/span>\n    <span class=\"na\">php_fpm_display_errors<\/span><span class=\"pi\">:<\/span> <span class=\"s2\">\"<\/span><span class=\"s\">{{<\/span><span class=\"nv\"> <\/span><span class=\"s\">myapp_environment<\/span><span class=\"nv\"> <\/span><span class=\"s\">==<\/span><span class=\"nv\"> <\/span><span class=\"s\">'production'<\/span><span class=\"nv\"> <\/span><span class=\"s\">}}\"<\/span><\/code><\/pre><\/figure>\n\n<p>Now, php-fpm role is completely decoupled from myapp role, and the production\nsetting is completely transparent to the role user. Setting <code class=\"highlighter-rouge\">myapp_environment<\/code>\nis enough to have the depending role set variables accordingly. You don\u2019t even\nhave to be aware of the <code class=\"highlighter-rouge\">myapp<\/code> role dependency. If you swap, let say\nnginx\/php-fpm for apache\/php, you just have to change the role dependency and\nhave no impact on your inventory. If you want to name your production\nenvironment \u201clive\u201d, you can do so by changing <code class=\"highlighter-rouge\">meta\/main.yml<\/code> and not\ntouching anything else.<\/p>\n\n<p>Keeping role decoupled is the best way to have manageable and reusable\nroles. Try to make them self sufficient, and avoid cross variables or\neven worse, group names in roles !<\/p>\n\n\n    <p><a href=\"https:\/\/leucos.github.io\/decoupling-your-ansible-roles\">Decoupling your Ansible roles<\/a> was originally published by Michel Blanc at <a href=\"https:\/\/leucos.github.io\">Random stuff<\/a> on June 27, 2015.<\/p>\n  "},{"title":{"@attributes":{"type":"html"}},"link":{"@attributes":{"rel":"alternate","type":"text\/html","href":"https:\/\/leucos.github.io\/articles\/transparent-vault-revisited\/"}},"id":"https:\/\/leucos.github.io\/articles\/transparent-vault-revisited","updated":"2015-05-25T00:00:00-00:00","published":"2015-05-26T00:00:00+00:00","author":{"name":"Michel Blanc","uri":"https:\/\/leucos.github.io","email":"mb@mbnet.fr"},"content":"\n    <h2 id=\"doing-it-the-wrong-way\">Doing it the wrong way<\/h2>\n\n<p><a href=\"\/articles\/ansible-transparent-vault\/\">Last attempt<\/a>\nto make ansible vault encryption\/decryption transparent wasn\u2019t quite\nright. Decrypting files after commit wasn\u2019t a good idea as\n<a href=\"https:\/\/github.com\/ralovely\">Raphael Campardou<\/a> noticed.<\/p>\n\n<p>In search for a better idea, I eventually realized that hooks where not\nthe right place to do it: yes, you can guard from commiting files that\nshould be encrypted, but hacking around hooks to build a crypt\/decrypt\npipeline is doomed to failure.<\/p>\n\n<h2 id=\"doing-it-better\">Doing it better<\/h2>\n\n<p>While looking for alternate ways, I remembered I hacked around with\ngit filters back in the days to see clear-text diffs for OpenOffice\nfiles.<\/p>\n\n<p>Git let\u2019s you apply <code class=\"highlighter-rouge\">smudge<\/code>, <code class=\"highlighter-rouge\">clean<\/code> and <code class=\"highlighter-rouge\">textconv<\/code> filters to files\nwhich are applied this way:<\/p>\n\n<ul>\n  <li>filter\/smudge: after checkout, reads blob from STDIN and outputs the\nworkfile from STDOUT<\/li>\n  <li>filter\/clean: converts the worktree file to blob upon check in<\/li>\n  <li>diff\/textconv: applied before diffing files<\/li>\n<\/ul>\n\n<p>So, for our needs, <em>smudge<\/em> and <em>textconv<\/em> are good places to decrypt,\nwhile <em>clean<\/em> is the place to encrypt.<\/p>\n\n<h2 id=\"implementation\">Implementation<\/h2>\n\n<p>The implementation requires to write the 3 filters (<em>smudge<\/em>, <em>clean<\/em>,\n<em>textconv<\/em>) and configure your git repos to use the filters.<\/p>\n\n<p>Those filters should be executable.<\/p>\n\n<p>As we did in last post, we will use a <code class=\"highlighter-rouge\">.vault_password<\/code> file in the\nproject root directory containing the vault key (don\u2019t forget to add it\nto your <code class=\"highlighter-rouge\">.gitignore<\/code> file !). The filters fail if the file is not\npresent.<\/p>\n\n<h3 id=\"smudge\">Smudge<\/h3>\n\n<p>The problem that came up to write the smudge &amp; clean filters is that the\nblob content is fed on STDIN, and <code class=\"highlighter-rouge\">ansible-vault<\/code> can only\nencrypt\/decrypt files <em>in-place<\/em>.<\/p>\n\n<p>So we have to write the blob in a temporary file. While this is not\nreally a problem for the smudge filter, it is for the clean filter since\nthe temporary file contains the clear-text version of the file. The temp\nfile is created with restricted permissions, but you\u2019ve been warned.<\/p>\n\n<p>Smudge\u2019s filter job is simple:<\/p>\n<ul>\n  <li>write STDIN content to temp file<\/li>\n  <li>decrypt the temp file and swallow the output in a variable (using\n<code class=\"highlighter-rouge\">ansible-vault view<\/code> after setting the PAGER to <code class=\"highlighter-rouge\">cat<\/code>)<\/li>\n  <li>if the file was a vault encrypted file, display the variable, else,\nbail out.<\/li>\n<\/ul>\n\n<figure class=\"highlight\"><pre><code class=\"language-bash\" data-lang=\"bash\"><span class=\"c\">#!\/bin\/sh<\/span>\n\n<span class=\"k\">if<\/span> <span class=\"o\">[<\/span> <span class=\"o\">!<\/span> <span class=\"nt\">-r<\/span> <span class=\"s1\">'.vault_password'<\/span> <span class=\"o\">]<\/span><span class=\"p\">;<\/span> <span class=\"k\">then\n  <\/span><span class=\"nb\">exit <\/span>1\n<span class=\"k\">fi\n\n<\/span><span class=\"nv\">tmp<\/span><span class=\"o\">=<\/span><span class=\"sb\">`<\/span>mktemp<span class=\"sb\">`<\/span>\n<span class=\"nb\">cat<\/span> <span class=\"o\">&gt;<\/span> <span class=\"nv\">$tmp<\/span>\n\n<span class=\"nb\">export <\/span><span class=\"nv\">PAGER<\/span><span class=\"o\">=<\/span><span class=\"s1\">'cat'<\/span>\n<span class=\"nv\">CONTENT<\/span><span class=\"o\">=<\/span><span class=\"sb\">`<\/span>ansible-vault view <span class=\"s2\">\"<\/span><span class=\"nv\">$tmp<\/span><span class=\"s2\">\"<\/span> <span class=\"nt\">--vault-password-file<\/span><span class=\"o\">=<\/span>.vault_password 2&gt; \/dev\/null<span class=\"sb\">`<\/span>\n\n<span class=\"k\">if <\/span><span class=\"nb\">echo<\/span> <span class=\"nv\">$CONTENT<\/span> | <span class=\"nb\">grep<\/span> <span class=\"s1\">'ERROR: data is not encrypted'<\/span> <span class=\"o\">&gt;<\/span> \/dev\/null<span class=\"p\">;<\/span> <span class=\"k\">then\n  <\/span><span class=\"nb\">echo<\/span> <span class=\"s2\">\"Looks like one file was commited clear text\"<\/span>\n  <span class=\"nb\">echo<\/span> <span class=\"s2\">\"Please fix this before continuing !\"<\/span>\n  <span class=\"nb\">exit <\/span>1\n<span class=\"k\">else\n  <\/span><span class=\"nb\">echo<\/span> <span class=\"nv\">$CONTENT<\/span>\n<span class=\"k\">fi\n\n<\/span>rm <span class=\"nv\">$tmp<\/span><\/code><\/pre><\/figure>\n\n<p>As you guessed, <code class=\"highlighter-rouge\">ansible-vault<\/code> does not output errors on STDERR but on\nSTDOUT.<\/p>\n\n<h3 id=\"clean\">Clean<\/h3>\n\n<p>The clean filter works almost the same way:<\/p>\n<ul>\n  <li>write STDIN to a temp file<\/li>\n  <li>encrypt the temp file in place<\/li>\n  <li>write the temp file to STDOUT<\/li>\n<\/ul>\n\n<figure class=\"highlight\"><pre><code class=\"language-bash\" data-lang=\"bash\"><span class=\"c\">#!\/bin\/sh<\/span>\n\n<span class=\"k\">if<\/span> <span class=\"o\">[<\/span> <span class=\"o\">!<\/span> <span class=\"nt\">-r<\/span> <span class=\"s1\">'.vault_password'<\/span> <span class=\"o\">]<\/span><span class=\"p\">;<\/span> <span class=\"k\">then\n  <\/span><span class=\"nb\">exit <\/span>1\n<span class=\"k\">fi\n\n<\/span><span class=\"nv\">tmp<\/span><span class=\"o\">=<\/span><span class=\"sb\">`<\/span>mktemp<span class=\"sb\">`<\/span>\n<span class=\"nb\">cat<\/span> <span class=\"o\">&gt;<\/span> <span class=\"nv\">$tmp<\/span>\n\nansible-vault encrypt <span class=\"nv\">$tmp<\/span> <span class=\"nt\">--vault-password-file<\/span><span class=\"o\">=<\/span>.vault_password <span class=\"o\">&gt;<\/span> \/dev\/null 2&gt;&amp;1\n\n<span class=\"nb\">cat<\/span> <span class=\"s2\">\"<\/span><span class=\"nv\">$tmp<\/span><span class=\"s2\">\"<\/span>\nrm <span class=\"nv\">$tmp<\/span><\/code><\/pre><\/figure>\n\n<p>This one was quite easy. We could also use modelines, by encrypting only if\n\u201cvault: true\u201d is present in the 4 first lines. This way, we could apply\nthe filters to all the files. However I ditched the idea for performance\nreasons (see below).<\/p>\n\n<h3 id=\"diff-filter\">Diff filter<\/h3>\n\n<p>The filter works like the smudge filter except that it uses the file\nname passed as a parameter.<\/p>\n\n<figure class=\"highlight\"><pre><code class=\"language-bash\" data-lang=\"bash\"><span class=\"c\">#!\/bin\/sh<\/span>\n\n<span class=\"k\">if<\/span> <span class=\"o\">[<\/span> <span class=\"o\">!<\/span> <span class=\"nt\">-r<\/span> <span class=\"s1\">'.vault_password'<\/span> <span class=\"o\">]<\/span><span class=\"p\">;<\/span> <span class=\"k\">then\n  <\/span><span class=\"nb\">exit <\/span>1\n<span class=\"k\">fi\n\n<\/span><span class=\"nb\">export <\/span><span class=\"nv\">PAGER<\/span><span class=\"o\">=<\/span><span class=\"s1\">'cat'<\/span>\n<span class=\"nv\">CONTENT<\/span><span class=\"o\">=<\/span><span class=\"sb\">`<\/span>ansible-vault view <span class=\"s2\">\"<\/span><span class=\"nv\">$1<\/span><span class=\"s2\">\"<\/span> <span class=\"nt\">--vault-password-file<\/span><span class=\"o\">=<\/span>.vault_password 2&gt; \/dev\/null<span class=\"sb\">`<\/span>\n\n<span class=\"k\">if <\/span><span class=\"nb\">echo<\/span> <span class=\"s2\">\"<\/span><span class=\"nv\">$CONTENT<\/span><span class=\"s2\">\"<\/span> | <span class=\"nb\">grep<\/span> <span class=\"s1\">'ERROR: data is not encrypted'<\/span> <span class=\"o\">&gt;<\/span> \/dev\/null<span class=\"p\">;<\/span> <span class=\"k\">then\n  <\/span><span class=\"nb\">cat<\/span> <span class=\"s2\">\"<\/span><span class=\"nv\">$1<\/span><span class=\"s2\">\"<\/span>\n<span class=\"k\">else\n  <\/span><span class=\"nb\">echo<\/span> <span class=\"s2\">\"<\/span><span class=\"nv\">$CONTENT<\/span><span class=\"s2\">\"<\/span>\n<span class=\"k\">fi<\/span><\/code><\/pre><\/figure>\n\n<h3 id=\"git-configuration\">Git configuration<\/h3>\n\n<h4 id=\"attributes\">Attributes<\/h4>\n\n<p>Now that the various filters are out and chmoded +x, we need to set-up\nout git repos to use them.<\/p>\n\n<p>For this, we need to tell git on which files we want to apply the\nfilters, using a <code class=\"highlighter-rouge\">.gitattributes<\/code> file in our project top directory.<\/p>\n\n<p>The following <code class=\"highlighter-rouge\">.gitattributes<\/code> file<\/p>\n\n<figure class=\"highlight\"><pre><code class=\"language-text\" data-lang=\"text\">*_vault* filter=vault diff=vault<\/code><\/pre><\/figure>\n\n<p>will run filters on repository blobs\/files that match <code class=\"highlighter-rouge\">*_vault*<\/code>.<\/p>\n\n<p>I initially intended to run the filters on all files, using modelines.\nHowever, performance was really bad, so I finally ended up removing a\nfull wildcard (<code class=\"highlighter-rouge\">*<\/code>) and restrict filter selection to specific files.\nYou can repeat the lines ad nauseam if you want to catch multiple\nfileglobs.<\/p>\n\n<h4 id=\"gitconfig\">Gitconfig<\/h4>\n\n<p>I put my filters in <code class=\"highlighter-rouge\">~\/.bin\/<\/code>, but the location doesn\u2019t matter. You can\nevent add them to the project and commit them, so everyone has them.<\/p>\n\n<p>The following section needs to be added to the project\u2019s <code class=\"highlighter-rouge\">.git\/config<\/code>\nfile:<\/p>\n\n<figure class=\"highlight\"><pre><code class=\"language-text\" data-lang=\"text\">[filter \"vault\"]\n  smudge = ~\/.bin\/smudge_vault\n  clean  = ~\/.bin\/clean_vault\n\n[diff \"vault\"]\n  textconv = ~\/.bin\/diff_vault\n <\/code><\/pre><\/figure>\n\n<h3 id=\"test\">Test<\/h3>\n\n<p>Adding a file that matches a glob in <code class=\"highlighter-rouge\">.gitattributes<\/code> should now trigger\ntransparent encryption.<\/p>\n\n<p>Here is a sample transcript.<\/p>\n\n<script type=\"text\/javascript\" src=\"https:\/\/asciinema.org\/a\/7oaviuh8v2pi39zeojxrn8434.js\" id=\"asciicast-7oaviuh8v2pi39zeojxrn8434\" async=\"\"><\/script>\n\n<h3 id=\"big-fat-warning\">Big fat warning<\/h3>\n\n<p>The <code class=\"highlighter-rouge\">git cat-file<\/code> part is not here for decoration. At least the first\ntime, ensure that encryption works.<\/p>\n\n<h3 id=\"the-filters\">The filters<\/h3>\n\n<script src=\"https:\/\/gist.github.com\/leucos\/1bfcfc7252e8c262956e.js\"> <\/script>\n\n\n    <p><a href=\"https:\/\/leucos.github.io\/articles\/transparent-vault-revisited\/\">Transparent encryption with ansible vault revisited<\/a> was originally published by Michel Blanc at <a href=\"https:\/\/leucos.github.io\">Random stuff<\/a> on May 26, 2015.<\/p>\n  "},{"title":{"@attributes":{"type":"html"}},"link":{"@attributes":{"rel":"alternate","type":"text\/html","href":"https:\/\/leucos.github.io\/articles\/ansible-transparent-vault\/"}},"id":"https:\/\/leucos.github.io\/articles\/ansible-transparent-vault","updated":"2015-05-25T00:00:00-00:00","published":"2015-05-25T00:00:00+00:00","author":{"name":"Michel Blanc","uri":"https:\/\/leucos.github.io","email":"mb@mbnet.fr"},"content":"\n    <h1 id=\"big-fat-warning\">Big Fat Warning<\/h1>\n\n<p><strong>THIS FILE IS LEFT HERE FOR REFERENCE<\/strong><\/p>\n\n<p>However, the method described here is WRONG. Check out \n<a href=\"\/articles\/transparent-vault-revisited\/\">next post<\/a> instead !<\/p>\n\n<h2 id=\"pain-points\">Pain points<\/h2>\n\n<p><code class=\"highlighter-rouge\">ansible-vault<\/code> is handy. You can crypt your stuff before commiting it so your\nprivate stuff (AWS\/DigitalOcean\/\u2026 keys, passwords, \u2026) don\u2019t end up\nworld-readable on GitHub.<\/p>\n\n<p>However, it is too easy to decrypt your stuff, forget about it, and commit it \nwithout encrypting it back. It is also quite tedious to ansible-vault\nencrypt\/decrypt all day long.<\/p>\n\n<h2 id=\"solution\">Solution<\/h2>\n\n<p><a href=\"https:\/\/github.com\/ralovely\">Raphael Campardou<\/a> proposed a <a href=\"https:\/\/gist.github.com\/ralovely\/9367737\">nice\nsolution<\/a> to prevent commiting\nansible vault files.<\/p>\n\n<p>In his solution, you have to name your files <code class=\"highlighter-rouge\">*_vault.yml<\/code> so they get busted\nby a pre-commit hook if they are not currently encrypted.<\/p>\n\n<p>This is nice: by naming your files appropriately, you can not commit them unless\nthey are ansible-vault crypted beforehand.<\/p>\n\n<p>I extended his idea so it can apply to any file in an Ansible repository, with\nvery little configuration, and added a post-commit hook so files gets\ntransparently decrypted after being commited.<\/p>\n\n<h2 id=\"transparent-encryptiondecryption\">Transparent encryption\/decryption<\/h2>\n\n<p>The goal is simple: automagically encrypt the proper files before commit,\ncommit them, then decrypt them afterwards so we can hack again without\nany manual intervention. All this with minimal configuration.<\/p>\n\n<h3 id=\"marking-file-for-encryption\">Marking file for encryption<\/h3>\n\n<p>The center trick is to find a way to mark a file for encryption. Modelines\n(a.k.a. emacs local variable lines) to the rescue.<\/p>\n\n<p>To tell git hooks that a file requires encryption, we\u2019ll add this line to\nthe top of the file (or on line 2 if the file already has a shebang\nline) :<\/p>\n\n<div class=\"highlighter-rouge\"><div class=\"highlight\"><pre class=\"highlight\"><code># -*- vault: true; -*-\n<\/code><\/pre><\/div><\/div>\n\n<p>Any file having <code class=\"highlighter-rouge\">vault: true<\/code> in a modeline is set to <strong>require encryption before\ncommit<\/strong>.<\/p>\n\n<p>The icing on the cake is that you can use this modeline to set the filetype\ntoo[1], and help your editor to find out the proper file content, which is\nquite handy with some files not ending in <code class=\"highlighter-rouge\">yml<\/code>:<\/p>\n\n<div class=\"highlighter-rouge\"><div class=\"highlight\"><pre class=\"highlight\"><code><span class=\"c1\"># -*- mode: yaml; vault: true; -*-<\/span>\n<\/code><\/pre><\/div><\/div>\n\n<p>This is supported out of the box by vim and Emacs. If you use SublimeText,\nyou can use the <a href=\"https:\/\/github.com\/kvs\/STEmacsModelines\">STEmacsModelines<\/a>\npackage.<\/p>\n\n<h3 id=\"using-the-hooks\">Using the hooks<\/h3>\n\n<p>The pre-commit hook will encrypt files marked with <code class=\"highlighter-rouge\">vault: true<\/code>. If a\n<code class=\"highlighter-rouge\">.vault_password_hooks<\/code> file is present in the project root directory, it will be\nused as the password.<\/p>\n\n<p>If this file doesn\u2019t exist, you\u2019ll be promted for an encryption password and\nthis password will be saved in <code class=\"highlighter-rouge\">.vault_password_hooks<\/code>, in your project\u2019s root.<\/p>\n\n<p>If <code class=\"highlighter-rouge\">.vault_password_hooks<\/code> is listed in <code class=\"highlighter-rouge\">.gitignore<\/code>, this file will persist and you\nwon\u2019t be asked for a password anymore for encryption as well as for decryption.\nOtherwise, <code class=\"highlighter-rouge\">.vault_password_hooks<\/code> will be erased after encryption to avoid commiting\nthe file.<\/p>\n\n<p>After commiting the files, the post-commit hook will use the same password to\ndecrypt the previously encrypted files.<\/p>\n\n<p><strong>TL;DR:<\/strong> add <code class=\"highlighter-rouge\">.vault_password_hooks<\/code> to your <code class=\"highlighter-rouge\">.gitignore<\/code>, add <code class=\"highlighter-rouge\"># -*-\nvault: true; -*-<\/code> to files that requires encryption and you\u2019re set.<\/p>\n\n<p>You end up with a workflow where your files are transparently encrypted before\ncommit and decrypted after.<\/p>\n\n<h2 id=\"hooks\">Hooks<\/h2>\n\n<p>Put the hooks in <code class=\"highlighter-rouge\">.git\/hooks\/<\/code> and don\u2019t forget to <code class=\"highlighter-rouge\">chmod +x\n{pre,post}-commit<\/code> them.<\/p>\n\n<script src=\"https:\/\/gist.github.com\/leucos\/405b406b9d6bde0c3d39.js\"> <\/script>\n\n\n    <p><a href=\"https:\/\/leucos.github.io\/articles\/ansible-transparent-vault\/\">Transparent encryption\/decryption with ansible vault<\/a> was originally published by Michel Blanc at <a href=\"https:\/\/leucos.github.io\">Random stuff<\/a> on May 25, 2015.<\/p>\n  "},{"title":{"@attributes":{"type":"html"}},"link":{"@attributes":{"rel":"alternate","type":"text\/html","href":"https:\/\/leucos.github.io\/articles\/ansible-contract-inventory\/"}},"id":"https:\/\/leucos.github.io\/articles\/ansible-contract-inventory","updated":"2015-05-11T00:00:00-00:00","published":"2015-05-03T00:00:00+00:00","author":{"name":"Michel Blanc","uri":"https:\/\/leucos.github.io","email":"mb@mbnet.fr"},"content":"\n    <h2 id=\"the-problem\">The problem<\/h2>\n\n<p>You\u2019ve been there too. Spinning up droplets on DigitalOcean with Ansible\nand using a dynamic inventory script is quite a pain.<\/p>\n\n<p>Most approaches use the <code class=\"highlighter-rouge\">digital_ocean<\/code> ansible module in\nplaybooks to spin up droplets, along with the <code class=\"highlighter-rouge\">digital_ocean.py<\/code> dynamic\ninventory script, using this kind of workflow:<\/p>\n\n<ul>\n  <li>define your droplets in a YAML file (eventually with size, region,\netc\u2026)<\/li>\n  <li>create a playbook that will loop over droplet list (<code class=\"highlighter-rouge\">with_items<\/code> or\nequivalent) and spin up the droplet<\/li>\n  <li>dynamically add started droplets to inventory<\/li>\n<\/ul>\n\n<p>This approach has many drawbacks, and, to be honest, is not really usable.<\/p>\n\n<h3 id=\"slooooooow\">Slooooooow<\/h3>\n\n<p>First, it is damn slow. Droplet creation is serialized. Since\n<code class=\"highlighter-rouge\">digital_ocean<\/code> waits for the droplet to come up, and since DO itself\nadvertizes \u2018Start your droplet in 55 seconds !\u2019, you can do the math.\nStarting a single droplet is quite long, so spinning up your multi-tier,\nfault-tolerant, distributed architecture will take ages.<\/p>\n\n<p>You probably can use <code class=\"highlighter-rouge\">async<\/code> + <code class=\"highlighter-rouge\">poll<\/code> to spin up the droplets. I didn\u2019t\ntry and don\u2019t know where this would lead. But you\u2019d still face the other\nissues.<\/p>\n\n<h3 id=\"naming\">Naming<\/h3>\n\n<p>You droplets won\u2019t have real names. They will be known by their IPs.\nSure, if you use the <code class=\"highlighter-rouge\">name<\/code> parameter during creation, you might be able\nto use it, but at best, this will be a group name.<\/p>\n\n<p>You could also use <code class=\"highlighter-rouge\">add_host<\/code> in your bootstrapping script, but this is\na run time hack, so forget about setting variables in <code class=\"highlighter-rouge\">host_vars<\/code>.<\/p>\n\n<p>Since droplets are mostly nameless, grouping them is hard. Sure, you can\ndo it at run time with <code class=\"highlighter-rouge\">add_host<\/code> too, but you won\u2019t leverage\n<code class=\"highlighter-rouge\">group_vars<\/code> usage.<\/p>\n\n<p>Anyway, all those run-time naming hacks will force you to loop over all\nyour droplets definitions, hit DO API to make sure they\u2019re alive, then\nloop over API responses to add hosts and groups EVERY time you execute a\nplaybook.<\/p>\n\n<h3 id=\"localhost-is-forced-in\">localhost is forced in<\/h3>\n\n<p>Spinning up instances on DO will require to run the <code class=\"highlighter-rouge\">digital_ocean<\/code>\nmodule as a <code class=\"highlighter-rouge\">local_action<\/code> or using <code class=\"highlighter-rouge\">delegate_to: localhost<\/code>. This means\nthat you are bound to declare localhost in your inventory. This is a\nreal pain, since it makes the <code class=\"highlighter-rouge\">all<\/code> group mostly unusable, unless you\nchange all your playbook hosts definitions from <code class=\"highlighter-rouge\">hosts: all<\/code> to <code class=\"highlighter-rouge\">hosts:\nall:!localhost<\/code>. Pretty bad for readability.<\/p>\n\n<p>Let\u2019s stop here, there are already enough reasons to find an alternate\nway. There are probably other cons, and certainly pros too for the\ndynamic approach, but I fell that this way of doing it is barely usable\nfor serious, repeatable stuff.<\/p>\n\n<h2 id=\"alternate-aproach\">Alternate aproach<\/h2>\n\n<p>In the end, we would like to work as we do with on-prem hardware: have a\nstatic inventory.<\/p>\n\n<p>The idea is to create this static inventory first, and then use a\nbootstrapping script that will use this inventory as a contract to apply\non DigitalOcean.<\/p>\n\n<p>The script will list all hosts in your inventory (using <code class=\"highlighter-rouge\">ansible\n--list-hosts<\/code>), and parallelize droplet creation on digital ocean.<\/p>\n\n<p>When all droplets are created, it will create a complementary inventory\nfile in your inventory directory containing hosts with their respective\nIPs.<\/p>\n\n<p>At this point, you have a perfectly static inventory, and can run your\nansible playbook normally, without hitting external APIs (serialized),\nwithout naming problems, \u2026 Things are just <em>normal<\/em>, fast and\nreliable, without edge cases introduced by dynamic inventories.<\/p>\n\n<p>Using this approach on a 8 droplets setup, the time to set-up instances\nwent from 9\u201933\u201d down to 1\u201956\u201d. And the time to destroy instances went\nfrom 0\u201955\u201d down to 0\u20193\u201d (see demo below). Of course, more droplets, \nmore gain.<\/p>\n\n<p>And these are just create\/destroy gains. You also benefit from static\ninventory for all your lifecycle playbook runs, since you never hit DO API and\ndon\u2019t have to build inventory at run time, which is always slower despite the\ninventory cache.<\/p>\n\n<h3 id=\"example\">Example<\/h3>\n\n<p>Assuming you have an inventory directory in <code class=\"highlighter-rouge\">inventories\/devel\/<\/code>,\ncontaining a <code class=\"highlighter-rouge\">hosts<\/code> file, you can spin up your droplets like this:<\/p>\n\n<figure class=\"highlight\"><pre><code class=\"language-bash\" data-lang=\"bash\">do_boot.sh inventories\/devel\/<\/code><\/pre><\/figure>\n\n<p>When you\u2019re finished with your infrastructure, call the same command with\nthe <code class=\"highlighter-rouge\">deleted<\/code> parameter:<\/p>\n\n<figure class=\"highlight\"><pre><code class=\"language-bash\" data-lang=\"bash\">do_boot.sh inventories\/devel\/ deleted<\/code><\/pre><\/figure>\n\n<p>That\u2019s all.<\/p>\n\n<p>The script has defaults regarding droplet size, region, image and ssh\nkey. You can change the defaults in the script to something that suits you, and\noverride these defaults per droplet in your inventory:<\/p>\n\n<figure class=\"highlight\"><pre><code class=\"language-yaml\" data-lang=\"yaml\"><span class=\"pi\">[<\/span><span class=\"nv\">www<\/span><span class=\"pi\">]<\/span>\n<span class=\"s\">www1<\/span>\n<span class=\"s\">www2<\/span>\n<span class=\"s\">www3 do_region=2<\/span>\n\n<span class=\"pi\">[<\/span><span class=\"nv\">database<\/span><span class=\"pi\">]<\/span>\n<span class=\"s\">db1 do_size=62 do_image=12345<\/span>\n\n<span class=\"pi\">[<\/span><span class=\"nv\">redis<\/span><span class=\"pi\">]<\/span>\n<span class=\"s\">redis1<\/span>\n\n<span class=\"pi\">[<\/span><span class=\"nv\">elastic<\/span><span class=\"pi\">]<\/span>\n<span class=\"s\">elastic1 do_size=60<\/span><\/code><\/pre><\/figure>\n\n<p>###Spinning up and down 8 droplets in 2\u201915\u201d<\/p>\n\n<script type=\"text\/javascript\" src=\"https:\/\/asciinema.org\/a\/19479.js\" id=\"asciicast-19479\" async=\"\"><\/script>\n\n<h2 id=\"script\">Script<\/h2>\n\n<p>You can grab the script in this <a href=\"https:\/\/gist.github.com\/leucos\/6f8d93de3493431acd29\">gist<\/a>.<\/p>\n\n<p><strong>UPDATE:<\/strong> if you run Ansible v2.0+, use <a href=\"https:\/\/gist.github.com\/leucos\/2c361f7d4767f8aea6dd\">this script\ninstead<\/a>. It will\nuse the new digital Ocean API (v2.0 too). You just need to set\n<code class=\"highlighter-rouge\">DO_API_TOKEN<\/code>.<\/p>\n\n<figure class=\"highlight\"><pre><code class=\"language-bash\" data-lang=\"bash\"><span class=\"c\">#!\/bin\/bash<\/span>\n<span class=\"c\">#<\/span>\n<span class=\"c\"># Change defaults below<\/span>\n<span class=\"c\"># ---------------------<\/span>\n<span class=\"c\">#<\/span>\n<span class=\"c\"># Digital Ocean default values<\/span>\n<span class=\"c\"># You can override them using do_something in your inventory file<\/span>\n<span class=\"c\"># Example:<\/span>\n<span class=\"c\"># <\/span>\n<span class=\"c\"># [www]<\/span>\n<span class=\"c\"># www1 do_size=62 do_image=12345<\/span>\n<span class=\"c\"># ...<\/span>\n<span class=\"c\">#<\/span>\n<span class=\"c\"># If you don't override in your inventory, the defaults below will apply<\/span>\n<span class=\"nv\">DEFAULT_SIZE<\/span><span class=\"o\">=<\/span>66       <span class=\"c\"># 512mb (override with do_size)<\/span>\n<span class=\"nv\">DEFAULT_REGION<\/span><span class=\"o\">=<\/span>5      <span class=\"c\"># ams2 (override with do_region)<\/span>\n<span class=\"nv\">DEFAULT_IMAGE<\/span><span class=\"o\">=<\/span>9801950 <span class=\"c\"># Ubuntu 14.04 x64 (override with do_image)<\/span>\n<span class=\"nv\">DEFAULT_KEY<\/span><span class=\"o\">=<\/span>785648    <span class=\"c\"># SSH key, change this ! (override with do_key)<\/span>\n\n<span class=\"c\"># localhost entry for temporary inventory<\/span>\n<span class=\"c\"># This is a temp inventory generated to start the DO droplets<\/span>\n<span class=\"c\"># You might want to change ansible_python_interpreter<\/span>\n<span class=\"nv\">LOCALHOST_ENTRY<\/span><span class=\"o\">=<\/span><span class=\"s2\">\"localhost ansible_python_interpreter=\/usr\/bin\/python2\"<\/span> \n\n<span class=\"c\"># Set state to present by default<\/span>\n<span class=\"nv\">STATE<\/span><span class=\"o\">=<\/span><span class=\"k\">${<\/span><span class=\"nv\">2<\/span><span class=\"k\">:-<\/span><span class=\"s2\">\"present\"<\/span><span class=\"k\">}<\/span>\n\n<span class=\"c\"># digital_ocean module command to use<\/span>\n<span class=\"c\"># name, size, region, image and key will be filled automatically<\/span>\n<span class=\"nv\">COMMAND<\/span><span class=\"o\">=<\/span><span class=\"s2\">\"state=<\/span><span class=\"nv\">$STATE<\/span><span class=\"s2\"> command=droplet private_networking=yes unique_name=yes\"<\/span>\n<span class=\"c\"># ---------------------<\/span>\n\n<span class=\"k\">function <\/span>bail_out <span class=\"o\">{<\/span>\n  <span class=\"nb\">echo<\/span> <span class=\"nv\">$1<\/span>\n  <span class=\"nb\">echo<\/span> <span class=\"nt\">-e<\/span> <span class=\"s2\">\"Usage: <\/span><span class=\"nv\">$0<\/span><span class=\"s2\"> &lt;inventory_directory&gt; [present|deleted]<\/span><span class=\"se\">\\n<\/span><span class=\"s2\">\"<\/span>\n  <span class=\"nb\">echo<\/span> <span class=\"nt\">-e<\/span> <span class=\"s2\">\"<\/span><span class=\"se\">\\t<\/span><span class=\"s2\">inventory_directory: the directory containing the inventory goal (compulsory)\"<\/span>\n  <span class=\"nb\">echo<\/span> <span class=\"nt\">-e<\/span> <span class=\"s2\">\"<\/span><span class=\"se\">\\t<\/span><span class=\"s2\">present: the droplet will be created if it doesn't exist (default)\"<\/span>\n  <span class=\"nb\">echo<\/span> <span class=\"nt\">-e<\/span> <span class=\"s2\">\"<\/span><span class=\"se\">\\t<\/span><span class=\"s2\">deleted: the droplet will be destroyed if it exists\"<\/span>\n  <span class=\"nb\">exit <\/span>1\n<span class=\"o\">}<\/span>\n\n<span class=\"c\"># Check that inventory is a directory<\/span>\n<span class=\"c\"># We need this since we generate a complementary inventory with IP addresses for hosts<\/span>\n<span class=\"nv\">INVENTORY<\/span><span class=\"o\">=<\/span><span class=\"nv\">$1<\/span>\n<span class=\"o\">[[<\/span> <span class=\"o\">!<\/span> <span class=\"nt\">-d<\/span> <span class=\"s2\">\"<\/span><span class=\"nv\">$INVENTORY<\/span><span class=\"s2\">\"<\/span> <span class=\"o\">]]<\/span>  <span class=\"o\">&amp;&amp;<\/span> bail_out <span class=\"s2\">\"Inventory does not exist, is not a\ndirectory, or is not set\"<\/span>\n<span class=\"o\">[[<\/span> <span class=\"o\">!<\/span> <span class=\"nt\">-e<\/span> <span class=\"nv\">$DO_CLIENT_ID<\/span> <span class=\"o\">]]<\/span> <span class=\"o\">||<\/span> bail_out <span class=\"s2\">\"DO_CLIENT_ID not set\"<\/span>\n<span class=\"o\">[[<\/span> <span class=\"o\">!<\/span> <span class=\"nt\">-e<\/span> <span class=\"nv\">$DO_API_KEY<\/span> <span class=\"o\">]]<\/span>   <span class=\"o\">||<\/span> bail_out <span class=\"s2\">\"DO_API_KEY not set\"<\/span>\n\n<span class=\"c\"># Get a list of hosts from inventory dir<\/span>\n<span class=\"nv\">HOSTS<\/span><span class=\"o\">=<\/span><span class=\"k\">$(<\/span>ansible <span class=\"nt\">-i<\/span> <span class=\"nv\">$1<\/span> <span class=\"nt\">--list-hosts<\/span> all | awk <span class=\"s1\">'{ print $1 }'<\/span> | tr <span class=\"s1\">'\\n'<\/span> <span class=\"s1\">' '<\/span><span class=\"k\">)<\/span>\n\n<span class=\"c\"># Clean up previously generated inventory<\/span>\nrm <span class=\"nv\">$INVENTORY<\/span>\/generated\n\n<span class=\"c\"># Creating temporary inventory with only localhost in it<\/span>\n<span class=\"nv\">TEMP_INVENTORY<\/span><span class=\"o\">=<\/span><span class=\"k\">$(<\/span>mktemp<span class=\"k\">)<\/span>\n<span class=\"nb\">echo <\/span>Creating temporary inventory <span class=\"k\">in<\/span> <span class=\"nv\">$TEMP_INVENTORY<\/span>\n<span class=\"nb\">echo<\/span> <span class=\"nv\">$LOCALHOST<\/span> <span class=\"o\">&gt;<\/span> <span class=\"nv\">$TEMP_INVENTORY<\/span>\n\n<span class=\"c\"># Create droplets in \/\/<\/span>\n<span class=\"k\">for <\/span>i <span class=\"k\">in<\/span> <span class=\"nv\">$HOSTS<\/span><span class=\"p\">;<\/span> <span class=\"k\">do \n  <\/span><span class=\"nv\">SIZE<\/span><span class=\"o\">=<\/span><span class=\"k\">$(<\/span><span class=\"nb\">grep<\/span> <span class=\"nv\">$i<\/span> <span class=\"nv\">$1<\/span>\/hosts | <span class=\"nb\">grep <\/span>do_size | sed <span class=\"nt\">-e<\/span> <span class=\"s1\">'s\/.*do_size=\\(\\d*\\)\/\\1\/'<\/span><span class=\"k\">)<\/span>\n  <span class=\"nv\">REGION<\/span><span class=\"o\">=<\/span><span class=\"k\">$(<\/span><span class=\"nb\">grep<\/span> <span class=\"nv\">$i<\/span> <span class=\"nv\">$1<\/span>\/hosts | <span class=\"nb\">grep <\/span>do_region | sed <span class=\"nt\">-e<\/span> <span class=\"s1\">'s\/.*do_region=\\(\\d*\\)\/\\1\/'<\/span><span class=\"k\">)<\/span>\n  <span class=\"nv\">IMAGE<\/span><span class=\"o\">=<\/span><span class=\"k\">$(<\/span><span class=\"nb\">grep<\/span> <span class=\"nv\">$i<\/span> <span class=\"nv\">$1<\/span>\/hosts | <span class=\"nb\">grep <\/span>do_image | sed <span class=\"nt\">-e<\/span> <span class=\"s1\">'s\/.*do_image=\\(\\d*\\)\/\\1\/'<\/span><span class=\"k\">)<\/span>\n  <span class=\"nv\">KEY<\/span><span class=\"o\">=<\/span><span class=\"k\">$(<\/span><span class=\"nb\">grep<\/span> <span class=\"nv\">$i<\/span> <span class=\"nv\">$1<\/span>\/hosts | <span class=\"nb\">grep <\/span>do_key | sed <span class=\"nt\">-e<\/span> <span class=\"s1\">'s\/.*do_key=\\(\\d*\\)\/\\1\/'<\/span><span class=\"k\">)<\/span>\n\n  <span class=\"nv\">SIZE<\/span><span class=\"o\">=<\/span><span class=\"k\">${<\/span><span class=\"nv\">SIZE<\/span><span class=\"k\">:-<\/span><span class=\"nv\">$DEFAULT_SIZE<\/span><span class=\"k\">}<\/span>\n  <span class=\"nv\">REGION<\/span><span class=\"o\">=<\/span><span class=\"k\">${<\/span><span class=\"nv\">REGION<\/span><span class=\"k\">:-<\/span><span class=\"nv\">$DEFAULT_REGION<\/span><span class=\"k\">}<\/span>\n  <span class=\"nv\">IMAGE<\/span><span class=\"o\">=<\/span><span class=\"k\">${<\/span><span class=\"nv\">IMAGE<\/span><span class=\"k\">:-<\/span><span class=\"nv\">$DEFAULT_IMAGE<\/span><span class=\"k\">}<\/span>\n  <span class=\"nv\">KEY<\/span><span class=\"o\">=<\/span><span class=\"k\">${<\/span><span class=\"nv\">KEY<\/span><span class=\"k\">:-<\/span><span class=\"nv\">$DEFAULT_KEY<\/span><span class=\"k\">}<\/span>\n\n  <span class=\"k\">if<\/span> <span class=\"o\">[<\/span> <span class=\"s2\">\"<\/span><span class=\"k\">${<\/span><span class=\"nv\">STATE<\/span><span class=\"k\">}<\/span><span class=\"s2\">\"<\/span> <span class=\"o\">==<\/span> <span class=\"s2\">\"present\"<\/span> <span class=\"o\">]<\/span><span class=\"p\">;<\/span> <span class=\"k\">then\n    <\/span><span class=\"nb\">echo<\/span> <span class=\"s2\">\"Creating <\/span><span class=\"nv\">$i<\/span><span class=\"s2\"> of size <\/span><span class=\"nv\">$SIZE<\/span><span class=\"s2\"> using image <\/span><span class=\"nv\">$IMAGE<\/span><span class=\"s2\"> in region <\/span><span class=\"nv\">$REGION<\/span><span class=\"s2\"> with key <\/span><span class=\"nv\">$KEY<\/span><span class=\"s2\">\"<\/span>\n  <span class=\"k\">else\n    <\/span><span class=\"nb\">echo<\/span> <span class=\"s2\">\"Deleting <\/span><span class=\"nv\">$i<\/span><span class=\"s2\">\"<\/span>\n  <span class=\"k\">fi<\/span>\n  <span class=\"c\"># echo \" =&gt; $COMMAND name=$i size_id=$SIZE image_id=$IMAGE region_id=$REGION ssh_key_ids=$KEY\"<\/span>\n  ansible localhost <span class=\"nt\">-c<\/span> <span class=\"nb\">local<\/span> <span class=\"nt\">-i<\/span> <span class=\"nv\">$TEMP_INVENTORY<\/span> <span class=\"nt\">-m<\/span> digital_ocean <span class=\"se\">\\<\/span>\n    <span class=\"nt\">-a<\/span> <span class=\"s2\">\"<\/span><span class=\"nv\">$COMMAND<\/span><span class=\"s2\"> name=<\/span><span class=\"nv\">$i<\/span><span class=\"s2\"> size_id=<\/span><span class=\"nv\">$SIZE<\/span><span class=\"s2\"> image_id=<\/span><span class=\"nv\">$IMAGE<\/span><span class=\"s2\"> region_id=<\/span><span class=\"nv\">$REGION<\/span><span class=\"s2\"> ssh_key_ids=<\/span><span class=\"nv\">$KEY<\/span><span class=\"s2\">\"<\/span> &amp;\n<span class=\"k\">done\n\n<\/span><span class=\"nb\">wait<\/span>\n\n<span class=\"c\"># Now do it again to fill up complementary inventory<\/span>\n<span class=\"k\">if<\/span> <span class=\"o\">[<\/span> <span class=\"s2\">\"<\/span><span class=\"k\">${<\/span><span class=\"nv\">STATE<\/span><span class=\"k\">}<\/span><span class=\"s2\">\"<\/span> <span class=\"o\">==<\/span> <span class=\"s2\">\"present\"<\/span> <span class=\"o\">]<\/span><span class=\"p\">;<\/span> <span class=\"k\">then\n  for <\/span>i <span class=\"k\">in<\/span> <span class=\"nv\">$HOSTS<\/span><span class=\"p\">;<\/span> <span class=\"k\">do \n    <\/span><span class=\"nb\">echo <\/span>Checking droplet <span class=\"nv\">$i<\/span>\n    <span class=\"nv\">IP<\/span><span class=\"o\">=<\/span><span class=\"k\">$(<\/span>ansible localhost <span class=\"nt\">-c<\/span> <span class=\"nb\">local<\/span> <span class=\"nt\">-i<\/span> <span class=\"nv\">$TEMP_INVENTORY<\/span> <span class=\"nt\">-m<\/span> digital_ocean <span class=\"nt\">-a<\/span> <span class=\"s2\">\"state=present command=droplet unique_name=yes name=<\/span><span class=\"nv\">$i<\/span><span class=\"s2\">\"<\/span> | <span class=\"nb\">grep<\/span> <span class=\"s2\">\"<\/span><span class=\"se\">\\\"<\/span><span class=\"s2\">ip_address\"<\/span> | awk <span class=\"s1\">'{ print $2 }'<\/span> | cut <span class=\"nt\">-f2<\/span> <span class=\"nt\">-d<\/span><span class=\"s1\">'\"'<\/span><span class=\"k\">)<\/span>\n    <span class=\"nb\">echo<\/span> <span class=\"s2\">\"<\/span><span class=\"nv\">$i<\/span><span class=\"s2\"> ansible_ssh_host=<\/span><span class=\"nv\">$IP<\/span><span class=\"s2\">\"<\/span> <span class=\"o\">&gt;&gt;<\/span> <span class=\"nv\">$INVENTORY<\/span>\/generated\n  <span class=\"k\">done\nfi\n\n<\/span><span class=\"nb\">echo<\/span> <span class=\"s2\">\"All done !\"<\/span><\/code><\/pre><\/figure>\n\n\n    <p><a href=\"https:\/\/leucos.github.io\/articles\/ansible-contract-inventory\/\">Making dynamic inventory usable with Ansible and Digital Ocean<\/a> was originally published by Michel Blanc at <a href=\"https:\/\/leucos.github.io\">Random stuff<\/a> on May 03, 2015.<\/p>\n  "},{"title":{"@attributes":{"type":"html"}},"link":{"@attributes":{"rel":"alternate","type":"text\/html","href":"https:\/\/leucos.github.io\/articles\/testing-ansible-roles-part-2\/"}},"id":"https:\/\/leucos.github.io\/articles\/testing-ansible-roles-part-2","published":"2015-03-15T00:00:00+00:00","updated":"2015-03-15T00:00:00+00:00","author":{"name":"Michel Blanc","uri":"https:\/\/leucos.github.io","email":"mb@mbnet.fr"},"content":"\n    <p>Now that we have created our basic role in <a href=\"\/articles\/testing-ansible-roles-part-1\/\">part 1<\/a>, we need to set-up a Vagrant machine and some tooling to run our tests.<\/p>\n\n<h2 id=\"creating-the-vagrant-machine\">Creating the Vagrant machine<\/h2>\n\n<h3 id=\"vagrantfile\">Vagrantfile<\/h3>\n\n<p>To spin up a Vagrant machine, we need to create a <code class=\"highlighter-rouge\">Vagrantfile<\/code>. We\u2019ll create it in our role top directory:<\/p>\n\n<figure class=\"highlight\"><pre><code class=\"language-ruby\" data-lang=\"ruby\"><span class=\"no\">Vagrant<\/span><span class=\"p\">.<\/span><span class=\"nf\">configure<\/span><span class=\"p\">(<\/span><span class=\"mi\">2<\/span><span class=\"p\">)<\/span> <span class=\"k\">do<\/span> <span class=\"o\">|<\/span><span class=\"n\">config<\/span><span class=\"o\">|<\/span>\n  <span class=\"n\">config<\/span><span class=\"p\">.<\/span><span class=\"nf\">vm<\/span><span class=\"p\">.<\/span><span class=\"nf\">box<\/span> <span class=\"o\">=<\/span> <span class=\"s2\">\"ubuntu\/trusty64\"<\/span>\n  <span class=\"n\">config<\/span><span class=\"p\">.<\/span><span class=\"nf\">vm<\/span><span class=\"p\">.<\/span><span class=\"nf\">define<\/span> <span class=\"s2\">\"nginx\"<\/span> <span class=\"k\">do<\/span> <span class=\"o\">|<\/span><span class=\"n\">nginx<\/span><span class=\"o\">|<\/span>\n  <span class=\"k\">end<\/span>\n  <span class=\"n\">config<\/span><span class=\"p\">.<\/span><span class=\"nf\">vm<\/span><span class=\"p\">.<\/span><span class=\"nf\">provision<\/span> <span class=\"s2\">\"shell\"<\/span><span class=\"p\">,<\/span>\n    <span class=\"ss\">:path<\/span> <span class=\"o\">=&gt;<\/span> <span class=\"s2\">\"vagrant_specs.sh\"<\/span><span class=\"p\">,<\/span>\n    <span class=\"ss\">:upload_path<\/span> <span class=\"o\">=&gt;<\/span> <span class=\"s2\">\"\/home\/vagrant\/specs\"<\/span><span class=\"p\">,<\/span>\n    <span class=\"c1\"># change role name below<\/span>\n    <span class=\"ss\">:args<\/span> <span class=\"o\">=&gt;<\/span> <span class=\"s2\">\"--install ansible-nginx\"<\/span>\n<span class=\"k\">end<\/span><\/code><\/pre><\/figure>\n\n<p>You can change <code class=\"highlighter-rouge\">config.vm.box<\/code> to another Vagrant box that better suits your\nneeds, but keep in mind RoleSpec is very Debian\/Ubuntu inclined. We\u2019ll provision\nthis machine with a shell script (not with Ansible, so we don\u2019t end up in an\ninception style situation).<\/p>\n\n<h3 id=\"provisionning-script\">Provisionning script<\/h3>\n\n<p>The provisionning script, <code class=\"highlighter-rouge\">vagrant_specs.sh<\/code> serves two purposes:<\/p>\n\n<ol>\n  <li>\n    <p>it takes care of installing RoleSpec and setting up the test directory when\ncalled with <code class=\"highlighter-rouge\">--install<\/code>. This happens only at vagrant provisionning time (e.g.\n<code class=\"highlighter-rouge\">vagrant up<\/code> of <code class=\"highlighter-rouge\">vagrant provision<\/code>)<\/p>\n  <\/li>\n  <li>\n    <p>it can be called to run the test suite; to make invocation easier, it will copy itself to <code class=\"highlighter-rouge\">\/usr\/local\/bin\/specs<\/code><\/p>\n  <\/li>\n<\/ol>\n\n<p>Create the <code class=\"highlighter-rouge\">vagrant_specs.sh<\/code> with the following content:<\/p>\n\n<figure class=\"highlight\"><pre><code class=\"language-bash\" data-lang=\"bash\"><span class=\"c\">#!\/bin\/bash<\/span>\n<span class=\"c\">#<\/span>\n<span class=\"c\"># Vagrant provisionning script<\/span>\n<span class=\"c\">#<\/span>\n<span class=\"c\"># Usage for provisionning VM &amp; running (in Vagrant file):<\/span>\n<span class=\"c\"># <\/span>\n<span class=\"c\"># script.sh --install &lt;role&gt;<\/span>\n<span class=\"c\">#<\/span>\n<span class=\"c\"># e.g. : <\/span>\n<span class=\"c\"># script.sh --install ansible-nginx<\/span>\n<span class=\"c\"># <\/span>\n<span class=\"c\"># Usage for running only (from host):<\/span>\n<span class=\"c\">#<\/span>\n<span class=\"c\"># vagrant ssh -c specs<\/span>\n<span class=\"c\">#<\/span>\n<span class=\"k\">if<\/span> <span class=\"o\">[<\/span> <span class=\"s2\">\"x<\/span><span class=\"nv\">$1<\/span><span class=\"s2\">\"<\/span> <span class=\"o\">==<\/span> <span class=\"s2\">\"x--install\"<\/span> <span class=\"o\">]<\/span><span class=\"p\">;<\/span> <span class=\"k\">then\n  <\/span>mv ~vagrant\/specs \/usr\/local\/bin\/specs\n  chmod 755 \/usr\/local\/bin\/specs\n  <span class=\"nb\">sudo <\/span>apt-get install <span class=\"nt\">-qqy<\/span> git\n  su vagrant <span class=\"nt\">-c<\/span> <span class=\"s1\">'git clone --depth 1 https:\/\/github.com\/nickjj\/rolespec'<\/span>\n  <span class=\"nb\">cd<\/span> ~vagrant\/rolespec <span class=\"o\">&amp;&amp;<\/span> make install\n  su vagrant <span class=\"nt\">-c<\/span> <span class=\"s1\">'rolespec -i ~\/testdir'<\/span>\n  su vagrant <span class=\"nt\">-c<\/span> <span class=\"s2\">\"ln -s \/vagrant\/ ~\/testdir\/roles\/<\/span><span class=\"nv\">$2<\/span><span class=\"s2\">\"<\/span>\n  su vagrant <span class=\"nt\">-c<\/span> <span class=\"s2\">\"ln -s \/vagrant\/tests\/<\/span><span class=\"nv\">$2<\/span><span class=\"s2\">\/ ~\/testdir\/tests\/\"<\/span>\n  <span class=\"nb\">exit\n<\/span><span class=\"k\">fi\n\n<\/span><span class=\"nb\">cd<\/span> ~vagrant\/testdir <span class=\"o\">&amp;&amp;<\/span> rolespec <span class=\"nt\">-r<\/span> <span class=\"k\">$(<\/span><span class=\"nb\">ls <\/span>roles<span class=\"k\">)<\/span> <span class=\"s2\">\"<\/span><span class=\"nv\">$*<\/span><span class=\"s2\">\"<\/span><\/code><\/pre><\/figure>\n\n<p>and make it executable (<code class=\"highlighter-rouge\">chmod +x vagrant_specs.sh<\/code>).<\/p>\n\n<h3 id=\"running-the-vagrat-box\">Running the Vagrat box<\/h3>\n\n<p>Now, let\u2019s check this ! It might take a while if you don\u2019t already have the\nvagrant image on your box:<\/p>\n\n<div class=\"highlighter-rouge\"><div class=\"highlight\"><pre class=\"highlight\"><code>$ vagrant up\nBringing machine 'nginx' up with 'virtualbox' provider...\n==&gt; nginx: Importing base box 'ubuntu\/trusty64'...\n==&gt; nginx: Matching MAC address for NAT networking...\n==&gt; nginx: Checking if box 'ubuntu\/trusty64' is up to date...\n==&gt; nginx: Setting the name of the VM: ansible-nginx_nginx_1426331325901_88232\n...\n==&gt; nginx: Cloning into 'rolespec'...\n==&gt; nginx: Installing RoleSpec scripts in \/usr\/local\/bin ...\n==&gt; nginx: Installing RoleSpec libs in \/usr\/local\/lib\/rolespec ...\n==&gt; nginx: Initialized new RoleSpec directory in \/home\/vagrant\/testdir\n$\n<\/code><\/pre><\/div><\/div>\n\n<h2 id=\"creating-tests\">Creating tests<\/h2>\n\n<p>We\u2019re almost done. Only two files left to create. First, we RoleSpec needs an inventory. Nothing fancy here, we just need to create an inventory file with a single host, <code class=\"highlighter-rouge\">placeholder_fqdn<\/code>, RoleSpec will take care of the rest:<\/p>\n\n<figure class=\"highlight\"><pre><code class=\"language-bash\" data-lang=\"bash\"><span class=\"nv\">$ <\/span><span class=\"nb\">echo<\/span> <span class=\"s2\">\"placeholder_fqdn\"<\/span> <span class=\"o\">&gt;<\/span> tests\/ansible-nginx\/inventory\/hosts<\/code><\/pre><\/figure>\n\n<h3 id=\"writing-the-test-file\">Writing the test file<\/h3>\n\n<p>And finally, we need a test file, where we can check if our playbook works. We can check the syntax, the idempotency, the resulting templates, etc\u2026<\/p>\n\n<p>This test file is simply a bash script, in which we include some RoleSpec files to get access to its DSL.<\/p>\n\n<p>Let\u2019s start with a simple one, and create <code class=\"highlighter-rouge\">tests\/ansible-nginx\/test<\/code> with the following content:<\/p>\n\n<figure class=\"highlight\"><pre><code class=\"language-bash\" data-lang=\"bash\"><span class=\"c\">#!\/bin\/bash<\/span>\n<span class=\"c\"># -*- bash -*-<\/span>\n\n<span class=\"c\"># This gives you access to the custom DSL<\/span>\n<span class=\"nb\">.<\/span> <span class=\"s2\">\"<\/span><span class=\"k\">${<\/span><span class=\"nv\">ROLESPEC_LIB<\/span><span class=\"k\">}<\/span><span class=\"s2\">\/main\"<\/span>\n\n<span class=\"c\"># Install a specific version of Ansible<\/span>\ninstall_ansible <span class=\"s2\">\"v1.8.3\"<\/span>\n\n<span class=\"c\"># Check syntax first, and then that the playbook runs<\/span>\nassert_playbook_runs\n\n<span class=\"c\"># Check that the playbook is idempotent<\/span>\nassert_playbook_idempotent<\/code><\/pre><\/figure>\n\n<p>Don\u2019t forget to make the test file executable (<code class=\"highlighter-rouge\">chmod +x tests\/ansible-nginx\/test<\/code>).<\/p>\n\n<h3 id=\"runing-tests\">Runing tests<\/h3>\n\n<p>Our simple tests are setup. To run them, we need to execute\n<code class=\"highlighter-rouge\">\/usr\/local\/bin\/specs<\/code> in the Vagrant host.<\/p>\n\n<figure class=\"highlight\"><pre><code class=\"language-bash\" data-lang=\"bash\">vagrant ssh <span class=\"nt\">-c<\/span> <span class=\"s1\">'specs'<\/span><\/code><\/pre><\/figure>\n\n<p>RoleSpecs will then download Ansible (version 1.8.3 since this is what we\nasked for), install it, and run our test case.<\/p>\n\n<script type=\"text\/javascript\" src=\"https:\/\/asciinema.org\/a\/17711.js\" id=\"asciicast-17711\" async=\"\"><\/script>\n\n<p>As you can see in the recording, RoleSpec:<\/p>\n\n<ul>\n  <li>installs Ansible (<code class=\"highlighter-rouge\">ROLESPEC: [Install Ansible - v1.8.3]<\/code>)<\/li>\n  <li>executes the playbook with <code class=\"highlighter-rouge\">assert_playbook_runs<\/code> (<code class=\"highlighter-rouge\">TEST: [Run playbook syntax check]<\/code> and <code class=\"highlighter-rouge\">TEST: [Run playbook]<\/code>)<\/li>\n  <li>check that the playbook is idempotent with <code class=\"highlighter-rouge\">assert_playbook_idempotent<\/code> (<code class=\"highlighter-rouge\">TEST: [Re-run playbook]<\/code>)<\/li>\n<\/ul>\n\n<p>Pretty neat !<\/p>\n\n<h3 id=\"runing-tests-faster\">Runing tests faster<\/h3>\n\n<p>There is one downside though: it takes almost 3 minutes to run. However, you can\nspeed up subsequent runs as long as you don\u2019t have to change the Ansible\nversion: since Ansible is already installed, there is no need to install it\nagain every time. Using the <code class=\"highlighter-rouge\">-p<\/code> option will run in <em>playbook mode<\/em>, which means\nit will only run <code class=\"highlighter-rouge\">assert_playbook_runs<\/code> test.<\/p>\n\n<figure class=\"highlight\"><pre><code class=\"language-bash\" data-lang=\"bash\">vagrant ssh <span class=\"nt\">-c<\/span> <span class=\"s1\">'specs'<\/span><\/code><\/pre><\/figure>\n\n<script type=\"text\/javascript\" src=\"https:\/\/asciinema.org\/a\/17712.js\" id=\"asciicast-17712\" async=\"\"><\/script>\n\n<p>25 seconds only, we cut the runtime by six, not bad.<\/p>\n\n<h2 id=\"local-continuous-integration\">Local continuous integration<\/h2>\n\n<p>Now that we have reasonable playbook test run time, we can add local continuous integration to our setup.\nWe will use <a href=\"https:\/\/github.com\/guard\/guard\">Guard<\/a> for this.<\/p>\n\n<p>Assuming you have a ruby environment setup, just install <code class=\"highlighter-rouge\">guard<\/code> and <code class=\"highlighter-rouge\">guard-shell<\/code> gems.<\/p>\n\n<figure class=\"highlight\"><pre><code class=\"language-bash\" data-lang=\"bash\">gem install guard guard-shell <span class=\"nt\">--no-ri<\/span> <span class=\"nt\">--no-rdoc<\/span><\/code><\/pre><\/figure>\n\n<p>Then create a <code class=\"highlighter-rouge\">Guardfile<\/code> in the roles top directory, with the following content:<\/p>\n\n<figure class=\"highlight\"><pre><code class=\"language-ruby\" data-lang=\"ruby\"><span class=\"c1\"># -- -*- mode: ruby; -*-<\/span>\n<span class=\"n\">guard<\/span> <span class=\"ss\">:shell<\/span> <span class=\"k\">do<\/span>\n  <span class=\"n\">watch<\/span><span class=\"p\">(<\/span><span class=\"sr\">%r{^(?!tests).*\/.*<\/span><span class=\"se\">\\.<\/span><span class=\"sr\">yml$}<\/span><span class=\"p\">)<\/span> <span class=\"k\">do<\/span> <span class=\"o\">|<\/span><span class=\"n\">m<\/span><span class=\"o\">|<\/span>\n    <span class=\"nb\">puts<\/span> <span class=\"s2\">\"<\/span><span class=\"si\">#{<\/span><span class=\"n\">m<\/span><span class=\"p\">[<\/span><span class=\"mi\">0<\/span><span class=\"p\">]<\/span><span class=\"si\">}<\/span><span class=\"s2\"> changed - running tests\"<\/span>\n    <span class=\"nb\">system<\/span><span class=\"p\">(<\/span><span class=\"s1\">'vagrant ssh -c \"specs -p\"'<\/span><span class=\"p\">)<\/span>\n  <span class=\"k\">end<\/span>\n<span class=\"k\">end<\/span><\/code><\/pre><\/figure>\n\n<p>This file will ask <code class=\"highlighter-rouge\">guard<\/code> to execute <code class=\"highlighter-rouge\">vagrant ssh -c \"specs -p\"<\/code> everytime it\ndetects a change in a file ending with <code class=\"highlighter-rouge\">.yml<\/code> in the project\u2019s subdirectories.\nNote that we excluded the <code class=\"highlighter-rouge\">tests<\/code> directory since it contains somewhere a\n<code class=\"highlighter-rouge\">test.yml<\/code> playbook file generated by RoleSpec at run time. If we don\u2019t exclude\nit from the guard watch, the test will loop forever.<\/p>\n\n<p>Now run <code class=\"highlighter-rouge\">guard<\/code>, change a file (.e.g. <code class=\"highlighter-rouge\">touch tasks\/main.yml<\/code>), and see what happens.<\/p>\n\n<p>In the next part, we will add some more tests, and see what we can do with <a href=\"rolespec\">RoleSpec<\/a>.<\/p>\n\n\n    <p><a href=\"https:\/\/leucos.github.io\/articles\/testing-ansible-roles-part-2\/\">Testing Ansible roles, part 2<\/a> was originally published by Michel Blanc at <a href=\"https:\/\/leucos.github.io\">Random stuff<\/a> on March 15, 2015.<\/p>\n  "},{"title":{"@attributes":{"type":"html"}},"link":{"@attributes":{"rel":"alternate","type":"text\/html","href":"https:\/\/leucos.github.io\/articles\/testing-ansible-roles-part-1\/"}},"id":"https:\/\/leucos.github.io\/articles\/testing-ansible-roles-part-1","published":"2015-03-14T00:00:00+00:00","updated":"2015-03-14T00:00:00+00:00","author":{"name":"Michel Blanc","uri":"https:\/\/leucos.github.io","email":"mb@mbnet.fr"},"content":"\n    <p><a href=\"rolespec\">RoleSpec<\/a> does a great job helping out testing your roles. It is\nmaintained and used primarily to test the <a href=\"debops\">DebOps<\/a> role suite by the\nfine folks hanging out in <a href=\"irc:\/\/irc.freenode.net\/debops\">#debops<\/a> IRC channel.\nRoleSpec handles all the boiler plate to run tests (installing the right version\nof Ansible, adjusting paths, taking care of the inventory, wrapping your role in\na playbook, \u2026) and privides a simple DSL to write tests.<\/p>\n\n<p>However, in its current state, RoleSpec is mostly intended to run a test suite\non travis. And this test suite is separated from your role.<\/p>\n\n<p>I personally prefer to have my role tests along the Ansible role, in a <code class=\"highlighter-rouge\">tests<\/code>\ndirectory.<\/p>\n\n<p>We see below how we can achieve this with RoleSpec, and will leverage Vagrant\nfor this. We\u2019ll also use Guard to continuously test our role while writing it.<\/p>\n\n<h2 id=\"a-simple-role\">A simple role<\/h2>\n\n<p>Let\u2019s start by creating a simple nginx role:<\/p>\n\n<figure class=\"highlight\"><pre><code class=\"language-bash\" data-lang=\"bash\">mkdir <span class=\"nt\">-p<\/span> ansible-nginx\/<span class=\"o\">{<\/span>defaults,handlers,tasks,templates,tests\/ansible-nginx\/inventory<span class=\"o\">}<\/span><\/code><\/pre><\/figure>\n\n<p>The <code class=\"highlighter-rouge\">tests<\/code> directory will be used for our tests later.<\/p>\n\n<p>If you already have a role want to convert it, create the <code class=\"highlighter-rouge\">tests\/ansible-\nnginx\/<\/code> directory and skip straight to \n<a href=\"\/articles\/testing-ansible-roles-part-2\/\">part 2<\/a>.<\/p>\n\n<h3 id=\"defaults\">Defaults<\/h3>\n\n<p>In <code class=\"highlighter-rouge\">default\/main.yml<\/code>, we\u2019ll declare a few default values for our role. We won\u2019t do much,\nin our role, just install nginx and set a few variables, so let\u2019s keep this\nsimple:<\/p>\n\n<figure class=\"highlight\"><pre><code class=\"language-yaml\" data-lang=\"yaml\"><span class=\"na\">nginx_root<\/span><span class=\"pi\">:<\/span> <span class=\"s\">\/var\/lib\/nginx\/<\/span>\n<span class=\"na\">nginx_worker_connections<\/span><span class=\"pi\">:<\/span> <span class=\"s\">1024<\/span>\n<span class=\"na\">nginx_ie8_support<\/span><span class=\"pi\">:<\/span> <span class=\"s\">yes<\/span>\n<span class=\"na\">nginx_port<\/span><span class=\"pi\">:<\/span> <span class=\"s\">80<\/span><\/code><\/pre><\/figure>\n\n<h3 id=\"handlers\">Handlers<\/h3>\n\n<p>For the handlers part, <code class=\"highlighter-rouge\">handlers\/main.yml<\/code> will contain a basic restart handler, followed by a port check for good measure:<\/p>\n\n<figure class=\"highlight\"><pre><code class=\"language-yaml\" data-lang=\"yaml\"><span class=\"pi\">-<\/span> <span class=\"na\">name<\/span><span class=\"pi\">:<\/span> <span class=\"s\">Restart nginx<\/span>\n  <span class=\"na\">action<\/span><span class=\"pi\">:<\/span> <span class=\"s\">service name=nginx state=restarted<\/span>\n  <span class=\"na\">notify<\/span><span class=\"pi\">:<\/span> <span class=\"s\">Check nginx<\/span>\n\n<span class=\"pi\">-<\/span> <span class=\"na\">name<\/span><span class=\"pi\">:<\/span> <span class=\"s\">Check nginx<\/span>\n  <span class=\"na\">wait_for<\/span><span class=\"pi\">:<\/span> <span class=\"s\">port={{ nginx_port }} delay=5 timeout=10<\/span><\/code><\/pre><\/figure>\n\n<h3 id=\"tasks\">Tasks<\/h3>\n\n<p>Now the task part. I always put my tasks in a separate file, and include this\nfile from <code class=\"highlighter-rouge\">main.yml<\/code>. This trick will allow you to set a tag for the whole\nincluded file, like so:<\/p>\n\n<figure class=\"highlight\"><pre><code class=\"language-yaml\" data-lang=\"yaml\"><span class=\"pi\">-<\/span> <span class=\"na\">include<\/span><span class=\"pi\">:<\/span> <span class=\"s\">nginx.yml tags=nginx<\/span><\/code><\/pre><\/figure>\n\n<p>And then, in <code class=\"highlighter-rouge\">nginx.yml<\/code>, put the real tasks:<\/p>\n\n<figure class=\"highlight\"><pre><code class=\"language-yaml\" data-lang=\"yaml\"><span class=\"pi\">-<\/span> <span class=\"na\">name<\/span><span class=\"pi\">:<\/span> <span class=\"s\">Adds nginx ppa<\/span>\n  <span class=\"na\">apt_repository<\/span><span class=\"pi\">:<\/span>\n    <span class=\"s\">repo=ppa:nginx\/stable<\/span>\n\n<span class=\"pi\">-<\/span> <span class=\"na\">name<\/span><span class=\"pi\">:<\/span> <span class=\"s\">Adds PPA key<\/span>\n  <span class=\"na\">apt_key<\/span><span class=\"pi\">:<\/span> \n    <span class=\"s\">url=http:\/\/keyserver.ubuntu.com:11371\/pks\/lookup?op=get&amp;search=0x00A6F0A3C300EE8C<\/span>\n    <span class=\"s\">state=present<\/span>\n\n<span class=\"pi\">-<\/span> <span class=\"na\">name<\/span><span class=\"pi\">:<\/span> <span class=\"s\">Installs nginx<\/span>\n  <span class=\"na\">apt<\/span><span class=\"pi\">:<\/span>\n    <span class=\"s\">pkg=nginx-full<\/span>\n    <span class=\"s\">state=latest<\/span>\n\n<span class=\"pi\">-<\/span> <span class=\"na\">name<\/span><span class=\"pi\">:<\/span> <span class=\"s\">Writes nginx.conf<\/span>\n  <span class=\"na\">template<\/span><span class=\"pi\">:<\/span> \n    <span class=\"s\">src=\"..\/templates\/nginx.conf.j2\"<\/span>\n    <span class=\"s\">dest=\/etc\/nginx\/nginx.conf<\/span>\n    <span class=\"s\">validate='nginx -tc %s'<\/span>\n  <span class=\"na\">notify<\/span><span class=\"pi\">:<\/span>\n  <span class=\"pi\">-<\/span> <span class=\"s\">Restart nginx<\/span>\n\n<span class=\"pi\">-<\/span> <span class=\"na\">name<\/span><span class=\"pi\">:<\/span> <span class=\"s\">Replaces nginx default server<\/span>\n  <span class=\"na\">template<\/span><span class=\"pi\">:<\/span>\n    <span class=\"s\">src=\"..\/templates\/default.j2\"<\/span>\n    <span class=\"s\">dest=\/etc\/nginx\/sites-available\/default<\/span>\n  <span class=\"na\">notify<\/span><span class=\"pi\">:<\/span>\n    <span class=\"pi\">-<\/span> <span class=\"s\">Restart nginx<\/span><\/code><\/pre><\/figure>\n\n<h3 id=\"templates\">Templates<\/h3>\n\n<p>We just need to add 2 templates, and our role will be ready. The first one is the main <code class=\"highlighter-rouge\">nginx.conf.j2<\/code> file:<\/p>\n\n<figure class=\"highlight\"><pre><code class=\"language-jinja\" data-lang=\"jinja\">user www-data;\nworker_processes <span class=\"cp\">{{<\/span> <span class=\"nv\">ansible_processor_count<\/span> <span class=\"cp\">}}<\/span>;\n\npid         \/var\/run\/nginx.pid;\n\nevents {\n    worker_connections <span class=\"cp\">{{<\/span> <span class=\"nv\">nginx_worker_connections<\/span> <span class=\"cp\">}}<\/span>;\n    # multi_accept on;\n}\n\nhttp {\n    ##\n    # Basic Settings\n    ##\n    sendfile    on;\n    tcp_nopush  on;\n    tcp_nodelay on;\n\n    # SSL stuff\n    ssl_protocols TLSv1 TLSv1.1 TLSv1.2;\n\n<span class=\"cp\">{%<\/span> <span class=\"k\">if<\/span> <span class=\"nv\">nginx_ie8_support<\/span> <span class=\"cp\">%}<\/span>\n    ssl_ciphers \"ECDHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-SHA384:ECDHE-RSA-AES128-SHA256:ECDHE-RSA-AES256-SHA:ECDHE-RSA-AES128-SHA:DHE-RSA-AES256-SHA256:DHE-RSA-AES128-SHA256:DHE-RSA-AES256-SHA:DHE-RSA-AES128-SHA:ECDHE-RSA-DES-CBC3-SHA:EDH-RSA-DES-CBC3-SHA:AES256-GCM-SHA384:AES128-GCM-SHA256:AES256-SHA256:AES128-SHA256:AES256-SHA:AES128-SHA:DES-CBC3-SHA:HIGH:!aNULL:!eNULL:!EXPORT:!DES:!MD5:!PSK:!RC4\";\n<span class=\"cp\">{%<\/span> <span class=\"k\">else<\/span> <span class=\"cp\">%}<\/span>\n    ssl_ciphers \"EECDH+ECDSA+AESGCM:EECDH+aRSA+AESGCM:EECDH+ECDSA+SHA384:EECDH+ECDSA+SHA256:EECDH+aRSA+SHA384:EECDH+aRSA+SHA256:EECDH+aRSA+RC4:EECDH:EDH+aRSA:!RC4:!aNULL:!eNULL:!LOW:!3DES:!MD5:!EXPORT:!PSK:!SRP:!DSS\";\n<span class=\"cp\">{%<\/span> <span class=\"k\">endif<\/span> <span class=\"cp\">%}<\/span>\n\n    ssl_session_cache shared:SSL:32m;\n    ssl_buffer_size 8k;\n    ssl_session_timeout 10m;\n\n    keepalive_timeout     65;\n    types_hash_max_size 2048;\n\n    server_tokens off;\n\n    include       \/etc\/nginx\/mime.types;\n    default_type  application\/octet-stream;\n\n    ##\n    # Logging Settings\n    ##\n    access_log \/var\/log\/nginx\/access.log;\n    error_log \/var\/log\/nginx\/error.log;\n\n    ##\n    # Gzip Settings\n    ##\n    gzip on;\n    gzip_disable \"msie6\";\n    gzip_comp_level 6;\n    gzip_buffers 16 8k;\n    gzip_types  application\/javascript \n                application\/json \n                application\/x-javascript \n                application\/xml \n                application\/xml+rss \n                image\/svg+xml\n                text\/css \n                text\/plain\n                text\/xml \n                text\/javascript;\n\n    ##\n    # If HTTPS, then set a variable so it can be passed along.\n    ##\n    map $scheme $server_https {\n        default off;\n        https on;\n    }\n\n    ##\n    # Virtual Host Configs\n    ##\n    include \/etc\/nginx\/conf.d\/*.conf;\n    include \/etc\/nginx\/sites-enabled\/*;\n}<\/code><\/pre><\/figure>\n\n<p>The file is a bit long, but it just contains basic settings. Note that we\u2019re\naligning the number of worker processes to the number of processors reported by\nAnsible for the host.<\/p>\n\n<p>We are also switching cipher suites depending on whether we want to support IE8\nor not.<\/p>\n\n<p>Then, we just add a default virtualhost on our server:<\/p>\n\n<figure class=\"highlight\"><pre><code class=\"language-nginx\" data-lang=\"nginx\"><span class=\"k\">server<\/span> <span class=\"p\">{<\/span>\n  <span class=\"kn\">listen<\/span> <span class=\"p\">{<\/span><span class=\"err\">{<\/span> <span class=\"kn\">nginx_port<\/span> <span class=\"err\">}}<\/span><span class=\"p\">;<\/span>\n\n  <span class=\"kn\">root<\/span> <span class=\"p\">{<\/span><span class=\"err\">{<\/span> <span class=\"kn\">nginx_root<\/span> <span class=\"err\">}}<\/span><span class=\"p\">;<\/span>\n  <span class=\"kn\">index<\/span> <span class=\"s\">index.html<\/span> <span class=\"s\">index.htm<\/span><span class=\"p\">;<\/span>\n\n  <span class=\"c1\"># Make site accessible from http:\/\/localhost\/\n<\/span>  <span class=\"kn\">server_name<\/span> <span class=\"s\">_<\/span><span class=\"p\">;<\/span>\n\n  <span class=\"kn\">location<\/span> <span class=\"n\">\/<\/span> <span class=\"p\">{<\/span>\n    <span class=\"kn\">try_files<\/span> <span class=\"nv\">$uri<\/span> <span class=\"nv\">$uri<\/span><span class=\"n\">\/<\/span> <span class=\"n\">\/index.php?q=<\/span><span class=\"nv\">$uri<\/span><span class=\"s\">&amp;<\/span><span class=\"nv\">$args<\/span><span class=\"p\">;<\/span>\n  <span class=\"p\">}<\/span>\n\n  <span class=\"kn\">error_page<\/span> <span class=\"mi\">404<\/span> <span class=\"n\">\/404.html<\/span><span class=\"p\">;<\/span>\n\n  <span class=\"c1\"># redirect server error pages to the static page \/50x.html\n<\/span>  <span class=\"c1\">#\n<\/span>  <span class=\"kn\">error_page<\/span> <span class=\"mi\">500<\/span> <span class=\"mi\">502<\/span> <span class=\"mi\">503<\/span> <span class=\"mi\">504<\/span> <span class=\"n\">\/50x.html<\/span><span class=\"p\">;<\/span>\n  <span class=\"kn\">location<\/span> <span class=\"p\">=<\/span> <span class=\"n\">\/50x.html<\/span> <span class=\"p\">{<\/span>\n    <span class=\"kn\">root<\/span> <span class=\"n\">\/usr\/share\/nginx\/html\/<\/span><span class=\"p\">;<\/span>\n  <span class=\"p\">}<\/span>\n\n  <span class=\"c1\"># deny access to .htaccess files, if Apache's document root\n<\/span>  <span class=\"c1\"># concurs with nginx's one\n<\/span>  <span class=\"c1\">#\n<\/span>  <span class=\"kn\">location<\/span> <span class=\"p\">~<\/span> <span class=\"sr\">\/\\.ht<\/span> <span class=\"p\">{<\/span>\n    <span class=\"kn\">deny<\/span> <span class=\"s\">all<\/span><span class=\"p\">;<\/span>\n  <span class=\"p\">}<\/span>\n<span class=\"p\">}<\/span><\/code><\/pre><\/figure>\n\n<p>Our role is now ready. We can now setup the tooling for our tests as explained in <a href=\"\/articles\/testing-ansible-roles-part-2\/\">part 2<\/a><\/p>\n\n\n    <p><a href=\"https:\/\/leucos.github.io\/articles\/testing-ansible-roles-part-1\/\">Testing Ansible roles, part 1<\/a> was originally published by Michel Blanc at <a href=\"https:\/\/leucos.github.io\">Random stuff<\/a> on March 14, 2015.<\/p>\n  "},{"title":{"@attributes":{"type":"html"}},"link":{"@attributes":{"rel":"alternate","type":"text\/html","href":"https:\/\/leucos.github.io\/articles\/expiring-redis-cache-from-ruby\/"}},"id":"https:\/\/leucos.github.io\/articles\/expiring-redis-cache-from-ruby","updated":"2015-03-01T00:00:00-00:00","published":"2015-03-01T00:00:00+00:00","author":{"name":"Michel Blanc","uri":"https:\/\/leucos.github.io","email":"mb@mbnet.fr"},"content":"\n    <h2 id=\"redis-as-padrino-cache\">REDIS as Padrino cache<\/h2>\n\n<p>Using REDIS as and application cache is very handy. You can easily use\nit in, say, <a href=\"http:\/\/www.padrinorb.com\/\">Padrino<\/a> like this:<\/p>\n\n<figure class=\"highlight\"><pre><code class=\"language-ruby\" data-lang=\"ruby\"><span class=\"k\">module<\/span> <span class=\"nn\">MyApp<\/span>\n  <span class=\"k\">class<\/span> <span class=\"nc\">App<\/span> <span class=\"o\">&lt;<\/span> <span class=\"no\">Padrino<\/span><span class=\"o\">::<\/span><span class=\"no\">Application<\/span>\n    <span class=\"n\">enable<\/span> <span class=\"ss\">:caching<\/span>\n    <span class=\"n\">set<\/span> <span class=\"ss\">:cache<\/span><span class=\"p\">,<\/span> <span class=\"no\">Padrino<\/span><span class=\"o\">::<\/span><span class=\"no\">Cache<\/span><span class=\"p\">.<\/span><span class=\"nf\">new<\/span><span class=\"p\">(<\/span> <span class=\"ss\">:Redis<\/span><span class=\"p\">,<\/span> \n                                    <span class=\"ss\">:host<\/span> <span class=\"o\">=&gt;<\/span> <span class=\"no\">ENV<\/span><span class=\"p\">[<\/span><span class=\"s1\">'REDIS_SERVER'<\/span><span class=\"p\">],<\/span>\n                                    <span class=\"ss\">:port<\/span> <span class=\"o\">=&gt;<\/span> <span class=\"no\">ENV<\/span><span class=\"p\">[<\/span><span class=\"s1\">'REDIS_PORT'<\/span><span class=\"p\">],<\/span>\n                                    <span class=\"ss\">:db<\/span> <span class=\"o\">=&gt;<\/span> <span class=\"mi\">0<\/span><span class=\"p\">)<\/span>\n  <span class=\"k\">end<\/span>\n<span class=\"k\">end<\/span>\n\n<span class=\"no\">MyApp<\/span><span class=\"o\">::<\/span><span class=\"no\">App<\/span><span class=\"p\">.<\/span><span class=\"nf\">controllers<\/span> <span class=\"ss\">:test<\/span> <span class=\"k\">do<\/span>\n  <span class=\"n\">get<\/span> <span class=\"ss\">:index<\/span><span class=\"p\">,<\/span> <span class=\"ss\">:cache<\/span> <span class=\"o\">=&gt;<\/span> <span class=\"kp\">true<\/span> <span class=\"k\">do<\/span>\n    <span class=\"n\">cache_key<\/span> <span class=\"p\">{<\/span> <span class=\"n\">current_account<\/span><span class=\"p\">.<\/span><span class=\"nf\">email<\/span> <span class=\"o\">+<\/span> <span class=\"s2\">\":test:index\"<\/span> <span class=\"p\">}<\/span>\n    <span class=\"n\">expires<\/span> <span class=\"mi\">3600<\/span>\n    <span class=\"vi\">@title<\/span> <span class=\"o\">=<\/span> <span class=\"s2\">\"Some page with expensively computed values\"<\/span>\n    <span class=\"n\">render<\/span> <span class=\"s1\">'test\/index'<\/span>\n  <span class=\"k\">end<\/span>\n<span class=\"k\">end<\/span><\/code><\/pre><\/figure>\n\n<p>Note that we have a specific cache entry for each user\n(<code class=\"highlighter-rouge\">current_account.email<\/code>). For instance, a user with email <code class=\"highlighter-rouge\">foo@bar.com<\/code>\nwill have this entry cached at <code class=\"highlighter-rouge\">foo@bar.com:test:index<\/code><\/p>\n\n<h2 id=\"cache-invalidation\">Cache invalidation<\/h2>\n\n<p>Now, sometimes you need to expire the cache forcibly. For instance,\nlet\u2019s say you know you\u2019ve changed something in the database and that you\ndon\u2019t want stale data to be served, you can invalidate the cache\nmanually. Or may be you want to invalidate a complete user cache at\nlogin time.<\/p>\n\n<p>However, this is not easy in our case, since we want to remove all\nentries matching <code class=\"highlighter-rouge\">*:test:index<\/code> (or <code class=\"highlighter-rouge\">foo@bar.com:*<\/code> if we want to\ncompletely wipe out the user cache).<\/p>\n\n<p>The first idea that comes to mind is to use the Redis <code class=\"highlighter-rouge\">KEYS<\/code> command that can\naccept globs to match key names, like <code class=\"highlighter-rouge\">KEYS foo@bar.com:*<\/code>.<\/p>\n\n<p>But in the documentation<sup id=\"fnref:1\"><a href=\"#fn:1\" class=\"footnote\">1<\/a><\/sup>, you\u2019ll find a big fat warning about KEYS:<\/p>\n\n<blockquote>\n  <p>Warning: consider KEYS as a command that should only be used in production\nenvironments with extreme care. It may ruin performance when it is executed\nagainst large databases. This command is intended for debugging and special\noperations, such as changing your keyspace layout. Don\u2019t use KEYS in your\nregular application code. If you\u2019re looking for a way to find keys in a \nsubset of your keyspace, consider using SCAN or sets.<\/p>\n<\/blockquote>\n\n<p>Scary as it sounds.<\/p>\n\n<h2 id=\"cursors-to-the-rescue\">Cursors to the rescue<\/h2>\n\n<p>REDIS comes with a nice, but not very known feature since v2.8: SCAN. The SCAN\ncommand is a cursor based iterator. You give him a key pattern, and every time\nyou call it, it will return the next set of matching keys, and an index for the\nnext call.<\/p>\n\n<p>Here is a piece of code that can invalidate key wildcards from padrino :<\/p>\n\n<figure class=\"highlight\"><pre><code class=\"language-ruby\" data-lang=\"ruby\"><span class=\"no\">MyApp<\/span><span class=\"o\">::<\/span><span class=\"no\">App<\/span><span class=\"p\">.<\/span><span class=\"nf\">controllers<\/span> <span class=\"ss\">:test<\/span> <span class=\"k\">do<\/span>\n  <span class=\"n\">define_method<\/span> <span class=\"ss\">:invalidate_cache_like<\/span> <span class=\"k\">do<\/span> <span class=\"o\">|<\/span><span class=\"n\">wildcard<\/span><span class=\"o\">|<\/span>\n    <span class=\"n\">r<\/span> <span class=\"o\">=<\/span> <span class=\"no\">Redis<\/span><span class=\"p\">.<\/span><span class=\"nf\">new<\/span><span class=\"p\">(<\/span><span class=\"ss\">:host<\/span> <span class=\"o\">=&gt;<\/span> <span class=\"no\">ENV<\/span><span class=\"p\">[<\/span><span class=\"s1\">'REDIS_SERVER'<\/span><span class=\"p\">],<\/span> <span class=\"ss\">:port<\/span> <span class=\"o\">=&gt;<\/span> <span class=\"no\">ENV<\/span><span class=\"p\">[<\/span><span class=\"s1\">'REDIS_PORT'<\/span><span class=\"p\">])<\/span>\n    <span class=\"n\">cursor<\/span> <span class=\"o\">=<\/span> <span class=\"kp\">nil<\/span>\n\n    <span class=\"k\">while<\/span> <span class=\"n\">cursor<\/span> <span class=\"o\">!=<\/span> <span class=\"s2\">\"0\"<\/span> <span class=\"k\">do<\/span>\n      <span class=\"n\">cursor<\/span><span class=\"p\">,<\/span> <span class=\"n\">keys<\/span> <span class=\"o\">=<\/span> <span class=\"n\">r<\/span><span class=\"p\">.<\/span><span class=\"nf\">scan<\/span><span class=\"p\">(<\/span><span class=\"n\">cursor<\/span><span class=\"o\">||<\/span><span class=\"n\">e<\/span><span class=\"p\">,<\/span> <span class=\"p\">{<\/span> <span class=\"ss\">match: <\/span><span class=\"n\">wildcard<\/span><span class=\"p\">})<\/span>\n\n      <span class=\"n\">keys<\/span><span class=\"p\">.<\/span><span class=\"nf\">each<\/span> <span class=\"k\">do<\/span> <span class=\"o\">|<\/span><span class=\"n\">k<\/span><span class=\"o\">|<\/span>\n        <span class=\"n\">r<\/span><span class=\"p\">.<\/span><span class=\"nf\">del<\/span><span class=\"p\">(<\/span><span class=\"n\">k<\/span><span class=\"p\">)<\/span>\n      <span class=\"k\">end<\/span>\n    <span class=\"k\">end<\/span>\n  <span class=\"k\">end<\/span>\n<span class=\"k\">end<\/span><\/code><\/pre><\/figure>\n\n<p>You can not easily invalidate a cache wildcard calling\n<code class=\"highlighter-rouge\">invalidate_cache_like<\/code>.<\/p>\n\n<p>For instance, at user login, you could call :<\/p>\n\n<figure class=\"highlight\"><pre><code class=\"language-ruby\" data-lang=\"ruby\"><span class=\"sb\">`invalidate_cache_like \"<\/span><span class=\"si\">#{<\/span><span class=\"n\">current_account<\/span><span class=\"p\">.<\/span><span class=\"nf\">email<\/span><span class=\"si\">}<\/span><span class=\"sb\">:test:index\"<\/span><\/code><\/pre><\/figure>\n\n<p>and the user cache is now cleared.<\/p>\n\n<h2 id=\"benchmarking\">Benchmarking<\/h2>\n\n<p>Let\u2019s play with <code class=\"highlighter-rouge\">Benchmark<\/code> a bit to compare <code class=\"highlighter-rouge\">SCAN<\/code> and <code class=\"highlighter-rouge\">KEYS<\/code> performance on a\nmoderately sized database. While we\u2019re at it, we\u2019ll also check these commands using <code class=\"highlighter-rouge\">redis<\/code> and <code class=\"highlighter-rouge\">hiredis<\/code> drivers, to see if it makes any difference.<\/p>\n\n<p>I used the following piece of code for that:<\/p>\n\n<figure class=\"highlight\"><pre><code class=\"language-ruby\" data-lang=\"ruby\"><span class=\"c1\">#!\/bin\/env ruby<\/span>\n\n<span class=\"nb\">require<\/span> <span class=\"s1\">'hiredis'<\/span>\n<span class=\"nb\">require<\/span> <span class=\"s1\">'em-synchrony'<\/span>\n<span class=\"nb\">require<\/span> <span class=\"s1\">'redis'<\/span>\n<span class=\"nb\">require<\/span> <span class=\"s1\">'benchmark'<\/span>\n\n<span class=\"k\">def<\/span> <span class=\"nf\">build_cache<\/span><span class=\"p\">(<\/span><span class=\"n\">redis<\/span><span class=\"p\">)<\/span>\n  <span class=\"p\">(<\/span><span class=\"s1\">'aaaa'<\/span><span class=\"o\">..<\/span><span class=\"s1\">'zzzz'<\/span><span class=\"p\">).<\/span><span class=\"nf\">each<\/span> <span class=\"k\">do<\/span> <span class=\"o\">|<\/span><span class=\"n\">s<\/span><span class=\"o\">|<\/span>\n    <span class=\"n\">redis<\/span><span class=\"p\">.<\/span><span class=\"nf\">set<\/span><span class=\"p\">(<\/span><span class=\"n\">s<\/span><span class=\"p\">,<\/span> <span class=\"mi\">1<\/span><span class=\"p\">)<\/span>\n  <span class=\"k\">end<\/span>\n<span class=\"k\">end<\/span>\n\n<span class=\"k\">def<\/span> <span class=\"nf\">invalidate_cache_cursor<\/span><span class=\"p\">(<\/span><span class=\"n\">redis<\/span><span class=\"p\">,<\/span> <span class=\"n\">wildcard<\/span><span class=\"p\">)<\/span>\n  <span class=\"n\">cursor<\/span> <span class=\"o\">=<\/span> <span class=\"kp\">nil<\/span>\n\n  <span class=\"k\">while<\/span> <span class=\"n\">cursor<\/span> <span class=\"o\">!=<\/span> <span class=\"s2\">\"0\"<\/span> <span class=\"k\">do<\/span>\n    <span class=\"n\">cursor<\/span><span class=\"p\">,<\/span> <span class=\"n\">keys<\/span> <span class=\"o\">=<\/span> <span class=\"n\">redis<\/span><span class=\"p\">.<\/span><span class=\"nf\">scan<\/span><span class=\"p\">(<\/span><span class=\"n\">cursor<\/span><span class=\"o\">||<\/span><span class=\"mi\">0<\/span><span class=\"p\">,<\/span> <span class=\"p\">{<\/span> <span class=\"ss\">match: <\/span><span class=\"n\">wildcard<\/span><span class=\"p\">})<\/span>\n\n    <span class=\"n\">keys<\/span><span class=\"p\">.<\/span><span class=\"nf\">each<\/span> <span class=\"k\">do<\/span> <span class=\"o\">|<\/span><span class=\"n\">k<\/span><span class=\"o\">|<\/span>\n      <span class=\"n\">redis<\/span><span class=\"p\">.<\/span><span class=\"nf\">del<\/span><span class=\"p\">(<\/span><span class=\"n\">k<\/span><span class=\"p\">)<\/span>\n    <span class=\"k\">end<\/span>\n  <span class=\"k\">end<\/span>\n<span class=\"k\">end<\/span>\n\n<span class=\"k\">def<\/span> <span class=\"nf\">invalidate_cache_keys<\/span><span class=\"p\">(<\/span><span class=\"n\">redis<\/span><span class=\"p\">,<\/span> <span class=\"n\">wildcard<\/span><span class=\"p\">)<\/span>\n  <span class=\"n\">redis<\/span><span class=\"p\">.<\/span><span class=\"nf\">keys<\/span><span class=\"p\">(<\/span><span class=\"n\">wildcard<\/span><span class=\"p\">).<\/span><span class=\"nf\">each<\/span> <span class=\"k\">do<\/span> <span class=\"o\">|<\/span><span class=\"n\">k<\/span><span class=\"o\">|<\/span>\n    <span class=\"n\">redis<\/span><span class=\"p\">.<\/span><span class=\"nf\">del<\/span><span class=\"p\">(<\/span><span class=\"n\">k<\/span><span class=\"p\">)<\/span>\n  <span class=\"k\">end<\/span>\n<span class=\"k\">end<\/span>\n\n<span class=\"n\">hiredis<\/span> <span class=\"o\">=<\/span> <span class=\"no\">Redis<\/span><span class=\"p\">.<\/span><span class=\"nf\">new<\/span><span class=\"p\">(<\/span><span class=\"ss\">:driver<\/span> <span class=\"o\">=&gt;<\/span> <span class=\"ss\">:hiredis<\/span><span class=\"p\">)<\/span>\n<span class=\"n\">redis<\/span> <span class=\"o\">=<\/span> <span class=\"no\">Redis<\/span><span class=\"p\">.<\/span><span class=\"nf\">new<\/span><span class=\"p\">()<\/span>\n\n<span class=\"no\">Benchmark<\/span><span class=\"p\">.<\/span><span class=\"nf\">bm<\/span><span class=\"p\">(<\/span><span class=\"mi\">22<\/span><span class=\"p\">)<\/span> <span class=\"k\">do<\/span> <span class=\"o\">|<\/span><span class=\"n\">x<\/span><span class=\"o\">|<\/span>\n  <span class=\"p\">[<\/span><span class=\"ss\">:ruby<\/span><span class=\"p\">,<\/span> <span class=\"ss\">:hiredis<\/span><span class=\"p\">].<\/span><span class=\"nf\">each<\/span> <span class=\"k\">do<\/span> <span class=\"o\">|<\/span><span class=\"n\">d<\/span><span class=\"o\">|<\/span>\n    <span class=\"n\">r<\/span> <span class=\"o\">=<\/span> <span class=\"no\">Redis<\/span><span class=\"p\">.<\/span><span class=\"nf\">new<\/span><span class=\"p\">(<\/span><span class=\"ss\">:driver<\/span> <span class=\"o\">=&gt;<\/span> <span class=\"n\">d<\/span><span class=\"p\">)<\/span>\n    <span class=\"n\">build_cache<\/span><span class=\"p\">(<\/span><span class=\"n\">r<\/span><span class=\"p\">)<\/span>\n    <span class=\"n\">x<\/span><span class=\"p\">.<\/span><span class=\"nf\">report<\/span><span class=\"p\">(<\/span><span class=\"s2\">\"looping (<\/span><span class=\"si\">#{<\/span><span class=\"n\">d<\/span><span class=\"si\">}<\/span><span class=\"s2\">):\"<\/span><span class=\"p\">)<\/span> <span class=\"p\">{<\/span>\n      <span class=\"p\">(<\/span><span class=\"s1\">'aaa'<\/span><span class=\"o\">..<\/span><span class=\"s1\">'aaz'<\/span><span class=\"p\">).<\/span><span class=\"nf\">each<\/span> <span class=\"k\">do<\/span> <span class=\"o\">|<\/span><span class=\"n\">l<\/span><span class=\"o\">|<\/span>\n        <span class=\"n\">invalidate_cache_keys<\/span><span class=\"p\">(<\/span><span class=\"n\">r<\/span><span class=\"p\">,<\/span> <span class=\"s2\">\"<\/span><span class=\"si\">#{<\/span><span class=\"n\">l<\/span><span class=\"si\">}<\/span><span class=\"s2\">*\"<\/span><span class=\"p\">)<\/span>\n      <span class=\"k\">end<\/span>\n    <span class=\"p\">}<\/span>\n    <span class=\"n\">build_cache<\/span><span class=\"p\">(<\/span><span class=\"n\">r<\/span><span class=\"p\">)<\/span>\n    <span class=\"n\">x<\/span><span class=\"p\">.<\/span><span class=\"nf\">report<\/span><span class=\"p\">(<\/span><span class=\"s2\">\"scanning (<\/span><span class=\"si\">#{<\/span><span class=\"n\">d<\/span><span class=\"si\">}<\/span><span class=\"s2\">):\"<\/span><span class=\"p\">)<\/span> <span class=\"p\">{<\/span>\n      <span class=\"p\">(<\/span><span class=\"s1\">'aaa'<\/span><span class=\"o\">..<\/span><span class=\"s1\">'aaz'<\/span><span class=\"p\">).<\/span><span class=\"nf\">each<\/span> <span class=\"k\">do<\/span> <span class=\"o\">|<\/span><span class=\"n\">l<\/span><span class=\"o\">|<\/span>\n        <span class=\"n\">invalidate_cache_cursor<\/span><span class=\"p\">(<\/span><span class=\"n\">r<\/span><span class=\"p\">,<\/span> <span class=\"s2\">\"<\/span><span class=\"si\">#{<\/span><span class=\"n\">l<\/span><span class=\"si\">}<\/span><span class=\"s2\">*\"<\/span><span class=\"p\">)<\/span>\n      <span class=\"k\">end<\/span>\n    <span class=\"p\">}<\/span>\n  <span class=\"k\">end<\/span>\n<span class=\"k\">end<\/span><\/code><\/pre><\/figure>\n\n<p>After a few minutes running, I got those surprising results:<\/p>\n\n<div class=\"highlighter-rouge\"><div class=\"highlight\"><pre class=\"highlight\"><code>$ .\/redis-expire-wildcard.rb \n                            user     system      total        real\nlooping (ruby):          0.040000   0.010000   0.050000 (  1.059056)\nscanning (ruby):        49.000000  11.490000  60.490000 ( 61.113561)\nlooping (hiredis):       0.020000   0.010000   0.030000 (  1.073681)\nscanning (hiredis):     19.680000  12.880000  32.560000 ( 44.972220)\n<\/code><\/pre><\/div><\/div>\n\n<p>First, there is no much improvements using <code class=\"highlighter-rouge\">hiredis<\/code> over <code class=\"highlighter-rouge\">redis<\/code> when looping\nin our case. This sounds legit, since we loop only 26 times here and the\n<code class=\"highlighter-rouge\">hiredis<\/code> performance benefit doesn\u2019t rise with so few commands (<code class=\"highlighter-rouge\">hiredis<\/code> does\na much more better job if you change the tested range so more commands are\nissued).<\/p>\n\n<p>Second, using <code class=\"highlighter-rouge\">SCAN<\/code> here is <em>much<\/em> slower than using <code class=\"highlighter-rouge\">KEYS<\/code> !<\/p>\n\n<p>So why use <code class=\"highlighter-rouge\">SCAN<\/code> instead of <code class=\"highlighter-rouge\">KEYS<\/code> ? The problem with <code class=\"highlighter-rouge\">KEYS<\/code> is that it will block your server while retrieving all the keys. The cursor based approach will return small chunks of keys and won\u2019t block the server for the time of a whole key scan.<\/p>\n\n<p>However, handling cursor based expiration can be tricky in a web application.\nSince it takes so much longer (but is friendlier to Redis), you might have to\nhandle it in a separate task from your application process (in Sidekiq for instance).<\/p>\n\n<p>It all depends on your app. You can start using simply <code class=\"highlighter-rouge\">KEYS<\/code>, but will have to\nkeep in mind that cursors will be needed if usage or concurrent trafic rises and\nmonitor your Redis statistics for this.<\/p>\n\n<div class=\"footnotes\">\n  <ol>\n    <li id=\"fn:1\">\n      <p><a href=\"http:\/\/www.padrinorb.com\/\">http:\/\/www.padrinorb.com\/<\/a>\u00a0<a href=\"#fnref:1\" class=\"reversefootnote\">&#8617;<\/a><\/p>\n    <\/li>\n  <\/ol>\n<\/div>\n\n    <p><a href=\"https:\/\/leucos.github.io\/articles\/expiring-redis-cache-from-ruby\/\">Invalidating REDIS cache from Ruby<\/a> was originally published by Michel Blanc at <a href=\"https:\/\/leucos.github.io\">Random stuff<\/a> on March 01, 2015.<\/p>\n  "},{"title":{"@attributes":{"type":"html"}},"link":{"@attributes":{"rel":"alternate","type":"text\/html","href":"https:\/\/leucos.github.io\/articles\/restoring-arch-bootloader\/"}},"id":"https:\/\/leucos.github.io\/articles\/restoring-arch-bootloader","published":"2015-02-14T00:00:00+00:00","updated":"2015-02-14T00:00:00+00:00","author":{"name":"Michel Blanc","uri":"https:\/\/leucos.github.io","email":"mb@mbnet.fr"},"content":"\n    <ul>\n  <li>\n    <p>Grab latest Arch, create a bootable key (do this before you\u2019re doomed)<sup id=\"fnref:1\"><a href=\"#fn:1\" class=\"footnote\">1<\/a><\/sup>.<\/p>\n  <\/li>\n  <li>\n    <p>Press F2 at boot, change the boot order to start on the key in <strong>UEFI<\/strong> mode<\/p>\n  <\/li>\n  <li>\n    <p>Boot on Arch, then<\/p>\n  <\/li>\n<\/ul>\n\n<script src=\"https:\/\/gist.github.com\/leucos\/3f63c07d8326309d7fb1.js\"> <\/script>\n\n<ul>\n  <li>Cross fingers\u2026<\/li>\n<\/ul>\n\n<p>In case you need to reinstall all the things<sup id=\"fnref:2\"><a href=\"#fn:2\" class=\"footnote\">2<\/a><\/sup>:<\/p>\n\n<figure class=\"highlight\"><pre><code class=\"language-bash\" data-lang=\"bash\"><span class=\"nb\">sudo <\/span>pacman <span class=\"nt\">-Sy<\/span> <span class=\"sb\">`<\/span>yaourt <span class=\"nt\">-Q<\/span> | <span class=\"nb\">grep<\/span> <span class=\"nt\">-v<\/span> <span class=\"s1\">'^aur'<\/span>| <span class=\"nb\">grep<\/span> <span class=\"nt\">-v<\/span> <span class=\"s1\">'^local'<\/span> | cut <span class=\"nt\">-f2<\/span>\n<span class=\"nt\">-d<\/span><span class=\"s1\">'\/'<\/span> | awk <span class=\"s1\">'{ print $1 }'<\/span><span class=\"sb\">`<\/span><\/code><\/pre><\/figure>\n\n<div class=\"footnotes\">\n  <ol>\n    <li id=\"fn:1\">\n      <p>Multisystem is pretty handy for this. You can put several OSes on the key, and choose what to boot. <a href=\"http:\/\/liveusb.info\/dotclear\/\">http:\/\/liveusb.info\/dotclear\/<\/a>\u00a0<a href=\"#fnref:1\" class=\"reversefootnote\">&#8617;<\/a><\/p>\n    <\/li>\n    <li id=\"fn:2\">\n      <p>if you don\u2019t use <code class=\"highlighter-rouge\">yaourt<\/code>, remove the grep part\u00a0<a href=\"#fnref:2\" class=\"reversefootnote\">&#8617;<\/a><\/p>\n    <\/li>\n  <\/ol>\n<\/div>\n\n    <p><a href=\"https:\/\/leucos.github.io\/articles\/restoring-arch-bootloader\/\">Restoring Arch bootloader for the future self<\/a> was originally published by Michel Blanc at <a href=\"https:\/\/leucos.github.io\">Random stuff<\/a> on February 14, 2015.<\/p>\n  "}]}