{"title":"The Code Ship","link":[{"@attributes":{"href":"https:\/\/www.thecodeship.com\/","rel":"alternate"}},{"@attributes":{"href":"https:\/\/www.thecodeship.com\/feed\/all.atom.xml","rel":"self"}}],"id":"https:\/\/www.thecodeship.com\/","updated":"2020-06-11T21:06:00+03:00","entry":[{"title":"Transforming browser HAR logs into an analysis friendly format in Javascript","link":{"@attributes":{"href":"https:\/\/www.thecodeship.com\/web-development\/transforming-browser-har-logs-into-analysis-friendly-format\/","rel":"alternate"}},"published":"2020-06-11T21:06:00+03:00","updated":"2020-06-11T21:06:00+03:00","author":{"name":"Ayman Farhat"},"id":"tag:www.thecodeship.com,2020-06-11:\/web-development\/transforming-browser-har-logs-into-analysis-friendly-format\/","summary":"<p>HAR files extracted from HTTP sessions include a lot of useful data that can be utilized for creating custom network and performance audits. Their format though, is not that compatible with analysis tools like Pandas or Tableau out of the box. In this post, I go over a super easy approach towards parsing and transforming them via Objectron in JS.<\/p>","content":"<p>When working on front-end web performance optimizations, there are several tools out there to help in creating performance audits, identifying opportunities and estimating potential gains. <\/p>\n<p>At many times though, I find myself needing a bit more flexibility on building out custom analysis reports from raw data, especially if the analysis involves looking at resources across multiple pages of a user journey and understanding the requests made.<\/p>\n<p>While I wont be delving much into the analysis process in this article, I'll be covering the first step towards enabling that, which is extracting and transforming this data from a browser output to make it consumable by data analysis libraries such as pandas. The <a href=\"https:\/\/github.com\/mena-devs\/objectron\">Objectron<\/a> JS module will makes such task very simple to perform.<\/p>\n<h2>Introducing .HAR files<\/h2>\n<p>Several modern browsers allow the record and export of HTTP sessions via HAR files. The <a href=\"http:\/\/www.softwareishard.com\/blog\/har-12-spec\/\">HAR file specification<\/a> is a standard file used by several HTTP session tools to export captured data. The format is basically a JSON object with a particular field distribution. Some important characteristics of HAR files to note:<\/p>\n<ul>\n<li>They are usually pretty large in size with everything included in them, including request and response bodies<\/li>\n<li>They contain sensitive data including your cookies and whatever is in the requests you're sending or receiving<\/li>\n<li>While HAR is a great format for sharing and reading HTTP session information between different tools, the raw data usually needs some formatting before it's usable in custom analysis scripts, mainly due to it's non-tabular structure and unneeded data.<\/li>\n<\/ul>\n<h3>To extract from a FireFox session<\/h3>\n<ul>\n<li>Start Firefox Developer Tools in Network mode<\/li>\n<li>Perform a set of navigation (Typically, this would be a user journey\/scenario covering multiple pages)<\/li>\n<li>Save the capture by right-clicking on the grid and choosing \"Save all as HAR\"<\/li>\n<li>Export the capture to a HAR file<\/li>\n<\/ul>\n<p><img alt=\"\" src=\"https:\/\/storage.googleapis.com\/thecodeship\/network-tab-ff.png\"><\/p>\n<p>I've created a sample HAR file from a session covering multiple pages, you can <a href=\"https:\/\/storage.googleapis.com\/thecodeship\/menadevs.com.har\">find it here<\/a>.<\/p>\n<p>The request\/response logs we're looking into extracting is in <code>log.entries<\/code><\/p>\n<h2>Extracting and transforming HAR data to CSV<\/h2>\n<p>First step, is to decide which fields, per log item, make sense to keep that will be useful in the analysis and which drop. This, along with flattening the structure into a table will make the contents of the file usable for plugging into analysis libraries such as pandas.<\/p>\n<p>HAR fields that wouldn't be needed for analysis:<\/p>\n<ul>\n<li>Reply content \/ body<\/li>\n<li>Request payloads<\/li>\n<li>Cookie data <\/li>\n<li>Most header values, with a couple of exceptions<\/li>\n<\/ul>\n<p>Fields that can be interesting for a given log entry:<\/p>\n<ul>\n<li>Page id <\/li>\n<li>Request: method, url, httpVersion, headerSize and bodySize<\/li>\n<li>Response: status, content size, content type, cache control<\/li>\n<li>Header values around cache-control, gzip and content-length<\/li>\n<li>Summary of page request and response timings<\/li>\n<\/ul>\n<h3>Extracting data with Objectron<\/h3>\n<p>To extract and flatten the JSON file into a CSV we'd need to:<\/p>\n<ul>\n<li>Loop through the log entries<\/li>\n<li>For each entry, validate that the needed data exists and push it into a flat object. <\/li>\n<li>A log entry will need to be properly validated for having the required data, otherwise to be discarded. E.g. We might only need logs with GET requests, so that check will need to happen<\/li>\n<\/ul>\n<p>Writing a custom script to do all that actually is simple but prone to a lot of repetition and writing several conditions. Such as:<\/p>\n<ul>\n<li>Write multiple if statements to check if the entry object key has a value to begin with<\/li>\n<li>Check if the object key conforms to a specific pattern or value<\/li>\n<li>Write nested loops to extract specific entries in sub arrays such as those of header data, and do the same checks<\/li>\n<li>Manually insert into a flat object with a custom key name to be used for the table header later<\/li>\n<\/ul>\n<p><a href=\"https:\/\/github.com\/mena-devs\/objectron\">Objectron<\/a> makes this task around validation, extraction, and flattening super simple. With a declarative approach, it facilitates comparing a JS object to a generic model\/pattern to validate and extract matches based on regex patterns. Think of it as regex, but for objects!<\/p>\n<p>I won't be getting into all the basics, but you can learn more about the project, get an introduction and intro to other use cases <a href=\"https:\/\/github.com\/mena-devs\/objectron\">here<\/a>.<\/p>\n<p>Back to the use case of a HAR file, lets consider an individual log entry that looks something like:<\/p>\n<div class=\"highlight\"><pre><span><\/span><span class=\"p\">{<\/span>\n  <span class=\"nt\">&quot;startedDateTime&quot;<\/span><span class=\"p\">:<\/span> <span class=\"s2\">&quot;2020-05-26T06:59:37.215Z&quot;<\/span><span class=\"p\">,<\/span>\n  <span class=\"nt\">&quot;time&quot;<\/span><span class=\"p\">:<\/span> <span class=\"mf\">179.52900001546368<\/span><span class=\"p\">,<\/span>\n  <span class=\"nt\">&quot;request&quot;<\/span><span class=\"p\">:<\/span> <span class=\"p\">{<\/span>\n    <span class=\"nt\">&quot;method&quot;<\/span><span class=\"p\">:<\/span> <span class=\"s2\">&quot;GET&quot;<\/span><span class=\"p\">,<\/span>\n    <span class=\"nt\">&quot;url&quot;<\/span><span class=\"p\">:<\/span> <span class=\"s2\">&quot;https:\/\/menadevs.com\/directory\/users?name=f&quot;<\/span><span class=\"p\">,<\/span>\n    <span class=\"nt\">&quot;httpVersion&quot;<\/span><span class=\"p\">:<\/span> <span class=\"s2\">&quot;HTTP\/1.1&quot;<\/span><span class=\"p\">,<\/span>\n    <span class=\"nt\">&quot;headers&quot;<\/span><span class=\"p\">:<\/span> <span class=\"p\">[<\/span>\n      <span class=\"p\">{<\/span>\n        <span class=\"nt\">&quot;name&quot;<\/span><span class=\"p\">:<\/span> <span class=\"s2\">&quot;Host&quot;<\/span><span class=\"p\">,<\/span>\n        <span class=\"nt\">&quot;value&quot;<\/span><span class=\"p\">:<\/span> <span class=\"s2\">&quot;menadevs.com&quot;<\/span>\n      <span class=\"p\">},<\/span>\n      <span class=\"p\">{<\/span>\n        <span class=\"nt\">&quot;name&quot;<\/span><span class=\"p\">:<\/span> <span class=\"s2\">&quot;Connection&quot;<\/span><span class=\"p\">,<\/span>\n        <span class=\"nt\">&quot;value&quot;<\/span><span class=\"p\">:<\/span> <span class=\"s2\">&quot;keep-alive&quot;<\/span>\n      <span class=\"p\">},<\/span>\n      <span class=\"p\">{<\/span>\n        <span class=\"nt\">&quot;name&quot;<\/span><span class=\"p\">:<\/span> <span class=\"s2\">&quot;Upgrade-Insecure-Requests&quot;<\/span><span class=\"p\">,<\/span>\n        <span class=\"nt\">&quot;value&quot;<\/span><span class=\"p\">:<\/span> <span class=\"s2\">&quot;1&quot;<\/span>\n      <span class=\"p\">},<\/span>\n      <span class=\"p\">{<\/span>\n        <span class=\"nt\">&quot;name&quot;<\/span><span class=\"p\">:<\/span> <span class=\"s2\">&quot;DNT&quot;<\/span><span class=\"p\">,<\/span>\n        <span class=\"nt\">&quot;value&quot;<\/span><span class=\"p\">:<\/span> <span class=\"s2\">&quot;1&quot;<\/span>\n      <span class=\"p\">},<\/span>\n      <span class=\"p\">{<\/span>\n        <span class=\"nt\">&quot;name&quot;<\/span><span class=\"p\">:<\/span> <span class=\"s2\">&quot;User-Agent&quot;<\/span><span class=\"p\">,<\/span>\n        <span class=\"nt\">&quot;value&quot;<\/span><span class=\"p\">:<\/span> <span class=\"s2\">&quot;Mozilla\/5.0 (Macintosh; Intel Mac OS X 10_15_4) AppleWebKit\/537.36 (KHTML, like Gecko) Chrome\/81.0.4044.138 Safari\/537.36&quot;<\/span>\n      <span class=\"p\">},<\/span>\n      <span class=\"p\">{<\/span>\n        <span class=\"nt\">&quot;name&quot;<\/span><span class=\"p\">:<\/span> <span class=\"s2\">&quot;Accept&quot;<\/span><span class=\"p\">,<\/span>\n        <span class=\"nt\">&quot;value&quot;<\/span><span class=\"p\">:<\/span> <span class=\"s2\">&quot;text\/html,application\/xhtml+xml,application\/xml;q=0.9,image\/webp,image\/apng,*\/*;q=0.8,application\/signed-exchange;v=b3;q=0.9&quot;<\/span>\n      <span class=\"p\">},<\/span>\n      <span class=\"p\">{<\/span>\n        <span class=\"nt\">&quot;name&quot;<\/span><span class=\"p\">:<\/span> <span class=\"s2\">&quot;Sec-Fetch-Site&quot;<\/span><span class=\"p\">,<\/span>\n        <span class=\"nt\">&quot;value&quot;<\/span><span class=\"p\">:<\/span> <span class=\"s2\">&quot;same-origin&quot;<\/span>\n      <span class=\"p\">},<\/span>\n      <span class=\"p\">{<\/span>\n        <span class=\"nt\">&quot;name&quot;<\/span><span class=\"p\">:<\/span> <span class=\"s2\">&quot;Sec-Fetch-Mode&quot;<\/span><span class=\"p\">,<\/span>\n        <span class=\"nt\">&quot;value&quot;<\/span><span class=\"p\">:<\/span> <span class=\"s2\">&quot;navigate&quot;<\/span>\n      <span class=\"p\">},<\/span>\n      <span class=\"p\">{<\/span>\n        <span class=\"nt\">&quot;name&quot;<\/span><span class=\"p\">:<\/span> <span class=\"s2\">&quot;Sec-Fetch-User&quot;<\/span><span class=\"p\">,<\/span>\n        <span class=\"nt\">&quot;value&quot;<\/span><span class=\"p\">:<\/span> <span class=\"s2\">&quot;?1&quot;<\/span>\n      <span class=\"p\">},<\/span>\n      <span class=\"p\">{<\/span>\n        <span class=\"nt\">&quot;name&quot;<\/span><span class=\"p\">:<\/span> <span class=\"s2\">&quot;Sec-Fetch-Dest&quot;<\/span><span class=\"p\">,<\/span>\n        <span class=\"nt\">&quot;value&quot;<\/span><span class=\"p\">:<\/span> <span class=\"s2\">&quot;document&quot;<\/span>\n      <span class=\"p\">},<\/span>\n      <span class=\"p\">{<\/span>\n        <span class=\"nt\">&quot;name&quot;<\/span><span class=\"p\">:<\/span> <span class=\"s2\">&quot;Referer&quot;<\/span><span class=\"p\">,<\/span>\n        <span class=\"nt\">&quot;value&quot;<\/span><span class=\"p\">:<\/span> <span class=\"s2\">&quot;https:\/\/menadevs.com\/directory\/users?name=b&quot;<\/span>\n      <span class=\"p\">},<\/span>\n      <span class=\"p\">{<\/span>\n        <span class=\"nt\">&quot;name&quot;<\/span><span class=\"p\">:<\/span> <span class=\"s2\">&quot;Accept-Encoding&quot;<\/span><span class=\"p\">,<\/span>\n        <span class=\"nt\">&quot;value&quot;<\/span><span class=\"p\">:<\/span> <span class=\"s2\">&quot;gzip, deflate, br&quot;<\/span>\n      <span class=\"p\">},<\/span>\n      <span class=\"p\">{<\/span>\n        <span class=\"nt\">&quot;name&quot;<\/span><span class=\"p\">:<\/span> <span class=\"s2\">&quot;Accept-Language&quot;<\/span><span class=\"p\">,<\/span>\n        <span class=\"nt\">&quot;value&quot;<\/span><span class=\"p\">:<\/span> <span class=\"s2\">&quot;en-US,en;q=0.9,ar;q=0.8,es;q=0.7&quot;<\/span>\n      <span class=\"p\">},<\/span>\n      <span class=\"p\">{<\/span>\n        <span class=\"nt\">&quot;name&quot;<\/span><span class=\"p\">:<\/span> <span class=\"s2\">&quot;Cookie&quot;<\/span><span class=\"p\">,<\/span>\n        <span class=\"nt\">&quot;value&quot;<\/span><span class=\"p\">:<\/span> <span class=\"s2\">&quot;_mena_devs_session=180300e84e30e891209258840d29d9bd; _ga=GA1.2.320033917.1590476331; _gid=GA1.2.1421047438.1590476331; _gat=1&quot;<\/span>\n      <span class=\"p\">}<\/span>\n    <span class=\"p\">],<\/span>\n    <span class=\"nt\">&quot;queryString&quot;<\/span><span class=\"p\">:<\/span> <span class=\"p\">[<\/span>\n      <span class=\"p\">{<\/span>\n        <span class=\"nt\">&quot;name&quot;<\/span><span class=\"p\">:<\/span> <span class=\"s2\">&quot;name&quot;<\/span><span class=\"p\">,<\/span>\n        <span class=\"nt\">&quot;value&quot;<\/span><span class=\"p\">:<\/span> <span class=\"s2\">&quot;f&quot;<\/span>\n      <span class=\"p\">}<\/span>\n    <span class=\"p\">],<\/span>\n    <span class=\"nt\">&quot;cookies&quot;<\/span><span class=\"p\">:<\/span> <span class=\"p\">[<\/span>\n      <span class=\"p\">{<\/span>\n        <span class=\"nt\">&quot;name&quot;<\/span><span class=\"p\">:<\/span> <span class=\"s2\">&quot;_mena_devs_session&quot;<\/span><span class=\"p\">,<\/span>\n        <span class=\"nt\">&quot;value&quot;<\/span><span class=\"p\">:<\/span> <span class=\"s2\">&quot;180300e84e30e891209258840d29d9bd&quot;<\/span><span class=\"p\">,<\/span>\n        <span class=\"nt\">&quot;expires&quot;<\/span><span class=\"p\">:<\/span> <span class=\"kc\">null<\/span><span class=\"p\">,<\/span>\n        <span class=\"nt\">&quot;httpOnly&quot;<\/span><span class=\"p\">:<\/span> <span class=\"kc\">false<\/span><span class=\"p\">,<\/span>\n        <span class=\"nt\">&quot;secure&quot;<\/span><span class=\"p\">:<\/span> <span class=\"kc\">false<\/span>\n      <span class=\"p\">},<\/span>\n      <span class=\"p\">{<\/span>\n        <span class=\"nt\">&quot;name&quot;<\/span><span class=\"p\">:<\/span> <span class=\"s2\">&quot;_ga&quot;<\/span><span class=\"p\">,<\/span>\n        <span class=\"nt\">&quot;value&quot;<\/span><span class=\"p\">:<\/span> <span class=\"s2\">&quot;GA1.2.320033917.1590476331&quot;<\/span><span class=\"p\">,<\/span>\n        <span class=\"nt\">&quot;expires&quot;<\/span><span class=\"p\">:<\/span> <span class=\"kc\">null<\/span><span class=\"p\">,<\/span>\n        <span class=\"nt\">&quot;httpOnly&quot;<\/span><span class=\"p\">:<\/span> <span class=\"kc\">false<\/span><span class=\"p\">,<\/span>\n        <span class=\"nt\">&quot;secure&quot;<\/span><span class=\"p\">:<\/span> <span class=\"kc\">false<\/span>\n      <span class=\"p\">},<\/span>\n      <span class=\"p\">{<\/span>\n        <span class=\"nt\">&quot;name&quot;<\/span><span class=\"p\">:<\/span> <span class=\"s2\">&quot;_gid&quot;<\/span><span class=\"p\">,<\/span>\n        <span class=\"nt\">&quot;value&quot;<\/span><span class=\"p\">:<\/span> <span class=\"s2\">&quot;GA1.2.1421047438.1590476331&quot;<\/span><span class=\"p\">,<\/span>\n        <span class=\"nt\">&quot;expires&quot;<\/span><span class=\"p\">:<\/span> <span class=\"kc\">null<\/span><span class=\"p\">,<\/span>\n        <span class=\"nt\">&quot;httpOnly&quot;<\/span><span class=\"p\">:<\/span> <span class=\"kc\">false<\/span><span class=\"p\">,<\/span>\n        <span class=\"nt\">&quot;secure&quot;<\/span><span class=\"p\">:<\/span> <span class=\"kc\">false<\/span>\n      <span class=\"p\">},<\/span>\n      <span class=\"p\">{<\/span>\n        <span class=\"nt\">&quot;name&quot;<\/span><span class=\"p\">:<\/span> <span class=\"s2\">&quot;_gat&quot;<\/span><span class=\"p\">,<\/span>\n        <span class=\"nt\">&quot;value&quot;<\/span><span class=\"p\">:<\/span> <span class=\"s2\">&quot;1&quot;<\/span><span class=\"p\">,<\/span>\n        <span class=\"nt\">&quot;expires&quot;<\/span><span class=\"p\">:<\/span> <span class=\"kc\">null<\/span><span class=\"p\">,<\/span>\n        <span class=\"nt\">&quot;httpOnly&quot;<\/span><span class=\"p\">:<\/span> <span class=\"kc\">false<\/span><span class=\"p\">,<\/span>\n        <span class=\"nt\">&quot;secure&quot;<\/span><span class=\"p\">:<\/span> <span class=\"kc\">false<\/span>\n      <span class=\"p\">}<\/span>\n    <span class=\"p\">],<\/span>\n    <span class=\"nt\">&quot;headersSize&quot;<\/span><span class=\"p\">:<\/span> <span class=\"mi\">766<\/span><span class=\"p\">,<\/span>\n    <span class=\"nt\">&quot;bodySize&quot;<\/span><span class=\"p\">:<\/span> <span class=\"mi\">0<\/span>\n  <span class=\"p\">},<\/span>\n  <span class=\"nt\">&quot;response&quot;<\/span><span class=\"p\">:<\/span> <span class=\"p\">{<\/span>\n    <span class=\"nt\">&quot;status&quot;<\/span><span class=\"p\">:<\/span> <span class=\"mi\">200<\/span><span class=\"p\">,<\/span>\n    <span class=\"nt\">&quot;statusText&quot;<\/span><span class=\"p\">:<\/span> <span class=\"s2\">&quot;OK&quot;<\/span><span class=\"p\">,<\/span>\n    <span class=\"nt\">&quot;httpVersion&quot;<\/span><span class=\"p\">:<\/span> <span class=\"s2\">&quot;HTTP\/1.1&quot;<\/span><span class=\"p\">,<\/span>\n    <span class=\"nt\">&quot;headers&quot;<\/span><span class=\"p\">:<\/span> <span class=\"p\">[<\/span>\n      <span class=\"p\">{<\/span>\n        <span class=\"nt\">&quot;name&quot;<\/span><span class=\"p\">:<\/span> <span class=\"s2\">&quot;Server&quot;<\/span><span class=\"p\">,<\/span>\n        <span class=\"nt\">&quot;value&quot;<\/span><span class=\"p\">:<\/span> <span class=\"s2\">&quot;nginx\/1.14.0 (Ubuntu)&quot;<\/span>\n      <span class=\"p\">},<\/span>\n      <span class=\"p\">{<\/span>\n        <span class=\"nt\">&quot;name&quot;<\/span><span class=\"p\">:<\/span> <span class=\"s2\">&quot;Date&quot;<\/span><span class=\"p\">,<\/span>\n        <span class=\"nt\">&quot;value&quot;<\/span><span class=\"p\">:<\/span> <span class=\"s2\">&quot;Tue, 26 May 2020 06:59:37 GMT&quot;<\/span>\n      <span class=\"p\">},<\/span>\n      <span class=\"p\">{<\/span>\n        <span class=\"nt\">&quot;name&quot;<\/span><span class=\"p\">:<\/span> <span class=\"s2\">&quot;Content-Type&quot;<\/span><span class=\"p\">,<\/span>\n        <span class=\"nt\">&quot;value&quot;<\/span><span class=\"p\">:<\/span> <span class=\"s2\">&quot;text\/html; charset=utf-8&quot;<\/span>\n      <span class=\"p\">},<\/span>\n      <span class=\"p\">{<\/span>\n        <span class=\"nt\">&quot;name&quot;<\/span><span class=\"p\">:<\/span> <span class=\"s2\">&quot;Transfer-Encoding&quot;<\/span><span class=\"p\">,<\/span>\n        <span class=\"nt\">&quot;value&quot;<\/span><span class=\"p\">:<\/span> <span class=\"s2\">&quot;chunked&quot;<\/span>\n      <span class=\"p\">},<\/span>\n      <span class=\"p\">{<\/span>\n        <span class=\"nt\">&quot;name&quot;<\/span><span class=\"p\">:<\/span> <span class=\"s2\">&quot;Connection&quot;<\/span><span class=\"p\">,<\/span>\n        <span class=\"nt\">&quot;value&quot;<\/span><span class=\"p\">:<\/span> <span class=\"s2\">&quot;keep-alive&quot;<\/span>\n      <span class=\"p\">},<\/span>\n      <span class=\"p\">{<\/span>\n        <span class=\"nt\">&quot;name&quot;<\/span><span class=\"p\">:<\/span> <span class=\"s2\">&quot;ETag&quot;<\/span><span class=\"p\">,<\/span>\n        <span class=\"nt\">&quot;value&quot;<\/span><span class=\"p\">:<\/span> <span class=\"s2\">&quot;W\/\\&quot;9dd057e36d9f985cec4fd1f68809d3c5\\&quot;&quot;<\/span>\n      <span class=\"p\">},<\/span>\n      <span class=\"p\">{<\/span>\n        <span class=\"nt\">&quot;name&quot;<\/span><span class=\"p\">:<\/span> <span class=\"s2\">&quot;Cache-Control&quot;<\/span><span class=\"p\">,<\/span>\n        <span class=\"nt\">&quot;value&quot;<\/span><span class=\"p\">:<\/span> <span class=\"s2\">&quot;max-age=0, private, must-revalidate&quot;<\/span>\n      <span class=\"p\">},<\/span>\n      <span class=\"p\">{<\/span>\n        <span class=\"nt\">&quot;name&quot;<\/span><span class=\"p\">:<\/span> <span class=\"s2\">&quot;X-Request-Id&quot;<\/span><span class=\"p\">,<\/span>\n        <span class=\"nt\">&quot;value&quot;<\/span><span class=\"p\">:<\/span> <span class=\"s2\">&quot;2ca2f8db-3de2-4bb7-a768-f7286368b67b&quot;<\/span>\n      <span class=\"p\">},<\/span>\n      <span class=\"p\">{<\/span>\n        <span class=\"nt\">&quot;name&quot;<\/span><span class=\"p\">:<\/span> <span class=\"s2\">&quot;X-Runtime&quot;<\/span><span class=\"p\">,<\/span>\n        <span class=\"nt\">&quot;value&quot;<\/span><span class=\"p\">:<\/span> <span class=\"s2\">&quot;0.031040&quot;<\/span>\n      <span class=\"p\">},<\/span>\n      <span class=\"p\">{<\/span>\n        <span class=\"nt\">&quot;name&quot;<\/span><span class=\"p\">:<\/span> <span class=\"s2\">&quot;Strict-Transport-Security&quot;<\/span><span class=\"p\">,<\/span>\n        <span class=\"nt\">&quot;value&quot;<\/span><span class=\"p\">:<\/span> <span class=\"s2\">&quot;max-age=631139040&quot;<\/span>\n      <span class=\"p\">},<\/span>\n      <span class=\"p\">{<\/span>\n        <span class=\"nt\">&quot;name&quot;<\/span><span class=\"p\">:<\/span> <span class=\"s2\">&quot;X-Content-Type-Options&quot;<\/span><span class=\"p\">,<\/span>\n        <span class=\"nt\">&quot;value&quot;<\/span><span class=\"p\">:<\/span> <span class=\"s2\">&quot;nosniff&quot;<\/span>\n      <span class=\"p\">},<\/span>\n      <span class=\"p\">{<\/span>\n        <span class=\"nt\">&quot;name&quot;<\/span><span class=\"p\">:<\/span> <span class=\"s2\">&quot;X-Frame-Options&quot;<\/span><span class=\"p\">,<\/span>\n        <span class=\"nt\">&quot;value&quot;<\/span><span class=\"p\">:<\/span> <span class=\"s2\">&quot;SAMEORIGIN&quot;<\/span>\n      <span class=\"p\">},<\/span>\n      <span class=\"p\">{<\/span>\n        <span class=\"nt\">&quot;name&quot;<\/span><span class=\"p\">:<\/span> <span class=\"s2\">&quot;X-Permitted-Cross-Domain-Policies&quot;<\/span><span class=\"p\">,<\/span>\n        <span class=\"nt\">&quot;value&quot;<\/span><span class=\"p\">:<\/span> <span class=\"s2\">&quot;none&quot;<\/span>\n      <span class=\"p\">},<\/span>\n      <span class=\"p\">{<\/span>\n        <span class=\"nt\">&quot;name&quot;<\/span><span class=\"p\">:<\/span> <span class=\"s2\">&quot;X-XSS-Protection&quot;<\/span><span class=\"p\">,<\/span>\n        <span class=\"nt\">&quot;value&quot;<\/span><span class=\"p\">:<\/span> <span class=\"s2\">&quot;1; mode=block&quot;<\/span>\n      <span class=\"p\">},<\/span>\n      <span class=\"p\">{<\/span>\n        <span class=\"nt\">&quot;name&quot;<\/span><span class=\"p\">:<\/span> <span class=\"s2\">&quot;X-Cache-Status&quot;<\/span><span class=\"p\">,<\/span>\n        <span class=\"nt\">&quot;value&quot;<\/span><span class=\"p\">:<\/span> <span class=\"s2\">&quot;MISS&quot;<\/span>\n      <span class=\"p\">},<\/span>\n      <span class=\"p\">{<\/span>\n        <span class=\"nt\">&quot;name&quot;<\/span><span class=\"p\">:<\/span> <span class=\"s2\">&quot;Content-Encoding&quot;<\/span><span class=\"p\">,<\/span>\n        <span class=\"nt\">&quot;value&quot;<\/span><span class=\"p\">:<\/span> <span class=\"s2\">&quot;gzip&quot;<\/span>\n      <span class=\"p\">}<\/span>\n    <span class=\"p\">],<\/span>\n    <span class=\"nt\">&quot;cookies&quot;<\/span><span class=\"p\">:<\/span> <span class=\"p\">[],<\/span>\n    <span class=\"nt\">&quot;content&quot;<\/span><span class=\"p\">:<\/span> <span class=\"p\">{<\/span>\n      <span class=\"nt\">&quot;size&quot;<\/span><span class=\"p\">:<\/span> <span class=\"mi\">14011<\/span><span class=\"p\">,<\/span>\n      <span class=\"nt\">&quot;mimeType&quot;<\/span><span class=\"p\">:<\/span> <span class=\"s2\">&quot;text\/html&quot;<\/span><span class=\"p\">,<\/span>\n      <span class=\"nt\">&quot;compression&quot;<\/span><span class=\"p\">:<\/span> <span class=\"mi\">9960<\/span>\n    <span class=\"p\">},<\/span>\n    <span class=\"nt\">&quot;redirectURL&quot;<\/span><span class=\"p\">:<\/span> <span class=\"s2\">&quot;&quot;<\/span><span class=\"p\">,<\/span>\n    <span class=\"nt\">&quot;headersSize&quot;<\/span><span class=\"p\">:<\/span> <span class=\"mi\">576<\/span><span class=\"p\">,<\/span>\n    <span class=\"nt\">&quot;bodySize&quot;<\/span><span class=\"p\">:<\/span> <span class=\"mi\">4051<\/span><span class=\"p\">,<\/span>\n    <span class=\"nt\">&quot;_transferSize&quot;<\/span><span class=\"p\">:<\/span> <span class=\"mi\">4627<\/span>\n  <span class=\"p\">},<\/span>\n  <span class=\"nt\">&quot;cache&quot;<\/span><span class=\"p\">:<\/span> <span class=\"p\">{},<\/span>\n  <span class=\"nt\">&quot;timings&quot;<\/span><span class=\"p\">:<\/span> <span class=\"p\">{<\/span>\n    <span class=\"nt\">&quot;blocked&quot;<\/span><span class=\"p\">:<\/span> <span class=\"mf\">2.917000008152798<\/span><span class=\"p\">,<\/span>\n    <span class=\"nt\">&quot;dns&quot;<\/span><span class=\"p\">:<\/span> <span class=\"mi\">-1<\/span><span class=\"p\">,<\/span>\n    <span class=\"nt\">&quot;ssl&quot;<\/span><span class=\"p\">:<\/span> <span class=\"mi\">-1<\/span><span class=\"p\">,<\/span>\n    <span class=\"nt\">&quot;connect&quot;<\/span><span class=\"p\">:<\/span> <span class=\"mi\">-1<\/span><span class=\"p\">,<\/span>\n    <span class=\"nt\">&quot;send&quot;<\/span><span class=\"p\">:<\/span> <span class=\"mf\">0.125<\/span><span class=\"p\">,<\/span>\n    <span class=\"nt\">&quot;wait&quot;<\/span><span class=\"p\">:<\/span> <span class=\"mf\">173.83599999024347<\/span><span class=\"p\">,<\/span>\n    <span class=\"nt\">&quot;receive&quot;<\/span><span class=\"p\">:<\/span> <span class=\"mf\">2.6510000170674175<\/span><span class=\"p\">,<\/span>\n    <span class=\"nt\">&quot;_blocked_queueing&quot;<\/span><span class=\"p\">:<\/span> <span class=\"mf\">1.6790000081527978<\/span>\n  <span class=\"p\">},<\/span>\n  <span class=\"nt\">&quot;serverIPAddress&quot;<\/span><span class=\"p\">:<\/span> <span class=\"s2\">&quot;188.166.50.85&quot;<\/span><span class=\"p\">,<\/span>\n  <span class=\"nt\">&quot;_initiator&quot;<\/span><span class=\"p\">:<\/span> <span class=\"p\">{<\/span>\n    <span class=\"nt\">&quot;type&quot;<\/span><span class=\"p\">:<\/span> <span class=\"s2\">&quot;other&quot;<\/span>\n  <span class=\"p\">},<\/span>\n  <span class=\"nt\">&quot;_priority&quot;<\/span><span class=\"p\">:<\/span> <span class=\"s2\">&quot;VeryHigh&quot;<\/span><span class=\"p\">,<\/span>\n  <span class=\"nt\">&quot;_resourceType&quot;<\/span><span class=\"p\">:<\/span> <span class=\"s2\">&quot;document&quot;<\/span><span class=\"p\">,<\/span>\n  <span class=\"nt\">&quot;connection&quot;<\/span><span class=\"p\">:<\/span> <span class=\"s2\">&quot;658445&quot;<\/span><span class=\"p\">,<\/span>\n  <span class=\"nt\">&quot;pageref&quot;<\/span><span class=\"p\">:<\/span> <span class=\"s2\">&quot;page_7&quot;<\/span>\n<span class=\"p\">}<\/span><span class=\"err\">,<\/span>\n<\/pre><\/div>\n\n\n<p>The model to parse, extract and flatten the data would look something like this:<\/p>\n<div class=\"highlight\"><pre><span><\/span>  <span class=\"kr\">const<\/span> <span class=\"nx\">entryPattern<\/span> <span class=\"o\">=<\/span> <span class=\"p\">{<\/span>\n    <span class=\"nx\">pageref<\/span><span class=\"o\">:<\/span> <span class=\"sr\">\/(?&lt;pageRef&gt;.*)\/<\/span><span class=\"p\">,<\/span>\n    <span class=\"nx\">startedDateTime<\/span><span class=\"o\">:<\/span> <span class=\"sr\">\/(?&lt;startedDateTime&gt;.*)\/<\/span><span class=\"p\">,<\/span>\n    <span class=\"nx\">request<\/span><span class=\"o\">:<\/span> <span class=\"p\">{<\/span>\n      <span class=\"nx\">method<\/span><span class=\"o\">:<\/span> <span class=\"sr\">\/(?&lt;requestMethod&gt;GET|POST)\/<\/span><span class=\"p\">,<\/span>\n      <span class=\"nx\">url<\/span><span class=\"o\">:<\/span> <span class=\"sr\">\/(?&lt;requestUrl&gt;[-a-zA-Z0-9@:%._\\+~#=]{1,256}\\.[a-zA-Z0-9()]{1,6}\\b([-a-zA-Z0-9()@:%_\\+.~#?&amp;\/\/=]*))\/<\/span><span class=\"p\">,<\/span>\n      <span class=\"nx\">httpVersion<\/span><span class=\"o\">:<\/span> <span class=\"sr\">\/(?&lt;requestHttpVersion&gt;.*)\/<\/span><span class=\"p\">,<\/span>\n      <span class=\"nx\">headersSize<\/span><span class=\"o\">:<\/span> <span class=\"sr\">\/^(?&lt;requestHeaderSize&gt;\\-?(\\d+\\.?\\d*|\\d*\\.?\\d+))$\/<\/span><span class=\"p\">,<\/span>\n      <span class=\"nx\">bodySize<\/span><span class=\"o\">:<\/span> <span class=\"sr\">\/^(?&lt;requestHeaderSize&gt;\\-?(\\d+\\.?\\d*|\\d*\\.?\\d+))$\/<\/span><span class=\"p\">,<\/span>\n    <span class=\"p\">},<\/span>\n    <span class=\"nx\">response<\/span><span class=\"o\">:<\/span> <span class=\"p\">{<\/span>\n      <span class=\"nx\">status<\/span><span class=\"o\">:<\/span> <span class=\"sr\">\/^(?&lt;responseStatus&gt;[0-9]{3})\/<\/span><span class=\"p\">,<\/span>\n      <span class=\"nx\">content<\/span><span class=\"o\">:<\/span> <span class=\"p\">{<\/span>\n        <span class=\"nx\">size<\/span><span class=\"o\">:<\/span> <span class=\"sr\">\/^(?&lt;responseContentSize&gt;\\-?(\\d+\\.?\\d*|\\d*\\.?\\d+))$\/<\/span><span class=\"p\">,<\/span>\n      <span class=\"p\">},<\/span>\n      <span class=\"nx\">headers<\/span><span class=\"o\">:<\/span> <span class=\"p\">[<\/span>\n        <span class=\"p\">{<\/span> <span class=\"nx\">name<\/span><span class=\"o\">:<\/span> <span class=\"sr\">\/^content-type$\/i<\/span><span class=\"p\">,<\/span> <span class=\"nx\">value<\/span><span class=\"o\">:<\/span> <span class=\"sr\">\/(?&lt;responseContentType&gt;.*)\/<\/span> <span class=\"p\">},<\/span>\n        <span class=\"p\">{<\/span> <span class=\"nx\">name<\/span><span class=\"o\">:<\/span> <span class=\"sr\">\/^content-length$\/i<\/span><span class=\"p\">,<\/span> <span class=\"nx\">value<\/span><span class=\"o\">:<\/span> <span class=\"sr\">\/(?&lt;responseContentLength&gt;.*)\/<\/span> <span class=\"p\">},<\/span>\n        <span class=\"p\">{<\/span> <span class=\"nx\">name<\/span><span class=\"o\">:<\/span> <span class=\"sr\">\/^cache-control$\/i<\/span><span class=\"p\">,<\/span> <span class=\"nx\">value<\/span><span class=\"o\">:<\/span> <span class=\"sr\">\/(?&lt;responseCacheControl&gt;.*)\/<\/span> <span class=\"p\">},<\/span>\n      <span class=\"p\">]<\/span>\n    <span class=\"p\">},<\/span>\n    <span class=\"nx\">timings<\/span><span class=\"o\">:<\/span> <span class=\"p\">(<\/span><span class=\"nx\">val<\/span><span class=\"p\">)<\/span> <span class=\"p\">=&gt;<\/span> <span class=\"nx\">val<\/span><span class=\"p\">,<\/span>\n    <span class=\"nx\">time<\/span><span class=\"o\">:<\/span> <span class=\"sr\">\/^(?&lt;time&gt;\\-?(\\d+\\.?\\d*|\\d*\\.?\\d+))$\/<\/span>\n  <span class=\"p\">};<\/span>\n<\/pre><\/div>\n\n\n<p>The above pattern is pretty much all the code we'd need to parse an entry! <\/p>\n<p>In the above defined model we can:<\/p>\n<ul>\n<li>Declaratively mimic the structure of the object we're comparing against. From there, Objectron will recursively access, validate and extract the matching keys and values.<\/li>\n<li>Define static values to match against keys and values<\/li>\n<li>Define regex patterns for keys and values, with optional named groups.<\/li>\n<\/ul>\n<p>Passing in the harLogEntry and pattern to the Objectron match function:<\/p>\n<div class=\"highlight\"><pre><span><\/span><span class=\"kr\">const<\/span> <span class=\"nx\">match<\/span> <span class=\"o\">=<\/span> <span class=\"nx\">require<\/span><span class=\"p\">(<\/span><span class=\"s1\">&#39;@menadevs\/objectron&#39;<\/span><span class=\"p\">);<\/span>\n\n<span class=\"kr\">const<\/span> <span class=\"nx\">result<\/span> <span class=\"o\">=<\/span> <span class=\"nx\">match<\/span><span class=\"p\">(<\/span>\n  <span class=\"nx\">harEntry<\/span><span class=\"p\">,<\/span> <span class=\"nx\">entryPattern<\/span>\n<span class=\"p\">);<\/span>\n<\/pre><\/div>\n\n\n<p>Would return a result object that looks like:<\/p>\n<div class=\"highlight\"><pre><span><\/span><span class=\"p\">{<\/span>\n  <span class=\"nx\">match<\/span><span class=\"o\">:<\/span> <span class=\"kc\">true<\/span><span class=\"p\">,<\/span>\n  <span class=\"nx\">total<\/span><span class=\"o\">:<\/span> <span class=\"mi\">15<\/span><span class=\"p\">,<\/span>\n  <span class=\"nx\">matches<\/span><span class=\"o\">:<\/span> <span class=\"p\">{<\/span>\n    <span class=\"nx\">pageref<\/span><span class=\"o\">:<\/span> <span class=\"s1\">&#39;page_7&#39;<\/span><span class=\"p\">,<\/span>\n    <span class=\"nx\">startedDateTime<\/span><span class=\"o\">:<\/span> <span class=\"s1\">&#39;2020-05-26T06:59:37.215Z&#39;<\/span><span class=\"p\">,<\/span>\n    <span class=\"nx\">request<\/span><span class=\"o\">:<\/span> <span class=\"p\">{<\/span>\n      <span class=\"nx\">method<\/span><span class=\"o\">:<\/span> <span class=\"s1\">&#39;GET&#39;<\/span><span class=\"p\">,<\/span>\n      <span class=\"nx\">url<\/span><span class=\"o\">:<\/span> <span class=\"s1\">&#39;https:\/\/menadevs.com\/directory\/users?name=f&#39;<\/span><span class=\"p\">,<\/span>\n      <span class=\"nx\">httpVersion<\/span><span class=\"o\">:<\/span> <span class=\"s1\">&#39;HTTP\/1.1&#39;<\/span><span class=\"p\">,<\/span>\n      <span class=\"nx\">headersSize<\/span><span class=\"o\">:<\/span> <span class=\"mi\">766<\/span><span class=\"p\">,<\/span>\n      <span class=\"nx\">bodySize<\/span><span class=\"o\">:<\/span> <span class=\"mi\">0<\/span>\n    <span class=\"p\">},<\/span>\n    <span class=\"nx\">response<\/span><span class=\"o\">:<\/span> <span class=\"p\">{<\/span>\n      <span class=\"nx\">status<\/span><span class=\"o\">:<\/span> <span class=\"mi\">200<\/span><span class=\"p\">,<\/span>\n      <span class=\"nx\">content<\/span><span class=\"o\">:<\/span> <span class=\"p\">{<\/span> <span class=\"nx\">size<\/span><span class=\"o\">:<\/span> <span class=\"mi\">14011<\/span> <span class=\"p\">},<\/span>\n      <span class=\"nx\">headers<\/span><span class=\"o\">:<\/span> <span class=\"p\">[<\/span>\n        <span class=\"p\">{<\/span> <span class=\"nx\">name<\/span><span class=\"o\">:<\/span> <span class=\"s1\">&#39;Content-Type&#39;<\/span><span class=\"p\">,<\/span> <span class=\"nx\">value<\/span><span class=\"o\">:<\/span> <span class=\"s1\">&#39;text\/html; charset=utf-8&#39;<\/span> <span class=\"p\">},<\/span>\n        <span class=\"p\">{<\/span>\n          <span class=\"nx\">name<\/span><span class=\"o\">:<\/span> <span class=\"s1\">&#39;Cache-Control&#39;<\/span><span class=\"p\">,<\/span>\n          <span class=\"nx\">value<\/span><span class=\"o\">:<\/span> <span class=\"s1\">&#39;max-age=0, private, must-revalidate&#39;<\/span>\n        <span class=\"p\">}<\/span>\n      <span class=\"p\">]<\/span>\n    <span class=\"p\">},<\/span>\n    <span class=\"nx\">timings<\/span><span class=\"o\">:<\/span> <span class=\"p\">{<\/span>\n      <span class=\"nx\">blocked<\/span><span class=\"o\">:<\/span> <span class=\"mf\">2.917000008152798<\/span><span class=\"p\">,<\/span>\n      <span class=\"nx\">dns<\/span><span class=\"o\">:<\/span> <span class=\"o\">-<\/span><span class=\"mi\">1<\/span><span class=\"p\">,<\/span>\n      <span class=\"nx\">ssl<\/span><span class=\"o\">:<\/span> <span class=\"o\">-<\/span><span class=\"mi\">1<\/span><span class=\"p\">,<\/span>\n      <span class=\"nx\">connect<\/span><span class=\"o\">:<\/span> <span class=\"o\">-<\/span><span class=\"mi\">1<\/span><span class=\"p\">,<\/span>\n      <span class=\"nx\">send<\/span><span class=\"o\">:<\/span> <span class=\"mf\">0.125<\/span><span class=\"p\">,<\/span>\n      <span class=\"nx\">wait<\/span><span class=\"o\">:<\/span> <span class=\"mf\">173.83599999024347<\/span><span class=\"p\">,<\/span>\n      <span class=\"nx\">receive<\/span><span class=\"o\">:<\/span> <span class=\"mf\">2.6510000170674175<\/span><span class=\"p\">,<\/span>\n      <span class=\"nx\">_blocked_queueing<\/span><span class=\"o\">:<\/span> <span class=\"mf\">1.6790000081527978<\/span>\n    <span class=\"p\">},<\/span>\n    <span class=\"nx\">time<\/span><span class=\"o\">:<\/span> <span class=\"mf\">179.52900001546368<\/span>\n  <span class=\"p\">},<\/span>\n  <span class=\"nx\">groups<\/span><span class=\"o\">:<\/span> <span class=\"p\">{<\/span>\n    <span class=\"nx\">pageRef<\/span><span class=\"o\">:<\/span> <span class=\"s1\">&#39;page_7&#39;<\/span><span class=\"p\">,<\/span>\n    <span class=\"nx\">startedDateTime<\/span><span class=\"o\">:<\/span> <span class=\"s1\">&#39;2020-05-26T06:59:37.215Z&#39;<\/span><span class=\"p\">,<\/span>\n    <span class=\"nx\">requestMethod<\/span><span class=\"o\">:<\/span> <span class=\"s1\">&#39;GET&#39;<\/span><span class=\"p\">,<\/span>\n    <span class=\"nx\">requestUrl<\/span><span class=\"o\">:<\/span> <span class=\"s1\">&#39;menadevs.com\/directory\/users?name=f&#39;<\/span><span class=\"p\">,<\/span>\n    <span class=\"nx\">requestHttpVersion<\/span><span class=\"o\">:<\/span> <span class=\"s1\">&#39;HTTP\/1.1&#39;<\/span><span class=\"p\">,<\/span>\n    <span class=\"nx\">requestHeaderSize<\/span><span class=\"o\">:<\/span> <span class=\"s1\">&#39;0&#39;<\/span><span class=\"p\">,<\/span>\n    <span class=\"nx\">responseStatus<\/span><span class=\"o\">:<\/span> <span class=\"s1\">&#39;200&#39;<\/span><span class=\"p\">,<\/span>\n    <span class=\"nx\">responseContentSize<\/span><span class=\"o\">:<\/span> <span class=\"s1\">&#39;14011&#39;<\/span><span class=\"p\">,<\/span>\n    <span class=\"nx\">responseContentType<\/span><span class=\"o\">:<\/span> <span class=\"s1\">&#39;text\/html; charset=utf-8&#39;<\/span><span class=\"p\">,<\/span>\n    <span class=\"nx\">responseCacheControl<\/span><span class=\"o\">:<\/span> <span class=\"s1\">&#39;max-age=0, private, must-revalidate&#39;<\/span><span class=\"p\">,<\/span>\n    <span class=\"nx\">time<\/span><span class=\"o\">:<\/span> <span class=\"s1\">&#39;179.52900001546368&#39;<\/span>\n  <span class=\"p\">}<\/span>\n<span class=\"p\">}<\/span>\n<\/pre><\/div>\n\n\n<p>All the named capturing groups have been added to a flat groups object, based on the values we've selected via the model. In the next step, the groups result object, will be extracted and transformed into a row entry in the final CSV file.<\/p>\n<p>We could also be inserting additional values into our row which have been captured, such as the timings object. The values of the timings object aren't showing in the groups result because we didn't really chose a named group for them, but instead assigned a wildcard via <code>timings: (val) =&gt; val<\/code> in the model to indicate that we want to extract everything from that sub-object.<\/p>\n<h2>Putting it all together<\/h2>\n<p>From that point on, the overall task is pretty simple. Loop over <code>log.entries<\/code> and test for matches. On Each match, combine the matched groups with the timings result and create a new flat row into a list to be exported to CSV or a similar tabular format later.<\/p>\n<div class=\"highlight\"><pre><span><\/span><span class=\"kr\">const<\/span> <span class=\"nx\">csvOutputPath<\/span> <span class=\"o\">=<\/span> <span class=\"s1\">&#39;.&#39;<\/span><span class=\"p\">;<\/span>\n<span class=\"kd\">let<\/span> <span class=\"nx\">flatEntries<\/span> <span class=\"o\">=<\/span> <span class=\"p\">[];<\/span>\n\n<span class=\"nx\">harFile<\/span><span class=\"p\">.<\/span><span class=\"nx\">log<\/span><span class=\"p\">.<\/span><span class=\"nx\">entries<\/span><span class=\"p\">.<\/span><span class=\"nx\">forEach<\/span><span class=\"p\">((<\/span><span class=\"nx\">entry<\/span><span class=\"p\">,<\/span> <span class=\"nx\">entryIndex<\/span><span class=\"p\">)<\/span> <span class=\"p\">=&gt;<\/span> <span class=\"p\">{<\/span>\n  <span class=\"kr\">const<\/span> <span class=\"nx\">currentEntry<\/span> <span class=\"o\">=<\/span> <span class=\"nx\">match<\/span><span class=\"p\">(<\/span>\n    <span class=\"nx\">entry<\/span><span class=\"p\">,<\/span> <span class=\"nx\">entryPattern<\/span>\n  <span class=\"p\">);<\/span>\n\n  <span class=\"k\">if<\/span><span class=\"p\">(<\/span><span class=\"nx\">currentEntry<\/span><span class=\"p\">.<\/span><span class=\"nx\">match<\/span><span class=\"p\">)<\/span> <span class=\"p\">{<\/span>\n    <span class=\"kr\">const<\/span> <span class=\"nx\">flatEntry<\/span> <span class=\"o\">=<\/span> <span class=\"p\">{<\/span>\n      <span class=\"p\">...<\/span><span class=\"nx\">currentEntry<\/span><span class=\"p\">.<\/span><span class=\"nx\">groups<\/span><span class=\"p\">,<\/span>\n      <span class=\"p\">...<\/span><span class=\"nx\">currentEntry<\/span><span class=\"p\">.<\/span><span class=\"nx\">matches<\/span><span class=\"p\">.<\/span><span class=\"nx\">timings<\/span>\n    <span class=\"p\">};<\/span>\n\n    <span class=\"k\">if<\/span> <span class=\"p\">(<\/span><span class=\"nx\">entryIndex<\/span> <span class=\"o\">===<\/span> <span class=\"mi\">0<\/span><span class=\"p\">)<\/span> <span class=\"p\">{<\/span>\n      <span class=\"nx\">flatEntries<\/span><span class=\"p\">.<\/span><span class=\"nx\">push<\/span><span class=\"p\">(<\/span><span class=\"nb\">Object<\/span><span class=\"p\">.<\/span><span class=\"nx\">keys<\/span><span class=\"p\">(<\/span><span class=\"nx\">flatEntry<\/span><span class=\"p\">));<\/span>\n    <span class=\"p\">}<\/span>\n\n    <span class=\"nx\">flatEntries<\/span><span class=\"p\">.<\/span><span class=\"nx\">push<\/span><span class=\"p\">(<\/span><span class=\"nb\">Object<\/span><span class=\"p\">.<\/span><span class=\"nx\">values<\/span><span class=\"p\">(<\/span><span class=\"nx\">flatEntry<\/span><span class=\"p\">));<\/span>\n  <span class=\"p\">}<\/span>\n <span class=\"p\">});<\/span>\n\n <span class=\"nx\">stringify<\/span><span class=\"p\">(<\/span><span class=\"nx\">flatEntries<\/span><span class=\"p\">,<\/span> <span class=\"kd\">function<\/span><span class=\"p\">(<\/span><span class=\"nx\">err<\/span><span class=\"p\">,<\/span> <span class=\"nx\">output<\/span><span class=\"p\">)<\/span> <span class=\"p\">{<\/span>\n   <span class=\"nx\">fs<\/span><span class=\"p\">.<\/span><span class=\"nx\">writeFile<\/span><span class=\"p\">(<\/span><span class=\"nx\">csvOutputath<\/span><span class=\"p\">,<\/span> <span class=\"nx\">output<\/span><span class=\"p\">,<\/span> <span class=\"kd\">function<\/span> <span class=\"p\">(<\/span><span class=\"nx\">err<\/span><span class=\"p\">)<\/span> <span class=\"p\">{<\/span>\n     <span class=\"k\">if<\/span> <span class=\"p\">(<\/span><span class=\"nx\">err<\/span><span class=\"p\">)<\/span> <span class=\"k\">return<\/span> <span class=\"nx\">console<\/span><span class=\"p\">.<\/span><span class=\"nx\">log<\/span><span class=\"p\">(<\/span><span class=\"nx\">err<\/span><span class=\"p\">);<\/span>\n   <span class=\"p\">});<\/span>\n <span class=\"p\">});<\/span>\n<\/pre><\/div>\n\n\n<p>A resulting CSV can look something like <a href=\"https:\/\/storage.googleapis.com\/thecodeship\/menadevs-har-logs.csv\">this<\/a>. <\/p>\n<p>I started building a simple CLI utility which pretty much follows this approach. You can check it out <a href=\"https:\/\/github.com\/google\/har2csv\">here<\/a>.<\/p>\n<h2>Running an analysis<\/h2>\n<p>From this point it becomes trivial to load this data into a Jupyter notebook or similar tools. <\/p>\n<p>One interesting exercise I'd do with HAR data can be around recording an HTTP session that goes through multiple pages of the conversion funnel of an online purchase. Analyzing this dataset can be very useful to understand the number of times the same resource are being requested over and over again across pages, whether their requests are properly cached, pre-fetched or pre-connected to. This could uncover multiple ideas for optimization opportunities on resources utilized across multiple pages.<\/p>\n<p>In an upcoming article, I'll be going through a full exercise around uncovering optimization opportunities with such data. In the meantime, I just kicked that off with a Jupyter notebook and a couple of basic queries on a HAR CSV file to get started. You can check it out <a href=\"https:\/\/github.com\/aymanfarhat\/har-analysis-notebook-example\/blob\/master\/HAR%20File%20Analysis.ipynb\">here<\/a>.<\/p>\n<h2>That's all folks!<\/h2>\n<p>This was a quick intro to utilizing the Objectron module for parsing out HAR files into something a bit more useful. I hope you found it useful, would love to hear your thoughts! Looking forward to seeing you next time as we build on what we tried out today into a full analysis!<\/p>","category":[{"@attributes":{"term":"har"}},{"@attributes":{"term":"browsers"}},{"@attributes":{"term":"logs"}},{"@attributes":{"term":"http"}},{"@attributes":{"term":"javascript"}},{"@attributes":{"term":"objectron"}},{"@attributes":{"term":"nodejs"}},{"@attributes":{"term":"performance"}}]},{"title":"A guide to Service Workers - pitfalls and best practices","link":{"@attributes":{"href":"https:\/\/www.thecodeship.com\/web-development\/guide-service-worker-pitfalls-best-practices\/","rel":"alternate"}},"published":"2020-05-10T11:00:00+03:00","updated":"2020-05-10T11:00:00+03:00","author":{"name":"Ayman Farhat"},"id":"tag:www.thecodeship.com,2020-05-10:\/web-development\/guide-service-worker-pitfalls-best-practices\/","summary":"<p>Service workers is one of the most useful features that came to modern browsers. Implementation can be quite tricky though and prone to a lot of issues that could slip into production. I'll be discussing my set of tips and best practices on the subject.<\/p>","content":"<p>Service worker is one of the most useful features that came to modern browsers. I've always enjoyed the features they can bring into web apps such as caching assets, and enabling offline user experiences. Implementation can be quite tricky though and prone to a lot of issues that could easily slip into production if not tested properly.<\/p>\n<p>In this post, I'll be discussing my set of \"tips from the trenches\" that I've documented along the way while building out and auditing several service worker implementations.<\/p>\n<h2>Make sure to set the right scope for your service worker<\/h2>\n<p>One of the early mistakes that one might make, is to include the service worker script into some sub-directory without specifying a scope. In a typical web app, adding JS related files into a specific sub-directory sounds pretty natural. For Service Worker files, this can be problematic by default since scope of the service worker is defined by its location relative to the web root of the web app.<\/p>\n<p>This means, if you put your script in a <code>\/assets\/js<\/code> directory its scope is limited to the <code>\/assets\/js<\/code>. In that case, the service worker can only handle or intercept requests through <code>examplesite.com\/assets\/js\/<\/code>, and completely ignore other requests such as those going through <code>examplesite.com<\/code> or  <code>examplesite.com\/news\/<\/code> for example. <\/p>\n<p>If you want the service worker to handle any request coming through any part of the site, you have have two options:<\/p>\n<ul>\n<li>Place the service worker file on the web root directory, which would set its scope to cover the whole site by default.<\/li>\n<li>Explicitly define the service worker's scope when registering it:<\/li>\n<\/ul>\n<div class=\"highlight\"><pre><span><\/span>  <span class=\"nx\">navigator<\/span><span class=\"p\">.<\/span><span class=\"nx\">serviceWorker<\/span><span class=\"p\">.<\/span><span class=\"nx\">register<\/span><span class=\"p\">(<\/span><span class=\"s1\">&#39;\/assets\/js\/sw.js&#39;<\/span><span class=\"p\">,<\/span> <span class=\"p\">{<\/span>\n    <span class=\"nx\">scope<\/span><span class=\"o\">:<\/span> <span class=\"s1\">&#39;\/&#39;<\/span>\n  <span class=\"p\">});<\/span>\n<\/pre><\/div>\n\n\n<h2>Delay registering a service worker until after a web app's load event fire<\/h2>\n<p>When a page is loading, all resources such as bandwidth and CPU time, should be prioritized to serving the minimum set of critical resources needed to display an interactive page to the user. \nThis is even more critical in a scenario such as a first time mobile browser accessing a page, not  running over a flaky 3G network. You wouldn't want the service worker to be competing on resources while the page is being rendered.<\/p>\n<p>It's recommended to always delay registering a service worker until the web app has loaded properly. A general rule of thumb would be register it on the window <code>load<\/code> event, example:<\/p>\n<div class=\"highlight\"><pre><span><\/span>  <span class=\"k\">if<\/span><span class=\"p\">(<\/span><span class=\"s1\">&#39;serviceWorker&#39;<\/span> <span class=\"k\">in<\/span> <span class=\"nx\">navigator<\/span><span class=\"p\">)<\/span> <span class=\"p\">{<\/span>\n    <span class=\"nb\">window<\/span><span class=\"p\">.<\/span><span class=\"nx\">addEventListener<\/span><span class=\"p\">(<\/span><span class=\"s1\">&#39;load&#39;<\/span><span class=\"p\">,<\/span> <span class=\"p\">()<\/span> <span class=\"p\">=&gt;<\/span> <span class=\"p\">{<\/span>\n      <span class=\"nx\">navigator<\/span><span class=\"p\">.<\/span><span class=\"nx\">serviceWorker<\/span><span class=\"p\">.<\/span><span class=\"nx\">register<\/span><span class=\"p\">(<\/span><span class=\"s1\">&#39;sw.js&#39;<\/span><span class=\"p\">);<\/span>\n    <span class=\"p\">});<\/span>\n  <span class=\"p\">}<\/span>\n<\/pre><\/div>\n\n\n<p>The above applies to many cases, but the right time to start the registration would highly depend on the web app and how it works. If you're using a framework that needs additional setup after the page has loaded, you might want to check for framework specific events that signal that the work is done.<\/p>\n<p>For more information on optimizing resource usage on page load, I would suggest checking out <a href=\"https:\/\/developers.google.com\/web\/fundamentals\/primers\/service-workers\/registration#a_users_first_visit\">a user's first journey<\/a>.<\/p>\n<h2>Don't worry about a service worker caching itself<\/h2>\n<p>If a service worker is set to intercept and cache all requests, one might worry that an older service worker could end up serving a cached version of itself rather serving and registering a new one.<\/p>\n<p>It turns out, a service worker's fetch event handler is not triggered when the page requests a service worker script to register or update. For example, when you call <code>navigator.serviceWorker.register('sw.js)<\/code> the request for <code>sw.js<\/code> isn't intercepted by any service worker's fetch event handler.<\/p>\n<p>For reference, you might want to look at the specification of <a href=\"https:\/\/w3c.github.io\/ServiceWorker\/#update-algorithm\">service worker's update algorithm<\/a>.<\/p>\n<h2>A service worker's cache is accessible from the main thread<\/h2>\n<p>In many cases, you might want to manage some aspects related to the app shell or content cache from within the main thread based on certain user interactions.<\/p>\n<p>The good news is that the service worker's cache is accessible from the main thread via <code>window.caches<\/code> just like you'd use <code>self.caches<\/code> within the service worker's code. <\/p>\n<p>This <a href=\"https:\/\/googlechrome.github.io\/samples\/service-worker\/window-caches\/\">snippet<\/a>, shows an example of that.<\/p>\n<h2>Use a library for messaging<\/h2>\n<p>As you might have noticed, the service worker can't access the main thread and its related resources, such as DOM and Window object. <\/p>\n<p>To communicate with the main thread, the service worker uses <a href=\"https:\/\/developer.mozilla.org\/en-US\/docs\/Web\/API\/Window\/postMessage\">postMessage<\/a> to send data and receive data.<\/p>\n<p>You might want to reduce the overall complexity of postMessage by using a wrapper library for that such as <a href=\"https:\/\/github.com\/bevacqua\/swivel\">Swivel<\/a><\/p>\n<h2>Clone the fetch response stream when it needs to be consumed more than once<\/h2>\n<p>As an optimization for efficient memory usage, <code>fetch<\/code> gets the request\/response body as a stream that doesn't get buffered into memory and can be consumed only once. This could get complicated when you're trying to read the response stream multiple times, such as pushing the result into the cache and returning the response back to the client. <\/p>\n<p>The solution in such case would be to clone the response stream. Below is an example of a cache first strategy using that approach. Notice how on line 6 the response object is cloned before adding it into the cache while the original response stream gets returned to the client.<\/p>\n<div class=\"highlight\"><pre><span><\/span><span class=\"nx\">self<\/span><span class=\"p\">.<\/span><span class=\"nx\">addEventListener<\/span><span class=\"p\">(<\/span><span class=\"s1\">&#39;fetch&#39;<\/span><span class=\"p\">,<\/span> <span class=\"p\">(<\/span><span class=\"nx\">event<\/span><span class=\"p\">)<\/span> <span class=\"p\">=&gt;<\/span> <span class=\"p\">{<\/span>\n  <span class=\"nx\">event<\/span><span class=\"p\">.<\/span><span class=\"nx\">respondWith<\/span><span class=\"p\">(<\/span>\n    <span class=\"nx\">caches<\/span><span class=\"p\">.<\/span><span class=\"nx\">open<\/span><span class=\"p\">(<\/span><span class=\"nx\">CACHE<\/span><span class=\"p\">).<\/span><span class=\"nx\">then<\/span><span class=\"p\">((<\/span><span class=\"nx\">cache<\/span><span class=\"p\">)<\/span> <span class=\"p\">=&gt;<\/span> <span class=\"p\">{<\/span>\n      <span class=\"k\">return<\/span> <span class=\"nx\">cache<\/span><span class=\"p\">.<\/span><span class=\"nx\">match<\/span><span class=\"p\">(<\/span><span class=\"nx\">event<\/span><span class=\"p\">.<\/span><span class=\"nx\">request<\/span><span class=\"p\">).<\/span><span class=\"nx\">then<\/span><span class=\"p\">((<\/span><span class=\"nx\">response<\/span><span class=\"p\">)<\/span> <span class=\"p\">=&gt;<\/span> <span class=\"p\">{<\/span>\n        <span class=\"k\">return<\/span> <span class=\"nx\">response<\/span> <span class=\"o\">||<\/span> <span class=\"nx\">fetch<\/span><span class=\"p\">(<\/span><span class=\"nx\">event<\/span><span class=\"p\">.<\/span><span class=\"nx\">request<\/span><span class=\"p\">).<\/span><span class=\"nx\">then<\/span><span class=\"p\">((<\/span><span class=\"nx\">response<\/span><span class=\"p\">)<\/span> <span class=\"p\">=&gt;<\/span> <span class=\"p\">{<\/span>\n          <span class=\"nx\">cache<\/span><span class=\"p\">.<\/span><span class=\"nx\">put<\/span><span class=\"p\">(<\/span><span class=\"nx\">event<\/span><span class=\"p\">.<\/span><span class=\"nx\">request<\/span><span class=\"p\">,<\/span> <span class=\"nx\">response<\/span><span class=\"p\">.<\/span><span class=\"nx\">clone<\/span><span class=\"p\">());<\/span>\n          <span class=\"k\">return<\/span> <span class=\"nx\">response<\/span><span class=\"p\">;<\/span>\n        <span class=\"p\">});<\/span>\n      <span class=\"p\">});<\/span>\n    <span class=\"p\">})<\/span>\n  <span class=\"p\">);<\/span>\n<span class=\"p\">});<\/span>\n<\/pre><\/div>\n\n\n<p>I highly recommend checking out the article on <a href=\"https:\/\/jakearchibald.com\/2014\/reading-responses\/\">what happens when you read a response?<\/a> explaining this concept in detail.<\/p>\n<h2>Avoid adding a \"pass-through\" fetch handler<\/h2>\n<p>If you're registering a service worker yet have no need to intercept and cache any request, you might feel you should define \"pass-through\" fetch event listener such as:<\/p>\n<div class=\"highlight\"><pre><span><\/span><span class=\"nx\">self<\/span><span class=\"p\">.<\/span><span class=\"nx\">addEventListener<\/span><span class=\"p\">(<\/span><span class=\"s1\">&#39;fetch&#39;<\/span><span class=\"p\">,<\/span> <span class=\"nx\">event<\/span> <span class=\"p\">=&gt;<\/span> <span class=\"p\">{<\/span>\n  <span class=\"nx\">event<\/span><span class=\"p\">.<\/span><span class=\"nx\">respondWith<\/span><span class=\"p\">(<\/span><span class=\"nx\">fetch<\/span><span class=\"p\">(<\/span><span class=\"nx\">event<\/span><span class=\"p\">.<\/span><span class=\"nx\">request<\/span><span class=\"p\">));<\/span>\n<span class=\"p\">});<\/span>\n<\/pre><\/div>\n\n\n<p>While this pass-through won't affect the behavior of the web app, it can introduce a performance overhead due to several reasons.<\/p>\n<ol>\n<li>\n<p>It adds a latency hit for every single network request made, mainly due to calling respondWith and fetch unnecessarily. <\/p>\n<\/li>\n<li>\n<p>If the service worker is not already running, there is usually an overhead in starting it up. Many browsers optimize on that by not starting up the service worker on network requests unless a fetch handler is defined. Registering a fetch handler that doesn't do anything leads to consuming resources on the user's device unnecessarily.<\/p>\n<\/li>\n<\/ol>\n<h2>Avoid caching bad responses by properly handling them in fetch<\/h2>\n<p>A fetch promise always returns a response object for the given request, which contains the status code of the response. This is true for all responses including those containing error status codes (such as 4xx and 5xx).<\/p>\n<p>It's important to keep that in mind when implementing caching strategies so that you don't end up caching bad responses. <\/p>\n<p>While the below cache-first strategy example would work, it's missing a very important detail around handling the status of fresh responses from the server. It will be caching pretty much any reply with any status!<\/p>\n<div class=\"highlight\"><pre><span><\/span><span class=\"nx\">self<\/span><span class=\"p\">.<\/span><span class=\"nx\">addEventListener<\/span><span class=\"p\">(<\/span><span class=\"s1\">&#39;fetch&#39;<\/span><span class=\"p\">,<\/span> <span class=\"p\">(<\/span><span class=\"nx\">event<\/span><span class=\"p\">)<\/span> <span class=\"p\">=&gt;<\/span> <span class=\"p\">{<\/span>\n  <span class=\"nx\">event<\/span><span class=\"p\">.<\/span><span class=\"nx\">respondWith<\/span><span class=\"p\">(<\/span>\n    <span class=\"nx\">caches<\/span><span class=\"p\">.<\/span><span class=\"nx\">open<\/span><span class=\"p\">(<\/span><span class=\"nx\">CACHE<\/span><span class=\"p\">).<\/span><span class=\"nx\">then<\/span><span class=\"p\">((<\/span><span class=\"nx\">cache<\/span><span class=\"p\">)<\/span> <span class=\"p\">=&gt;<\/span> <span class=\"p\">{<\/span>\n      <span class=\"k\">return<\/span> <span class=\"nx\">cache<\/span><span class=\"p\">.<\/span><span class=\"nx\">match<\/span><span class=\"p\">(<\/span><span class=\"nx\">event<\/span><span class=\"p\">.<\/span><span class=\"nx\">request<\/span><span class=\"p\">).<\/span><span class=\"nx\">then<\/span><span class=\"p\">((<\/span><span class=\"nx\">response<\/span><span class=\"p\">)<\/span> <span class=\"p\">=&gt;<\/span> <span class=\"p\">{<\/span>\n        <span class=\"k\">return<\/span> <span class=\"nx\">response<\/span> <span class=\"o\">||<\/span> <span class=\"nx\">fetch<\/span><span class=\"p\">(<\/span><span class=\"nx\">event<\/span><span class=\"p\">.<\/span><span class=\"nx\">request<\/span><span class=\"p\">).<\/span><span class=\"nx\">then<\/span><span class=\"p\">((<\/span><span class=\"nx\">response<\/span><span class=\"p\">)<\/span> <span class=\"p\">=&gt;<\/span> <span class=\"p\">{<\/span>\n          <span class=\"nx\">cache<\/span><span class=\"p\">.<\/span><span class=\"nx\">put<\/span><span class=\"p\">(<\/span><span class=\"nx\">event<\/span><span class=\"p\">.<\/span><span class=\"nx\">request<\/span><span class=\"p\">,<\/span> <span class=\"nx\">response<\/span><span class=\"p\">.<\/span><span class=\"nx\">clone<\/span><span class=\"p\">());<\/span>\n          <span class=\"k\">return<\/span> <span class=\"nx\">response<\/span><span class=\"p\">;<\/span>\n        <span class=\"p\">})<\/span>\n      <span class=\"p\">});<\/span>\n    <span class=\"p\">})<\/span>\n  <span class=\"p\">);<\/span>\n<span class=\"p\">});<\/span>\n<\/pre><\/div>\n\n\n<p>Always keep in mind that HTTP responses with errors (such as 4xx and 5xx) won't be throwing any exception. They will need to be handled in the <code>.then<\/code> clause by checking the status of the reply. An improvement that can be done over the example above would be to check the response status and only cache successful responses.<\/p>\n<div class=\"highlight\"><pre><span><\/span><span class=\"nx\">self<\/span><span class=\"p\">.<\/span><span class=\"nx\">addEventListener<\/span><span class=\"p\">(<\/span><span class=\"s1\">&#39;fetch&#39;<\/span><span class=\"p\">,<\/span> <span class=\"p\">(<\/span><span class=\"nx\">event<\/span><span class=\"p\">)<\/span> <span class=\"p\">=&gt;<\/span> <span class=\"p\">{<\/span>\n  <span class=\"nx\">event<\/span><span class=\"p\">.<\/span><span class=\"nx\">respondWith<\/span><span class=\"p\">(<\/span>\n    <span class=\"nx\">caches<\/span><span class=\"p\">.<\/span><span class=\"nx\">open<\/span><span class=\"p\">(<\/span><span class=\"nx\">CACHE<\/span><span class=\"p\">).<\/span><span class=\"nx\">then<\/span><span class=\"p\">((<\/span><span class=\"nx\">cache<\/span><span class=\"p\">)<\/span> <span class=\"p\">=&gt;<\/span> <span class=\"p\">{<\/span>\n      <span class=\"k\">return<\/span> <span class=\"nx\">cache<\/span><span class=\"p\">.<\/span><span class=\"nx\">match<\/span><span class=\"p\">(<\/span><span class=\"nx\">event<\/span><span class=\"p\">.<\/span><span class=\"nx\">request<\/span><span class=\"p\">).<\/span><span class=\"nx\">then<\/span><span class=\"p\">((<\/span><span class=\"nx\">response<\/span><span class=\"p\">)<\/span> <span class=\"p\">=&gt;<\/span> <span class=\"p\">{<\/span>\n        <span class=\"k\">return<\/span> <span class=\"nx\">response<\/span> <span class=\"o\">||<\/span> <span class=\"nx\">fetch<\/span><span class=\"p\">(<\/span><span class=\"nx\">event<\/span><span class=\"p\">.<\/span><span class=\"nx\">request<\/span><span class=\"p\">).<\/span><span class=\"nx\">then<\/span><span class=\"p\">((<\/span><span class=\"nx\">response<\/span><span class=\"p\">)<\/span> <span class=\"p\">=&gt;<\/span> <span class=\"p\">{<\/span>\n          <span class=\"k\">if<\/span> <span class=\"p\">(<\/span><span class=\"nx\">response<\/span><span class=\"p\">.<\/span><span class=\"nx\">status<\/span> <span class=\"o\">===<\/span> <span class=\"mi\">200<\/span><span class=\"p\">)<\/span> <span class=\"p\">{<\/span>\n            <span class=\"nx\">cache<\/span><span class=\"p\">.<\/span><span class=\"nx\">put<\/span><span class=\"p\">(<\/span><span class=\"nx\">event<\/span><span class=\"p\">.<\/span><span class=\"nx\">request<\/span><span class=\"p\">,<\/span> <span class=\"nx\">response<\/span><span class=\"p\">.<\/span><span class=\"nx\">clone<\/span><span class=\"p\">());<\/span>\n          <span class=\"p\">}<\/span>\n\n          <span class=\"k\">return<\/span> <span class=\"nx\">response<\/span><span class=\"p\">;<\/span>\n        <span class=\"p\">})<\/span>\n      <span class=\"p\">});<\/span>\n    <span class=\"p\">})<\/span>\n  <span class=\"p\">);<\/span>\n<span class=\"p\">});<\/span>\n<\/pre><\/div>\n\n\n<p>The only case where an exception is thrown by fetch is when the request completely fails due to network failure, such as being offline. You could take such an opportunity to handle the exception and serve a custom reply on the resource in that case. Example below:<\/p>\n<div class=\"highlight\"><pre><span><\/span><span class=\"kr\">const<\/span> <span class=\"nx\">offlineHTML<\/span> <span class=\"o\">=<\/span> <span class=\"sb\">`&lt;h1&gt;You&#39;re offline!&lt;\/h1&gt;`<\/span><span class=\"p\">;<\/span>\n\n<span class=\"nx\">self<\/span><span class=\"p\">.<\/span><span class=\"nx\">addEventListener<\/span><span class=\"p\">(<\/span><span class=\"s1\">&#39;fetch&#39;<\/span><span class=\"p\">,<\/span> <span class=\"p\">(<\/span><span class=\"nx\">event<\/span><span class=\"p\">)<\/span> <span class=\"p\">=&gt;<\/span> <span class=\"p\">{<\/span>\n  <span class=\"nx\">event<\/span><span class=\"p\">.<\/span><span class=\"nx\">respondWith<\/span><span class=\"p\">(<\/span>\n    <span class=\"nx\">caches<\/span><span class=\"p\">.<\/span><span class=\"nx\">open<\/span><span class=\"p\">(<\/span><span class=\"nx\">CACHE<\/span><span class=\"p\">).<\/span><span class=\"nx\">then<\/span><span class=\"p\">((<\/span><span class=\"nx\">cache<\/span><span class=\"p\">)<\/span> <span class=\"p\">=&gt;<\/span> <span class=\"p\">{<\/span>\n      <span class=\"k\">return<\/span> <span class=\"nx\">cache<\/span><span class=\"p\">.<\/span><span class=\"nx\">match<\/span><span class=\"p\">(<\/span><span class=\"nx\">event<\/span><span class=\"p\">.<\/span><span class=\"nx\">request<\/span><span class=\"p\">).<\/span><span class=\"nx\">then<\/span><span class=\"p\">((<\/span><span class=\"nx\">response<\/span><span class=\"p\">)<\/span> <span class=\"p\">=&gt;<\/span> <span class=\"p\">{<\/span>\n        <span class=\"k\">return<\/span> <span class=\"nx\">response<\/span> <span class=\"o\">||<\/span> <span class=\"nx\">fetch<\/span><span class=\"p\">(<\/span><span class=\"nx\">event<\/span><span class=\"p\">.<\/span><span class=\"nx\">request<\/span><span class=\"p\">).<\/span><span class=\"nx\">then<\/span><span class=\"p\">((<\/span><span class=\"nx\">response<\/span><span class=\"p\">)<\/span> <span class=\"p\">=&gt;<\/span> <span class=\"p\">{<\/span>\n          <span class=\"k\">if<\/span> <span class=\"p\">(<\/span><span class=\"nx\">response<\/span><span class=\"p\">.<\/span><span class=\"nx\">status<\/span> <span class=\"o\">===<\/span> <span class=\"mi\">200<\/span><span class=\"p\">)<\/span> <span class=\"p\">{<\/span>\n            <span class=\"nx\">cache<\/span><span class=\"p\">.<\/span><span class=\"nx\">put<\/span><span class=\"p\">(<\/span><span class=\"nx\">event<\/span><span class=\"p\">.<\/span><span class=\"nx\">request<\/span><span class=\"p\">,<\/span> <span class=\"nx\">response<\/span><span class=\"p\">.<\/span><span class=\"nx\">clone<\/span><span class=\"p\">());<\/span>\n          <span class=\"p\">}<\/span>\n\n          <span class=\"k\">return<\/span> <span class=\"nx\">response<\/span><span class=\"p\">;<\/span>\n        <span class=\"p\">}).<\/span><span class=\"k\">catch<\/span><span class=\"p\">(()<\/span> <span class=\"p\">=&gt;<\/span> <span class=\"p\">{<\/span>\n          <span class=\"k\">return<\/span> <span class=\"k\">new<\/span> <span class=\"nx\">Response<\/span><span class=\"p\">(<\/span><span class=\"nx\">offlineHTML<\/span><span class=\"p\">,<\/span> <span class=\"p\">{<\/span>\n            <span class=\"nx\">headers<\/span><span class=\"o\">:<\/span> <span class=\"p\">{<\/span>\n              <span class=\"s1\">&#39;Content-Type&#39;<\/span><span class=\"o\">:<\/span> <span class=\"s1\">&#39;text\/html;charset=utf-8&#39;<\/span>\n            <span class=\"p\">}<\/span>\n          <span class=\"p\">});<\/span>\n        <span class=\"p\">});<\/span>\n      <span class=\"p\">});<\/span>\n    <span class=\"p\">})<\/span>\n  <span class=\"p\">);<\/span>\n<span class=\"p\">});<\/span>\n<\/pre><\/div>\n\n\n<p><strong>Note:<\/strong> 2 crucial details are intentionally missing from the above examples to keep things simple for illustration purposes:<\/p>\n<ol>\n<li>Route and file type checking on the requested resource, which I'm assuming are HTML documents<\/li>\n<li>Checking and setting cache expiry<\/li>\n<\/ol>\n<p>Hence, <strong>I wouldn't advise copy\/pasting the above code into production as is.<\/strong><\/p>\n<h2>Setting web server cache-control policy the for service worker file<\/h2>\n<p>This issue is becoming less of a concern over time as recent major browser versions are shipping changes, implementing an <a href=\"https:\/\/github.com\/w3c\/ServiceWorker\/issues\/893\">update on the default service worker file caching behavior<\/a>, which now gets the browser to ignore the HTTP cache directive when checking for updates on the service worker script.<\/p>\n<p>For supporting older browser version, you might still want to make sure that the request to <code>sw.js<\/code> isn't cached by the browser. This can be simply done by setting <code>Cache-control: no-cache<\/code> and <code>max-age: 0<\/code> on the service worker file in your web server. In nginx, the configuration can looks something like this:<\/p>\n<div class=\"highlight\"><pre><span><\/span><span class=\"k\">location<\/span> <span class=\"o\">~*<\/span> <span class=\"p\">(<\/span><span class=\"n\">sw<\/span><span class=\"err\">\\<\/span><span class=\"p\">.<\/span><span class=\"n\">js<\/span><span class=\"p\">)<\/span><span class=\"err\">$<\/span> <span class=\"err\">{<\/span>\n    <span class=\"n\">add_header<\/span> <span class=\"s1\">&#39;Cache-Control&#39;<\/span> <span class=\"s1\">&#39;no-store, no-cache, must-revalidate, proxy-revalidate, max-age=0&#39;<\/span><span class=\"p\">;<\/span>\n    <span class=\"n\">expires<\/span> <span class=\"k\">off<\/span><span class=\"p\">;<\/span>\n    <span class=\"n\">proxy_no_cache<\/span> <span class=\"mi\">1<\/span><span class=\"p\">;<\/span>\n<span class=\"err\">}<\/span>\n<\/pre><\/div>\n\n\n<p>You can later test out the service worker's cache-control setting via: <code>curl -X GET -I http:\/\/yourwebsitehere.xyz\/sw.js<\/code><\/p>\n<h1>Evaluate caching strategy examples and never copy paste them directly<\/h1>\n<p>Looking at examples online is great, but always make sure to learn something from them over copy pasting the example into your code (This is recommended for the code shared here too). <\/p>\n<p>While this rule applies to any piece of code you're writing, it can be particularly dangerous with caching strategies, even if some might seem production ready.<\/p>\n<p>This is because every implementation will highly depend on the nature of the web app, how it behaves and the business logic involved. Here is a small checklist I keep in mind when reviewing a service worker code example:<\/p>\n<ul>\n<li>Check for routing and file type patterns, do they make sense?<\/li>\n<li>How does it implement cache expiry logic? Does that apply to my requirements?<\/li>\n<li>Cache TTL, is it uniform for all caches or specific to certain file types\/routes ?<\/li>\n<li>Which <a href=\"https:\/\/developers.google.com\/web\/fundamentals\/instant-and-offline\/offline-cookbook\">caching strategies<\/a> does it apply? For which resource types \/ routes ?<\/li>\n<\/ul>\n<p>I would go over the above criteria and evaluate how each one of those would apply\/make sense within the specific use case and modify accordingly.<\/p>\n<h1>You might not need to build a service worker from scratch<\/h1>\n<p>Finally, when going for production, you might not really need to build out the service worker from scratch. Libraries like <a href=\"https:\/\/developers.google.com\/web\/tools\/workbox\">Workbox<\/a> can provide several helper functions and wrappers out of the box for complex implementations which could save you several hours of troubleshooting, testing and headache.<\/p>\n<p>If you're just starting out, I would still recommend playing around with building a couple of vanilla implementations in order to get a grip at understanding how service workers work before jumping into using a library though.<\/p>\n<h1>Did I miss anything?<\/h1>\n<p>I would love to hear more about your tips on service worker implementation that I might have missed in the comments below. Would also appreciate any other feedback you might have!<\/p>","category":[{"@attributes":{"term":"service-workers"}},{"@attributes":{"term":"javascript"}},{"@attributes":{"term":"best"}},{"@attributes":{"term":"practices"}},{"@attributes":{"term":"advice"}},{"@attributes":{"term":"pitfalls"}},{"@attributes":{"term":"tips"}}]},{"title":"Tip on Converting Cocoa Core Data to Unix Timestamp in Python","link":{"@attributes":{"href":"https:\/\/www.thecodeship.com\/general\/converting-cocoa-unix-timestamp\/","rel":"alternate"}},"published":"2019-09-18T15:00:00+03:00","updated":"2019-09-18T15:00:00+03:00","author":{"name":"Ayman Farhat"},"id":"tag:www.thecodeship.com,2019-09-18:\/general\/converting-cocoa-unix-timestamp\/","summary":"<p>When accessing a Core Data database, you might notice that the timestamp is not exactly a unix epoch timestamp but an Apple specific one instead. In this post, I explain the difference and how to convert that into the popular Unix timestamp that we all love.<\/p>","content":"<p>I was writing a script that reads out from an sqlite database, originally created by an iOS app. Being clueless about iOS development, I noticed that one of the columns had an unusual time stamp. Turns out that this is a <a href=\"https:\/\/developer.apple.com\/documentation\/coredata\">Core Data<\/a> time stamp, which is also sometimes labeled as 'Mac absolute time'.<\/p>\n<p>Core Data is a storage framework to manage objects in iOS and OSX applications as part of the Cocoa API. In this framework, a Core Data timestamp is the number of seconds (or nanoseconds) since midnight, January 1, 2001, GMT. On the other hand, the Unix timestamp is the number of seconds since January 1, 1970 GMT.<\/p>\n<p>A Cocoa timestamp of 590517794 seconds or 590517794000000000 nano seconds represents the date time of <strong>Wednesday, September 18, 2019 4:43:14 PM<\/strong>. If mistakenly assumed as a Unix timestamp, the value would be represented as <strong>Saturday, September 17, 1988 4:43:14 PM<\/strong>. <\/p>\n<p>To do the conversion, you can simply generate a Unix time stamp for January 1, 2001 and add up that value with the Core Data timestamp you have. Here is a quick example in Python:<\/p>\n<div class=\"highlight\"><pre><span><\/span><span class=\"kn\">from<\/span> <span class=\"nn\">datetime<\/span> <span class=\"kn\">import<\/span> <span class=\"n\">datetime<\/span>\n<span class=\"kn\">from<\/span> <span class=\"nn\">time<\/span> <span class=\"kn\">import<\/span> <span class=\"n\">mktime<\/span>\n\n\n<span class=\"n\">coredata_timestamp<\/span> <span class=\"o\">=<\/span> <span class=\"mi\">590517794<\/span>\n\n<span class=\"n\">coredata_start_date<\/span> <span class=\"o\">=<\/span> <span class=\"n\">datetime<\/span><span class=\"p\">(<\/span><span class=\"mi\">2001<\/span><span class=\"p\">,<\/span> <span class=\"mi\">1<\/span><span class=\"p\">,<\/span> <span class=\"mi\">1<\/span><span class=\"p\">,<\/span> <span class=\"mi\">0<\/span><span class=\"p\">,<\/span> <span class=\"mi\">0<\/span><span class=\"p\">,<\/span> <span class=\"mi\">0<\/span><span class=\"p\">,<\/span> <span class=\"mi\">0<\/span><span class=\"p\">,<\/span> <span class=\"n\">tzinfo<\/span><span class=\"o\">=<\/span><span class=\"bp\">None<\/span><span class=\"p\">)<\/span>\n\n<span class=\"n\">coredata_start_unix<\/span> <span class=\"o\">=<\/span> <span class=\"nb\">int<\/span><span class=\"p\">(<\/span><span class=\"n\">mktime<\/span><span class=\"p\">(<\/span><span class=\"n\">coredata_start_date<\/span><span class=\"o\">.<\/span><span class=\"n\">timetuple<\/span><span class=\"p\">()))<\/span>\n\n<span class=\"n\">unix_timestamp<\/span> <span class=\"o\">=<\/span> <span class=\"n\">coredata_start_unix<\/span> <span class=\"o\">+<\/span> <span class=\"n\">coredata_timestamp<\/span>\n\n<span class=\"k\">print<\/span><span class=\"p\">(<\/span><span class=\"n\">unix_timestamp<\/span><span class=\"p\">)<\/span>\n<\/pre><\/div>\n\n\n<p>The approach is pretty straightforward, but one thing to keep in mind is to make sure you're converting between the same units. For example, I've noticed that some Core Data timestamp values are stored in nano seconds.<\/p>\n<p>The example above assumes that we're converting between two timestamps that are both in seconds. If you're looking into converting a nano second based Core Data timestamp to a second based Unix timestamp make sure to divide that value by 1000000000 before the addition. While if you're looking into keeping the result in nano seconds make sure to get the starting Unix timestamp representing the Core Data base date in nano seconds too.<\/p>\n<p>I hope you find this quick tip useful, happy to hear your questions or feedback in the comments!<\/p>","category":[{"@attributes":{"term":"coredata"}},{"@attributes":{"term":"apple"}},{"@attributes":{"term":"timestamp"}},{"@attributes":{"term":"epoch"}},{"@attributes":{"term":"unix"}},{"@attributes":{"term":"conversion"}},{"@attributes":{"term":"python"}}]},{"title":"A simple algorithm for generating a dataset based on combinations of multiple sets","link":{"@attributes":{"href":"https:\/\/www.thecodeship.com\/algorithms\/simple-algorithm-generating-dataset-combinations-multiple-sets\/","rel":"alternate"}},"published":"2018-08-18T12:00:00+03:00","updated":"2018-08-18T12:00:00+03:00","author":{"name":"Ayman Farhat"},"id":"tag:www.thecodeship.com,2018-08-18:\/algorithms\/simple-algorithm-generating-dataset-combinations-multiple-sets\/","summary":"<p>Generating sample data based on the combination of specific parameters and options is a common use case. In this post, I illustrate utilizing a simple recursive approach for generating all possible combinations from input into a dataset.<\/p>","content":"<h2>Use case background<\/h2>\n<p>I was looking into putting together a sample dataset related to a range of airline routes and their related information. With that I was aiming to create a table containing an exhaustive set of combinations of an airline's operating origin and destination routes along with all the possible variations related to those O&amp;Ds tickets such as codeshare, language, class options etc...<\/p>\n<p>The content of the generated data set isn't meant to be random data, but should come from a set of options available for each column. For example, the program is fed the options below<\/p>\n<div class=\"highlight\"><pre><span><\/span><span class=\"nv\">Origin<\/span> <span class=\"nv\">code<\/span>: [<span class=\"nv\">DXB<\/span>, <span class=\"nv\">DWC<\/span>, <span class=\"nv\">ABC<\/span>, <span class=\"nv\">KFC<\/span>]\n<span class=\"nv\">Destination<\/span> <span class=\"nv\">code<\/span>: [<span class=\"nv\">ABC<\/span>, <span class=\"nv\">BJG<\/span>, <span class=\"nv\">KBJ<\/span>, <span class=\"nv\">DWC<\/span>]\n<span class=\"nv\">Class<\/span>: [<span class=\"nv\">First<\/span>, <span class=\"nv\">Business<\/span>, <span class=\"nv\">Economy<\/span>, <span class=\"nv\">General<\/span>]\n<span class=\"nv\">Language<\/span>: [<span class=\"nv\">EN<\/span>, <span class=\"nv\">AR<\/span>, <span class=\"nv\">FR<\/span>]\n<span class=\"nv\">Tier<\/span>: [<span class=\"mi\">1<\/span>, <span class=\"mi\">2<\/span>, <span class=\"mi\">3<\/span>]\n<span class=\"k\">Return<\/span>: [<span class=\"nv\">yes<\/span>, <span class=\"nv\">no<\/span>]\n<\/pre><\/div>\n\n\n<p>The resulting data set, would end up having unique rows generated baed on all the possible combinations in one row for all possible sets of input. In the case of airline routes, each row would constitute a representation of a \"ticket\" with an inventory of ticket options. Example output would look something like this:<\/p>\n<div class=\"highlight\"><pre><span><\/span><span class=\"p\">[<\/span> <span class=\"n\">DXB<\/span><span class=\"p\">,<\/span> <span class=\"n\">ABC<\/span><span class=\"p\">,<\/span> <span class=\"k\">First<\/span><span class=\"p\">,<\/span> <span class=\"n\">EN<\/span><span class=\"p\">,<\/span> <span class=\"mi\">1<\/span><span class=\"p\">,<\/span> <span class=\"n\">yes<\/span> <span class=\"p\">],<\/span>\n<span class=\"p\">[<\/span> <span class=\"n\">DXB<\/span><span class=\"p\">,<\/span> <span class=\"n\">ABC<\/span><span class=\"p\">,<\/span> <span class=\"n\">Business<\/span><span class=\"p\">,<\/span> <span class=\"n\">EN<\/span><span class=\"p\">,<\/span> <span class=\"mi\">1<\/span><span class=\"p\">,<\/span> <span class=\"n\">yes<\/span><span class=\"p\">],<\/span>\n<span class=\"p\">[<\/span> <span class=\"n\">DXB<\/span><span class=\"p\">,<\/span> <span class=\"n\">ABC<\/span><span class=\"p\">,<\/span> <span class=\"n\">Business<\/span><span class=\"p\">,<\/span> <span class=\"n\">EN<\/span><span class=\"p\">,<\/span> <span class=\"mi\">2<\/span><span class=\"p\">,<\/span> <span class=\"n\">yes<\/span><span class=\"p\">],<\/span>\n<span class=\"p\">[<\/span> <span class=\"n\">DXB<\/span><span class=\"p\">,<\/span> <span class=\"n\">ABC<\/span><span class=\"p\">,<\/span> <span class=\"n\">Business<\/span><span class=\"p\">,<\/span> <span class=\"n\">EN<\/span><span class=\"p\">,<\/span> <span class=\"mi\">3<\/span><span class=\"p\">,<\/span> <span class=\"n\">yes<\/span><span class=\"p\">],<\/span>\n\n<span class=\"n\">etc<\/span><span class=\"p\">...<\/span>\n<\/pre><\/div>\n\n\n<p>For the above example, the number of resulting unique rows would be the multiplication of the length of each column options set. In this case: <code>4 * 4 * 4 * 3 * 3 * 2 = 1152<\/code>. For the actual use case, origin And destination code are definitely much more (around) 200 which obviously would result in a much larger dataset.<\/p>\n<h2>Options available<\/h2>\n<p>For generating datasets related to the above case, there are obviously several libraries that can be included for building that. Most of the libraries out there usually also include the option of generating random data with different data types. <\/p>\n<p>In my case, I have the constraint of limiting the amount of dependencies as much as possible especially that the use case is pretty basic. <\/p>\n<p>The problem is mainly around \"multiplying\" different set items with each other to generate rows as permutations of all possible combinations for column values. That's usually known as the cartesian product.<\/p>\n<h2>Utilizing recursion<\/h2>\n<p>Following a recursive approach, we'd reduce the problem down to generating all combinations from 2 sets (denoted by left and right), then bubble that up and apply the result against one more set to generate more combinations against of the rest of the option sets.<\/p>\n<p>Here is the implementation in Javascript, the function below expects a 2 dimensional array as input.<\/p>\n<div class=\"highlight\"><pre><span><\/span><span class=\"kd\">let<\/span> <span class=\"nx\">buildCombo<\/span> <span class=\"o\">=<\/span> <span class=\"p\">(<\/span><span class=\"nx\">input<\/span><span class=\"p\">)<\/span> <span class=\"p\">=&gt;<\/span> <span class=\"p\">{<\/span>\n    <span class=\"k\">if<\/span> <span class=\"p\">(<\/span><span class=\"nx\">input<\/span><span class=\"p\">.<\/span><span class=\"nx\">length<\/span> <span class=\"o\">==<\/span> <span class=\"mi\">1<\/span><span class=\"p\">)<\/span> <span class=\"p\">{<\/span>\n        <span class=\"k\">return<\/span> <span class=\"nx\">input<\/span><span class=\"p\">[<\/span><span class=\"mi\">0<\/span><span class=\"p\">];<\/span>\n    <span class=\"p\">}<\/span> <span class=\"k\">else<\/span> <span class=\"p\">{<\/span>\n        <span class=\"kd\">let<\/span> <span class=\"nx\">result<\/span> <span class=\"o\">=<\/span> <span class=\"p\">[];<\/span>\n        <span class=\"kd\">let<\/span> <span class=\"nx\">remainingCombo<\/span> <span class=\"o\">=<\/span> <span class=\"nx\">buildCombo<\/span><span class=\"p\">(<\/span><span class=\"nx\">input<\/span><span class=\"p\">.<\/span><span class=\"nx\">slice<\/span><span class=\"p\">(<\/span><span class=\"mi\">1<\/span><span class=\"p\">));<\/span>\n\n        <span class=\"k\">for<\/span> <span class=\"p\">(<\/span><span class=\"kd\">let<\/span> <span class=\"nx\">leftItem<\/span> <span class=\"k\">of<\/span> <span class=\"nx\">remainingCombo<\/span><span class=\"p\">)<\/span> <span class=\"p\">{<\/span>\n            <span class=\"k\">for<\/span> <span class=\"p\">(<\/span><span class=\"kd\">let<\/span> <span class=\"nx\">rightItem<\/span> <span class=\"k\">of<\/span> <span class=\"nx\">input<\/span><span class=\"p\">[<\/span><span class=\"mi\">0<\/span><span class=\"p\">])<\/span> <span class=\"p\">{<\/span>\n                <span class=\"nx\">result<\/span><span class=\"p\">.<\/span><span class=\"nx\">push<\/span><span class=\"p\">([<\/span><span class=\"nx\">rightItem<\/span><span class=\"p\">].<\/span><span class=\"nx\">concat<\/span><span class=\"p\">(<\/span><span class=\"nx\">leftItem<\/span><span class=\"p\">));<\/span>\n            <span class=\"p\">}<\/span>\n        <span class=\"p\">}<\/span>\n\n        <span class=\"k\">return<\/span> <span class=\"nx\">result<\/span><span class=\"p\">;<\/span>\n    <span class=\"p\">}<\/span>\n<span class=\"p\">}<\/span>\n<\/pre><\/div>\n\n\n<p>Example usage:<\/p>\n<div class=\"highlight\"><pre><span><\/span><span class=\"kd\">let<\/span> <span class=\"nx\">contentOptions<\/span> <span class=\"o\">=<\/span> <span class=\"p\">[<\/span>\n    <span class=\"p\">[<\/span><span class=\"s1\">&#39;DXB&#39;<\/span><span class=\"p\">,<\/span> <span class=\"s1\">&#39;DWC&#39;<\/span><span class=\"p\">,<\/span> <span class=\"s1\">&#39;XYZ&#39;<\/span><span class=\"p\">,<\/span> <span class=\"s1\">&#39;KFC&#39;<\/span><span class=\"p\">,<\/span> <span class=\"s1\">&#39;BCJ&#39;<\/span><span class=\"p\">,<\/span> <span class=\"s1\">&#39;PPP&#39;<\/span><span class=\"p\">],<\/span>\n    <span class=\"p\">[<\/span><span class=\"s1\">&#39;DXB&#39;<\/span><span class=\"p\">,<\/span> <span class=\"s1\">&#39;DWC&#39;<\/span><span class=\"p\">,<\/span> <span class=\"s1\">&#39;XYZ&#39;<\/span><span class=\"p\">,<\/span> <span class=\"s1\">&#39;KFC&#39;<\/span><span class=\"p\">,<\/span> <span class=\"s1\">&#39;BCJ&#39;<\/span><span class=\"p\">,<\/span> <span class=\"s1\">&#39;PPP&#39;<\/span><span class=\"p\">],<\/span>\n    <span class=\"p\">[<\/span><span class=\"s1\">&#39;AR&#39;<\/span><span class=\"p\">,<\/span> <span class=\"s1\">&#39;EN&#39;<\/span><span class=\"p\">,<\/span> <span class=\"s1\">&#39;FR&#39;<\/span><span class=\"p\">],<\/span>\n    <span class=\"p\">[<\/span><span class=\"s1\">&#39;First&#39;<\/span><span class=\"p\">,<\/span> <span class=\"s1\">&#39;Business&#39;<\/span><span class=\"p\">,<\/span> <span class=\"s1\">&#39;Economy&#39;<\/span><span class=\"p\">],<\/span>\n    <span class=\"p\">[<\/span><span class=\"s1\">&#39;yes&#39;<\/span><span class=\"p\">,<\/span> <span class=\"s1\">&#39;no&#39;<\/span><span class=\"p\">],<\/span>\n    <span class=\"p\">[<\/span><span class=\"s1\">&#39;yes&#39;<\/span><span class=\"p\">,<\/span> <span class=\"s1\">&#39;no&#39;<\/span><span class=\"p\">]<\/span>\n<span class=\"p\">];<\/span>\n\n<span class=\"kd\">let<\/span> <span class=\"nx\">results<\/span> <span class=\"o\">=<\/span> <span class=\"nx\">buildCombo<\/span><span class=\"p\">(<\/span><span class=\"nx\">contentOptions<\/span><span class=\"p\">);<\/span>\n<\/pre><\/div>\n\n\n<h2>Cleaning the results<\/h2>\n<p>As promised, the above function generates all possible combinations. One caveat though, is that in some cases the data needs to fit the use case constraints. For example, in the dataset I'm trying to generate, it wouldn't make sense to have rows containing an origin airport code equal to a destination airport code because obviously, for any ticket it's impossible for a trip to be from and to itself.<\/p>\n<p>This can be easily solved by applying a filter function on the resulting data to weed out the duplicates, for example:<\/p>\n<div class=\"highlight\"><pre><span><\/span><span class=\"kd\">let<\/span> <span class=\"nx\">cleanup<\/span> <span class=\"o\">=<\/span> <span class=\"nx\">results<\/span><span class=\"p\">.<\/span><span class=\"nx\">filter<\/span><span class=\"p\">(<\/span><span class=\"nx\">row<\/span> <span class=\"p\">=&gt;<\/span> <span class=\"nx\">row<\/span><span class=\"p\">[<\/span><span class=\"mi\">0<\/span><span class=\"p\">]<\/span> <span class=\"o\">!==<\/span> <span class=\"nx\">row<\/span><span class=\"p\">[<\/span><span class=\"mi\">1<\/span><span class=\"p\">]);<\/span>\n<\/pre><\/div>\n\n\n<p>From that point onward, it would be easy to export that into CSV after escaping the data.<\/p>\n<p>Hope you find this tip useful, happy to hear your comments on the solution and any suggested improvements!<\/p>","category":[{"@attributes":{"term":"datasets"}},{"@attributes":{"term":"combinations"}},{"@attributes":{"term":"input"}},{"@attributes":{"term":"recursion"}},{"@attributes":{"term":"permutation"}}]},{"title":"A guide on implementing free SSL certificate on Nginx using Let's Encrypt","link":{"@attributes":{"href":"https:\/\/www.thecodeship.com\/web-development\/guide-implementing-free-ssl-certificate-nginx-lets-encrypt\/","rel":"alternate"}},"published":"2017-05-07T17:35:00+03:00","updated":"2017-05-07T17:35:00+03:00","author":{"name":"Ayman Farhat"},"id":"tag:www.thecodeship.com,2017-05-07:\/web-development\/guide-implementing-free-ssl-certificate-nginx-lets-encrypt\/","summary":"<p>If you're reading this article, I'm assuming that you're already sold on the numerous benefits of securing your website with an SSL certificate. Wether it's for encrypting data transfers, establishing more credibility or even improving your <a href=\"http:\/\/searchengineland.com\/googles-gary-illyes-https-may-break-ties-between-two-equal-search-results-230691\">SEO ranking<\/a> (yes you heard it right) it's now easier than ever to obtain \u2026<\/p>","content":"<p>If you're reading this article, I'm assuming that you're already sold on the numerous benefits of securing your website with an SSL certificate. Wether it's for encrypting data transfers, establishing more credibility or even improving your <a href=\"http:\/\/searchengineland.com\/googles-gary-illyes-https-may-break-ties-between-two-equal-search-results-230691\">SEO ranking<\/a> (yes you heard it right) it's now easier than ever to obtain an SSL certificate for that.<\/p>\n<p>Traditionally, an SSL certificate costs anywhere from $30 to $100 a year depending on where you're hosting it. It has always been a struggle to find a truly reliable authority for obtaining a free SSL.<\/p>\n<p>Luckily, the Linux Foundation decided to step up and solve this problem by creating <a href=\"https:\/\/letsencrypt.org\/about\/\">Let's Encyrpt<\/a>, a free, automated, and open CA (Certificate Authority) for everyone. With Let's Encrypt, you can now <a href=\"https:\/\/blog.codinghorror.com\/lets-encrypt-everything\/\">encrypt everything<\/a> for free.<\/p>\n<p>Let's Encrypt is a fantastic service, but with the vast amount of 3rd party tutorials out there and so many changes that happened with the service itself through the different phases, I found most of the content either confusing or just obsolete. In the end, and after a lot of research from here and there, I managed to get it working.<\/p>\n<p>The process itself turned out to be pretty simple, so I've summarised a full tutorial on how to do it on Ubuntu 16.04 with Nginx in 4 straightforward steps. The steps should be doable under other Linux flavours or Ubuntu versions, with small differences in the commands.<\/p>\n<h2>Some background about Let's Encrypt<\/h2>\n<p>Before diving into the process I want to highlight a couple of details on the implementation. <\/p>\n<p>The basic idea for identifying domain control in order to issue or renew certificates is to serve letsencrypt \"ACME Challenges\" from your web server.<\/p>\n<p>There are mainly 2 modes of operation that you can setup your certificate on:<\/p>\n<ol>\n<li>\n<p><strong>Standalone<\/strong><\/p>\n<p>This mode is used to obtain a certificate if you don't want to use or   already have a web server setup. It does not rely on the web server     software (such as Nginx) to identify the domain and obtain a    certificate. In that case, letsencrypt would have to bind to port 80    and serve the challenges.<\/p>\n<p>More details about stand alone mode <a href=\"https:\/\/certbot.eff.org\/docs\/using.html#standalone\">here<\/a><\/p>\n<\/li>\n<li>\n<p><strong>Webroot<\/strong><\/p>\n<p>This mode allows you to serve the challenges from your locally  running web server (such as Nginx) from a known folder. You don't   need to stop the server during the issuance or renewal process while    it's already bound to port 80. More details will have to included   related to your project web root paths.<\/p>\n<p>The great thing about this mode is that it's straightforward to setup   wether you have an existing web server installed or a installing a  new one.<\/p>\n<p>More details about Webroot mode <a href=\"https:\/\/certbot.eff.org\/docs\/using.html#webroot\">here<\/a><\/p>\n<\/li>\n<\/ol>\n<p>For more information on how Let's Encrypt works, you check out this <a href=\"https:\/\/letsencrypt.org\/how-it-works\/\">link<\/a><\/p>\n<p>In this guide I'll be going with the webroot mode with Nginx, so let's dive in!<\/p>\n<h2>Step 1: Setup and prepare Nginx<\/h2>\n<p>First of all, we'll install and setup all the files related to Nginx and the base configuration, if you don't have a web server installed yet now is the time to do so.<\/p>\n<div class=\"highlight\"><pre><span><\/span><span class=\"n\">sudo<\/span> <span class=\"n\">apt<\/span><span class=\"o\">-<\/span><span class=\"k\">get<\/span> <span class=\"n\">install<\/span> <span class=\"n\">nginx<\/span>\n<\/pre><\/div>\n\n\n<p>Now that we have Nginx, lets setup 2 main config files that can be included into our main Nginx config later. Optionally, you could jusr include those sections directly into your main Nginx site config file but it's more flexible to separate and include them on demand. The files we'll be creating below can be re-used with multiple Nginx site configurations.<\/p>\n<h3>Let's encrypt challenge<\/h3>\n<p>As mentioned earlier, we need to create a hidden directory and configure our web server to serve the challenges from it, in order for the certificate authority to validate the domain.<\/p>\n<p>Let's create the file<\/p>\n<div class=\"highlight\"><pre><span><\/span><span class=\"o\">\/<\/span><span class=\"n\">etc<\/span><span class=\"o\">\/<\/span><span class=\"n\">nginx<\/span><span class=\"o\">\/<\/span><span class=\"n\">snippets<\/span><span class=\"o\">\/<\/span><span class=\"n\">letsencrypt<\/span><span class=\"p\">.<\/span><span class=\"n\">conf<\/span>\n<\/pre><\/div>\n\n\n<p>and add the following configuration into it<\/p>\n<div class=\"highlight\"><pre><span><\/span><span class=\"k\">location<\/span> <span class=\"o\">^~<\/span> <span class=\"o\">\/<\/span><span class=\"p\">.<\/span><span class=\"n\">well<\/span><span class=\"o\">-<\/span><span class=\"n\">known<\/span><span class=\"o\">\/<\/span><span class=\"n\">acme<\/span><span class=\"o\">-<\/span><span class=\"n\">challenge<\/span><span class=\"o\">\/<\/span> <span class=\"err\">{<\/span>\n    <span class=\"n\">default_type<\/span> <span class=\"ss\">&quot;text\/plain&quot;<\/span><span class=\"p\">;<\/span>\n    <span class=\"n\">root<\/span> <span class=\"o\">\/<\/span><span class=\"n\">var<\/span><span class=\"o\">\/<\/span><span class=\"n\">www<\/span><span class=\"o\">\/<\/span><span class=\"n\">letsencrypt<\/span><span class=\"p\">;<\/span>\n<span class=\"err\">}<\/span>\n<\/pre><\/div>\n\n\n<p>Then we can create an empty challenges directory, that's where Let's Ecnrypt will be adding the challenges to later on.<\/p>\n<div class=\"highlight\"><pre><span><\/span><span class=\"n\">sudo<\/span> <span class=\"n\">mkdir<\/span> <span class=\"o\">-<\/span><span class=\"n\">p<\/span> <span class=\"o\">\/<\/span><span class=\"n\">var<\/span><span class=\"o\">\/<\/span><span class=\"n\">www<\/span><span class=\"o\">\/<\/span><span class=\"n\">letsencrypt<\/span><span class=\"o\">\/<\/span><span class=\"p\">.<\/span><span class=\"n\">well<\/span><span class=\"o\">-<\/span><span class=\"n\">known<\/span><span class=\"o\">\/<\/span><span class=\"n\">acme<\/span><span class=\"o\">-<\/span><span class=\"n\">challenge<\/span>\n<\/pre><\/div>\n\n\n<p>Now that we have <code>\/etc\/nginx\/snippets\/letsencrypt.conf<\/code> available, we can later plug it into any individual Nginx website configuration and have it serve challenges for that domain on the web server.<\/p>\n<h3>SSL configuration<\/h3>\n<p>In order to run the web server on SSL, we need to provide some configuration related to the SSL certificate such as encryption type, headers, protocols etc...<\/p>\n<p>This configuration can also be added into a separate file, and later on plugged into any Nginx configuration that will use Let's Encrypt certificate.<\/p>\n<p>Lets create the file<\/p>\n<div class=\"highlight\"><pre><span><\/span><span class=\"o\">\/<\/span><span class=\"n\">etc<\/span><span class=\"o\">\/<\/span><span class=\"n\">nginx<\/span><span class=\"o\">\/<\/span><span class=\"n\">snippets<\/span><span class=\"o\">\/<\/span><span class=\"n\">ssl<\/span><span class=\"p\">.<\/span><span class=\"n\">conf<\/span>\n<\/pre><\/div>\n\n\n<p>with the following contents<\/p>\n<div class=\"highlight\"><pre><span><\/span><span class=\"nt\">ssl_session_timeout<\/span> <span class=\"nt\">1d<\/span><span class=\"o\">;<\/span>\n<span class=\"nt\">ssl_session_cache<\/span> <span class=\"nt\">shared<\/span><span class=\"p\">:<\/span><span class=\"nd\">SSL<\/span><span class=\"p\">:<\/span><span class=\"nd\">50m<\/span><span class=\"o\">;<\/span>\n<span class=\"nt\">ssl_session_tickets<\/span> <span class=\"nt\">off<\/span><span class=\"o\">;<\/span>\n\n<span class=\"nt\">ssl_protocols<\/span> <span class=\"nt\">TLSv1<\/span><span class=\"p\">.<\/span><span class=\"nc\">2<\/span><span class=\"o\">;<\/span>\n<span class=\"nt\">ssl_ciphers<\/span> <span class=\"nt\">EECDH<\/span><span class=\"o\">+<\/span><span class=\"nt\">AESGCM<\/span><span class=\"p\">:<\/span><span class=\"nd\">EECDH<\/span><span class=\"o\">+<\/span><span class=\"nt\">AES<\/span><span class=\"o\">;<\/span>\n<span class=\"nt\">ssl_ecdh_curve<\/span> <span class=\"nt\">secp384r1<\/span><span class=\"o\">;<\/span>\n<span class=\"nt\">ssl_prefer_server_ciphers<\/span> <span class=\"nt\">on<\/span><span class=\"o\">;<\/span>\n\n<span class=\"nt\">ssl_stapling<\/span> <span class=\"nt\">on<\/span><span class=\"o\">;<\/span>\n<span class=\"nt\">ssl_stapling_verify<\/span> <span class=\"nt\">on<\/span><span class=\"o\">;<\/span>\n\n<span class=\"nt\">add_header<\/span> <span class=\"nt\">Strict-Transport-Security<\/span> <span class=\"s2\">&quot;max-age=15768000; includeSubdomains; preload&quot;<\/span><span class=\"o\">;<\/span>\n<span class=\"nt\">add_header<\/span> <span class=\"nt\">X-Frame-Options<\/span> <span class=\"nt\">DENY<\/span><span class=\"o\">;<\/span>\n<span class=\"nt\">add_header<\/span> <span class=\"nt\">X-Content-Type-Options<\/span> <span class=\"nt\">nosniff<\/span><span class=\"o\">;<\/span>\n<\/pre><\/div>\n\n\n<h3>Nginx basic configuration<\/h3>\n<p>Once the above 2 files are ready, we can now include them in any website specific Nginx configuration. <\/p>\n<p>We'll first need set up a basic Nginx config (HTTP only) on our website domain in order to serve the challenges and identify the domain name ownership. We'll later on build up on this same Nginx config file for enabling SSL after obtaining the SSL certificate in step 2.<\/p>\n<p><strong>If you already have an established website being served on HTTP, you probably have a very similar setup to the below. All you will need to do in that case is to just reference <code>include \/etc\/nginx\/snippets\/letsencrypt.conf;<\/code> in your existing config file and restart your web server. Otherwise, you can create a config from scratch as specified below.<\/strong><\/p>\n<p>Create your Nginx config file<\/p>\n<div class=\"highlight\"><pre><span><\/span><span class=\"o\">\/<\/span><span class=\"n\">etc<\/span><span class=\"o\">\/<\/span><span class=\"n\">nginx<\/span><span class=\"o\">\/<\/span><span class=\"n\">sites<\/span><span class=\"o\">-<\/span><span class=\"n\">available<\/span><span class=\"o\">\/<\/span><span class=\"n\">mywebsite<\/span><span class=\"p\">.<\/span><span class=\"n\">conf<\/span>\n<\/pre><\/div>\n\n\n<p>With the contents similar to<\/p>\n<div class=\"highlight\"><pre><span><\/span><span class=\"nv\">server<\/span> {\n    <span class=\"nv\">listen<\/span> <span class=\"mi\">80<\/span> <span class=\"nv\">default_server<\/span><span class=\"c1\">;<\/span>\n    <span class=\"nv\">listen<\/span> [::]:<span class=\"mi\">80<\/span> <span class=\"nv\">default_server<\/span> <span class=\"nv\">ipv6only<\/span><span class=\"o\">=<\/span><span class=\"nv\">on<\/span><span class=\"c1\">;<\/span>\n    <span class=\"nv\">server_name<\/span> <span class=\"nv\">mywebsite<\/span>.<span class=\"nv\">com<\/span> <span class=\"nv\">www<\/span>.<span class=\"nv\">mywebsite<\/span>.<span class=\"nv\">com<\/span><span class=\"c1\">;<\/span>\n\n    <span class=\"k\">include<\/span> <span class=\"o\">\/<\/span><span class=\"nv\">etc<\/span><span class=\"o\">\/<\/span><span class=\"nv\">nginx<\/span><span class=\"o\">\/<\/span><span class=\"nv\">snippets<\/span><span class=\"o\">\/<\/span><span class=\"nv\">letsencrypt<\/span>.<span class=\"nv\">conf<\/span><span class=\"c1\">;<\/span>\n\n    <span class=\"nv\">root<\/span> <span class=\"o\">\/<\/span><span class=\"nv\">var<\/span><span class=\"o\">\/<\/span><span class=\"nv\">www<\/span><span class=\"o\">\/<\/span><span class=\"nv\">mywebsite<\/span><span class=\"c1\">;<\/span>\n    <span class=\"nv\">index<\/span> <span class=\"nv\">index<\/span>.<span class=\"nv\">html<\/span><span class=\"c1\">;<\/span>\n\n    <span class=\"nv\">location<\/span> <span class=\"o\">\/<\/span> {\n        <span class=\"nv\">try_files<\/span> $<span class=\"nv\">uri<\/span> $<span class=\"nv\">uri<\/span><span class=\"o\">\/<\/span> <span class=\"o\">=<\/span><span class=\"mi\">404<\/span><span class=\"c1\">;<\/span>\n    }\n}\n<\/pre><\/div>\n\n\n<p>Enable the config file<\/p>\n<div class=\"highlight\"><pre><span><\/span><span class=\"n\">ln<\/span> <span class=\"o\">-<\/span><span class=\"n\">s<\/span> <span class=\"o\">\/<\/span><span class=\"n\">etc<\/span><span class=\"o\">\/<\/span><span class=\"n\">nginx<\/span><span class=\"o\">\/<\/span><span class=\"n\">sites<\/span><span class=\"o\">-<\/span><span class=\"n\">available<\/span><span class=\"o\">\/<\/span><span class=\"n\">mywebsite<\/span><span class=\"p\">.<\/span><span class=\"n\">conf<\/span> <span class=\"o\">\/<\/span><span class=\"n\">etc<\/span><span class=\"o\">\/<\/span><span class=\"n\">nginx<\/span><span class=\"o\">\/<\/span><span class=\"n\">sites<\/span><span class=\"o\">-<\/span><span class=\"n\">enabled<\/span><span class=\"o\">\/<\/span><span class=\"n\">mywebsite<\/span><span class=\"p\">.<\/span><span class=\"n\">conf<\/span>\n<\/pre><\/div>\n\n\n<p>Restart the web server<\/p>\n<div class=\"highlight\"><pre><span><\/span><span class=\"n\">sudo<\/span> <span class=\"n\">service<\/span> <span class=\"n\">nginx<\/span> <span class=\"k\">restart<\/span>\n<\/pre><\/div>\n\n\n<p>Once this is done, the web server should be able to serve challenges under <code>.well-known\/acme-challenge<\/code> through your domain name. As you have noticed, this is after including <code>letsecrypt.conf<\/code> into our domain's config.<\/p>\n<h2>Step 2: Install and generate a Let's Encrypt certificate<\/h2>\n<p>Now that the challenges can be served under the domain, we'll be able to verify the ownership. Proceed with installing Let's Encrypt and obtaining a certificate.<\/p>\n<p>Install Let's Encrypt via <\/p>\n<div class=\"highlight\"><pre><span><\/span><span class=\"n\">sudo<\/span> <span class=\"n\">apt<\/span><span class=\"o\">-<\/span><span class=\"k\">get<\/span> <span class=\"n\">install<\/span> <span class=\"n\">letsencrypt<\/span>\n<\/pre><\/div>\n\n\n<p>Obtain the certificate via<\/p>\n<div class=\"highlight\"><pre><span><\/span><span class=\"n\">letsencrypt<\/span><span class=\"w\"> <\/span><span class=\"n\">certonly<\/span><span class=\"w\"> <\/span><span class=\"c1\">--webroot -w \/var\/www\/letsencrypt -d www.mywebsite.com -d mywebsite.com --email your@email.com --agree-tos<\/span>\n<\/pre><\/div>\n\n\n<p>Here is a breakdown explaining the command parameters above:<\/p>\n<ul>\n<li><strong>-w \/var\/www\/letsencrypt<\/strong> Specifies the root directory that we're serving your website challenges from<\/li>\n<li><strong>--webroot<\/strong> Specifies that we're using the web root mode<\/li>\n<li><strong>-d www.mywebsite.com<\/strong> Specifies the domain name we're issuing the certificate for, it can be multiple ones as it's above with www and without www<\/li>\n<li><strong>--email your@email.com<\/strong> Specifies the working email address to to associate with this certificate.<\/li>\n<\/ul>\n<p>After running the command, the SSL related keys and files will be available under<\/p>\n<div class=\"highlight\"><pre><span><\/span><span class=\"o\">\/<\/span><span class=\"n\">etc<\/span><span class=\"o\">\/<\/span><span class=\"n\">letsencrypt<\/span><span class=\"o\">\/<\/span><span class=\"n\">live<\/span><span class=\"o\">\/<\/span><span class=\"n\">www<\/span><span class=\"p\">.<\/span><span class=\"n\">mywebsite<\/span><span class=\"p\">.<\/span><span class=\"n\">com<\/span><span class=\"o\">\/<\/span>\n<\/pre><\/div>\n\n\n<p>For more information about the <code>letsencrypt<\/code> command check out the detailed <a href=\"http:\/\/letsencrypt.readthedocs.io\/en\/latest\/using.html#certbot-commands\">official docs<\/a>.<\/p>\n<h2>Step 3: Integrate the certificate into the config<\/h2>\n<p>Now that the certificate and Nginx domain config are ready, all we have to do is link to the certificate of the domain from the site's config. Open up <code>\/etc\/nginx\/sites-available\/mywebsite.conf<\/code>.<\/p>\n<p>Adding the contents below to the file will enable SSL on the <strong>www<\/strong> version of the domain.<\/p>\n<div class=\"highlight\"><pre><span><\/span> <span class=\"nv\">server<\/span> {\n    <span class=\"nv\">server_name<\/span> <span class=\"nv\">www<\/span>.<span class=\"nv\">mywebsite<\/span>.<span class=\"nv\">com<\/span><span class=\"c1\">;<\/span>\n    <span class=\"nv\">listen<\/span> <span class=\"mi\">443<\/span> <span class=\"nv\">ssl<\/span> <span class=\"nv\">http2<\/span> <span class=\"nv\">default_server<\/span><span class=\"c1\">;<\/span>\n    <span class=\"nv\">listen<\/span> [::]:<span class=\"mi\">443<\/span> <span class=\"nv\">ssl<\/span> <span class=\"nv\">http2<\/span> <span class=\"nv\">default_server<\/span> <span class=\"nv\">ipv6only<\/span><span class=\"o\">=<\/span><span class=\"nv\">on<\/span><span class=\"c1\">;<\/span>\n\n    <span class=\"nv\">ssl_certificate<\/span> <span class=\"o\">\/<\/span><span class=\"nv\">etc<\/span><span class=\"o\">\/<\/span><span class=\"nv\">letsencrypt<\/span><span class=\"o\">\/<\/span><span class=\"nv\">live<\/span><span class=\"o\">\/<\/span><span class=\"nv\">www<\/span>.<span class=\"nv\">mywebsite<\/span>.<span class=\"nv\">com<\/span><span class=\"o\">\/<\/span><span class=\"nv\">fullchain<\/span>.<span class=\"nv\">pem<\/span><span class=\"c1\">;<\/span>\n    <span class=\"nv\">ssl_certificate_key<\/span> <span class=\"o\">\/<\/span><span class=\"nv\">etc<\/span><span class=\"o\">\/<\/span><span class=\"nv\">letsencrypt<\/span><span class=\"o\">\/<\/span><span class=\"nv\">live<\/span><span class=\"o\">\/<\/span><span class=\"nv\">www<\/span>.<span class=\"nv\">mywebsite<\/span>.<span class=\"nv\">com<\/span><span class=\"o\">\/<\/span><span class=\"nv\">privkey<\/span>.<span class=\"nv\">pem<\/span><span class=\"c1\">;<\/span>\n    <span class=\"nv\">ssl_trusted_certificate<\/span> <span class=\"o\">\/<\/span><span class=\"nv\">etc<\/span><span class=\"o\">\/<\/span><span class=\"nv\">letsencrypt<\/span><span class=\"o\">\/<\/span><span class=\"nv\">live<\/span><span class=\"o\">\/<\/span><span class=\"nv\">www<\/span>.<span class=\"nv\">mywebsite<\/span>.<span class=\"nv\">com<\/span><span class=\"o\">\/<\/span><span class=\"nv\">fullchain<\/span>.<span class=\"nv\">pem<\/span><span class=\"c1\">;<\/span>\n    <span class=\"k\">include<\/span> <span class=\"o\">\/<\/span><span class=\"nv\">etc<\/span><span class=\"o\">\/<\/span><span class=\"nv\">nginx<\/span><span class=\"o\">\/<\/span><span class=\"nv\">snippets<\/span><span class=\"o\">\/<\/span><span class=\"nv\">ssl<\/span>.<span class=\"nv\">conf<\/span><span class=\"c1\">;<\/span>\n\n    <span class=\"nv\">root<\/span> <span class=\"o\">\/<\/span><span class=\"nv\">var<\/span><span class=\"o\">\/<\/span><span class=\"nv\">www<\/span><span class=\"o\">\/<\/span><span class=\"nv\">mywebsite<\/span>.<span class=\"nv\">com<\/span><span class=\"c1\">;<\/span>\n    <span class=\"nv\">index<\/span> <span class=\"nv\">index<\/span>.<span class=\"nv\">html<\/span><span class=\"c1\">;<\/span>\n    <span class=\"nv\">location<\/span> <span class=\"o\">\/<\/span> {\n        <span class=\"nv\">try_files<\/span> $<span class=\"nv\">uri<\/span> $<span class=\"nv\">uri<\/span><span class=\"o\">\/<\/span> <span class=\"o\">=<\/span><span class=\"mi\">404<\/span><span class=\"c1\">;<\/span>\n    }\n}\n<\/pre><\/div>\n\n\n<p>The above domain configuration is very similar to the initial one, the difference is that<\/p>\n<ul>\n<li>It listens on port 443 (the default port for SSL)<\/li>\n<li>It's for the www version of the domain<\/li>\n<li>It includes the <code>ssl.conf<\/code> file previously created in step 1 and links the generated certificate via <strong>ssl_certificate<\/strong>, <strong>ssl_certificate_key<\/strong>, and <strong>ssl_trusted_certificate<\/strong> properties.<\/li>\n<\/ul>\n<p>Restarting the web server via <code>sudo service nginx restart<\/code> will enable this configuration and allow us to access the website via <strong>https:\/\/www.mywebsite.com<\/strong> hurray! :)<\/p>\n<p>One important thing to notice is that the site is still accessible via non www and non https URls such as<\/p>\n<ul>\n<li>http:\/\/mywebsite.com<\/li>\n<li>http:\/\/www.mywebsite.com<\/li>\n<\/ul>\n<p>And <strong>https:\/\/mywebsite.com<\/strong> is not even accessible.<\/p>\n<p>It's usually a best practice (especially for SEO) to just force https on your domain and use only one style either with www or without www. This can be easily done through 301 redirects on those variations.<\/p>\n<p>First of all update the initially created non https config to redirect all requests to https:\/\/www.mywebsite.com via <\/p>\n<div class=\"highlight\"><pre><span><\/span><span class=\"nv\">server<\/span> {\n    <span class=\"nv\">listen<\/span> <span class=\"mi\">80<\/span> <span class=\"nv\">default_server<\/span><span class=\"c1\">;<\/span>\n    <span class=\"nv\">listen<\/span> [::]:<span class=\"mi\">80<\/span> <span class=\"nv\">default_server<\/span> <span class=\"nv\">ipv6only<\/span><span class=\"o\">=<\/span><span class=\"nv\">on<\/span><span class=\"c1\">;<\/span>\n    <span class=\"nv\">server_name<\/span> <span class=\"nv\">mydomain<\/span>.<span class=\"nv\">com<\/span> <span class=\"nv\">www<\/span>.<span class=\"nv\">mydomain<\/span>.<span class=\"nv\">com<\/span><span class=\"c1\">;<\/span>\n\n    <span class=\"k\">include<\/span> <span class=\"o\">\/<\/span><span class=\"nv\">etc<\/span><span class=\"o\">\/<\/span><span class=\"nv\">nginx<\/span><span class=\"o\">\/<\/span><span class=\"nv\">snippets<\/span><span class=\"o\">\/<\/span><span class=\"nv\">letsencrypt<\/span>.<span class=\"nv\">conf<\/span><span class=\"c1\">;<\/span>\n\n    <span class=\"nv\">location<\/span> <span class=\"o\">\/<\/span> {\n        <span class=\"k\">return<\/span> <span class=\"mi\">301<\/span> <span class=\"nv\">https<\/span>:<span class=\"o\">\/\/<\/span><span class=\"nv\">www<\/span>.<span class=\"nv\">mydomain<\/span>.<span class=\"nv\">com<\/span>$<span class=\"nv\">request_uri<\/span><span class=\"c1\">;<\/span>\n    }\n}\n<\/pre><\/div>\n\n\n<p>Then also make sure to redirect non www https to www by adding<\/p>\n<div class=\"highlight\"><pre><span><\/span><span class=\"nv\">server<\/span> {\n    <span class=\"nv\">listen<\/span> <span class=\"mi\">443<\/span> <span class=\"nv\">ssl<\/span> <span class=\"nv\">http2<\/span><span class=\"c1\">;<\/span>\n    <span class=\"nv\">listen<\/span> [::]:<span class=\"mi\">443<\/span> <span class=\"nv\">ssl<\/span> <span class=\"nv\">http2<\/span><span class=\"c1\">;<\/span>\n    <span class=\"nv\">server_name<\/span> <span class=\"nv\">mydomain<\/span>.<span class=\"nv\">com<\/span><span class=\"c1\">;<\/span>\n\n    <span class=\"nv\">ssl_certificate<\/span> <span class=\"o\">\/<\/span><span class=\"nv\">etc<\/span><span class=\"o\">\/<\/span><span class=\"nv\">letsencrypt<\/span><span class=\"o\">\/<\/span><span class=\"nv\">live<\/span><span class=\"o\">\/<\/span><span class=\"nv\">www<\/span>.<span class=\"nv\">mydomain<\/span>.<span class=\"nv\">com<\/span><span class=\"o\">\/<\/span><span class=\"nv\">fullchain<\/span>.<span class=\"nv\">pem<\/span><span class=\"c1\">;<\/span>\n    <span class=\"nv\">ssl_certificate_key<\/span> <span class=\"o\">\/<\/span><span class=\"nv\">etc<\/span><span class=\"o\">\/<\/span><span class=\"nv\">letsencrypt<\/span><span class=\"o\">\/<\/span><span class=\"nv\">live<\/span><span class=\"o\">\/<\/span><span class=\"nv\">www<\/span>.<span class=\"nv\">mydomain<\/span>.<span class=\"nv\">com<\/span><span class=\"o\">\/<\/span><span class=\"nv\">privkey<\/span>.<span class=\"nv\">pem<\/span><span class=\"c1\">;<\/span>\n    <span class=\"nv\">ssl_trusted_certificate<\/span> <span class=\"o\">\/<\/span><span class=\"nv\">etc<\/span><span class=\"o\">\/<\/span><span class=\"nv\">letsencrypt<\/span><span class=\"o\">\/<\/span><span class=\"nv\">live<\/span><span class=\"o\">\/<\/span><span class=\"nv\">www<\/span>.<span class=\"nv\">mydomain<\/span>.<span class=\"nv\">com<\/span><span class=\"o\">\/<\/span><span class=\"nv\">fullchain<\/span>.<span class=\"nv\">pem<\/span><span class=\"c1\">;<\/span>\n    <span class=\"k\">include<\/span> <span class=\"o\">\/<\/span><span class=\"nv\">etc<\/span><span class=\"o\">\/<\/span><span class=\"nv\">nginx<\/span><span class=\"o\">\/<\/span><span class=\"nv\">snippets<\/span><span class=\"o\">\/<\/span><span class=\"nv\">ssl<\/span>.<span class=\"nv\">conf<\/span><span class=\"c1\">;<\/span>\n\n    <span class=\"nv\">location<\/span> <span class=\"o\">\/<\/span> {\n        <span class=\"k\">return<\/span> <span class=\"mi\">301<\/span> <span class=\"nv\">https<\/span>:<span class=\"o\">\/\/<\/span><span class=\"nv\">www<\/span>.<span class=\"nv\">mydomain<\/span>.<span class=\"nv\">com<\/span>$<span class=\"nv\">request_uri<\/span><span class=\"c1\">;<\/span>\n    }\n}\n<\/pre><\/div>\n\n\n<p>Once the web server is restarted, we'd have https:\/\/wwww. serving the website by default. Congrats!<\/p>\n<h2>Step 4: Renewing certificates<\/h2>\n<p>A Let\u2019s Encrypt certificates is valid for 90 days after being issues, so it needs to be renewed often. The renewal process is pretty simple, all we have to do is run<\/p>\n<div class=\"highlight\"><pre><span><\/span><span class=\"n\">letsencrypt<\/span> <span class=\"n\">renew<\/span> <span class=\"c1\">--pre-hook &quot;service nginx stop&quot; --post-hook &quot;service nginx start&quot;<\/span>\n<\/pre><\/div>\n\n\n<p>This command will renew certificates expiring in less than 30 days. Notice the pre-hook and post-hook, those are the commands we want to be running before and after renewal, which in this case is stopping and restarting Nginx.<\/p>\n<p>For more details on renewing certificates command check out the docs <a href=\"http:\/\/letsencrypt.readthedocs.io\/en\/latest\/using.html#renewing-certificates\">here<\/a><\/p>\n<p>It would definitely be annoying to login to the server constantly to update the certificates. One thing we can do is add a cronjob to do it for us, open up the crontab file<\/p>\n<div class=\"highlight\"><pre><span><\/span><span class=\"n\">crontab<\/span> <span class=\"o\">-<\/span><span class=\"n\">e<\/span>\n<\/pre><\/div>\n\n\n<p>and include the following line<\/p>\n<div class=\"highlight\"><pre><span><\/span><span class=\"mi\">0<\/span> <span class=\"mi\">0<\/span> <span class=\"mi\">1<\/span> <span class=\"o\">*<\/span> <span class=\"o\">*<\/span> <span class=\"n\">letsencrypt<\/span> <span class=\"n\">renew<\/span> <span class=\"c1\">--pre-hook &quot;service nginx stop&quot; --post-hook &quot;service nginx start &gt;&gt; \/var\/log\/letsencrypt.log<\/span>\n<\/pre><\/div>\n\n\n<p>This sets the renewal command to run on the first of every month at 12:00AM, updating any certificates that are expiring in less than 30 days.<\/p>\n<p>After this step, your website should be all set and running on Let's Encrypt.<\/p>\n<p>Delivering more secure web products is crucial for your users. End users are already demanding the availability of secure web pages. As a matter of fact, Google Chrome is going to eventually <a href=\"https:\/\/security.googleblog.com\/2016\/09\/moving-towards-more-secure-web.html\">label all HTTP pages as non-secure<\/a> in very obvious ways for users. <\/p>\n<p>So what are you waiting for? Let's Encrypt!<\/p>","category":[{"@attributes":{"term":"SSL"}},{"@attributes":{"term":"free"}},{"@attributes":{"term":"ubuntu"}},{"@attributes":{"term":"nginx"}},{"@attributes":{"term":"letsencrypt"}},{"@attributes":{"term":"encrypt"}},{"@attributes":{"term":"https"}},{"@attributes":{"term":"security"}}]},{"title":"Bootstrap your vanilla JS game with this project setup and gulp build","link":{"@attributes":{"href":"https:\/\/www.thecodeship.com\/web-development\/bootstrap-vanilla-js-game-gulp-build-project-setup\/","rel":"alternate"}},"published":"2016-08-14T06:00:00+03:00","updated":"2016-08-14T06:00:00+03:00","author":{"name":"Ayman Farhat"},"id":"tag:www.thecodeship.com,2016-08-14:\/web-development\/bootstrap-vanilla-js-game-gulp-build-project-setup\/","summary":"<p>Building your own vanilla HTML5 game without dependencies might sound like a tough and odd job to do. In fact, it's a really fun experience and a huge opportunity to sharpen your JS and general programming skills. One awesome competition that empowers this movement is the JS13K competition. It ...<\/p>","content":"<p>Building your own vanilla HTML5 game without dependencies might sound like a tough and odd job to do. In fact, it's a really fun experience and a huge opportunity to sharpen your JS and general programming skills.<\/p>\n<p>One awesome competition that empowers this movement is the <a href=\"http:\/\/2016.js13kgames.com\/\">JS13K<\/a> competition. It's a 1 month competition where you build an HTML5 game with a 13kb file size limit. While this post is dedicated to this competition, the content applies to building small JS games in general.<\/p>\n<p>As a warm up for the competition, I've decided to prepare a project setup and gulp build, which can be found <a href=\"https:\/\/github.com\/aymanfarhat\/js13k-starter\">here<\/a> with all the setup instructions.<\/p>\n<p>This post will be explaining the project setup, where each piece falls and describe the gulp build process, in the bootstrap project linked above. Assuming you already have some experience with HTML5 games and basic concepts like the game loop.<\/p>\n<h2>Project structure<\/h2>\n<div class=\"highlight\"><pre><span><\/span><span class=\"p\">.<\/span>\n<span class=\"o\">+<\/span><span class=\"c1\">-- README.md<\/span>\n<span class=\"o\">+<\/span><span class=\"c1\">-- _build<\/span>\n<span class=\"o\">?<\/span>   <span class=\"o\">+<\/span><span class=\"c1\">-- game.min..css<\/span>\n<span class=\"o\">?<\/span>   <span class=\"o\">+<\/span><span class=\"c1\">-- game.min.js<\/span>\n<span class=\"o\">?<\/span>   <span class=\"err\">\\<\/span><span class=\"c1\">-- index.html<\/span>\n<span class=\"o\">+<\/span><span class=\"c1\">-- _dist<\/span>\n<span class=\"o\">?<\/span>   <span class=\"err\">\\<\/span><span class=\"c1\">-- game.zip<\/span>\n<span class=\"o\">+<\/span><span class=\"c1\">-- gulpfile.js<\/span>\n<span class=\"o\">+<\/span><span class=\"c1\">-- package.json<\/span>\n<span class=\"err\">\\<\/span><span class=\"c1\">-- src<\/span>\n    <span class=\"o\">+<\/span><span class=\"c1\">-- css<\/span>\n    <span class=\"o\">?<\/span>   <span class=\"err\">\\<\/span><span class=\"c1\">-- style.css<\/span>\n    <span class=\"o\">+<\/span><span class=\"c1\">-- images<\/span>\n    <span class=\"o\">+<\/span><span class=\"c1\">-- index.html<\/span>\n    <span class=\"err\">\\<\/span><span class=\"c1\">-- js<\/span>\n        <span class=\"o\">+<\/span><span class=\"c1\">-- draw.js<\/span>\n        <span class=\"o\">+<\/span><span class=\"c1\">-- game.js<\/span>\n        <span class=\"o\">+<\/span><span class=\"c1\">-- random_obj.js<\/span>\n        <span class=\"err\">\\<\/span><span class=\"c1\">-- util.js<\/span>\n<\/pre><\/div>\n\n\n<h3>At the root<\/h3>\n<p>Here, we have our gulp build file, our npm dependencies in <strong>package.json<\/strong> and two directories <code>_build<\/code> and <code>_dist<\/code>. The job of the gulp build process is to output the final concatenated, minified js, css and html code into _build, and then zip all of the contents and store them under the _dist directory.<\/p>\n<p>This process runs automatically every time the work is saved, which helps keep an eye on the final output especially on the size of the zip file that is being updated in the _dist directory, before finally submitting.<\/p>\n<h3>The source code<\/h3>\n<p>Everything related to the source of the project would go under <code>src<\/code> which would contain our css, images, and js code each in its respective directory.<\/p>\n<h3>Entry point<\/h3>\n<p>Also notice that <code>index.html<\/code> is at the root of the src folder, which is the entry point to our game, in it we'd initialise the canvas and include all dependencies. This would be the non-compiled version which would run on the local dev server and looks something like this:<\/p>\n<div class=\"highlight\"><pre><span><\/span><span class=\"cp\">&lt;!DOCTYPE HTML&gt;<\/span>\n<span class=\"p\">&lt;<\/span><span class=\"nt\">html<\/span> <span class=\"na\">lang<\/span><span class=\"o\">=<\/span><span class=\"s\">&quot;en&quot;<\/span><span class=\"p\">&gt;<\/span>\n    <span class=\"p\">&lt;<\/span><span class=\"nt\">head<\/span><span class=\"p\">&gt;<\/span>\n        <span class=\"p\">&lt;<\/span><span class=\"nt\">title<\/span><span class=\"p\">&gt;<\/span>JS13K Starter<span class=\"p\">&lt;\/<\/span><span class=\"nt\">title<\/span><span class=\"p\">&gt;<\/span>\n        <span class=\"p\">&lt;<\/span><span class=\"nt\">meta<\/span> <span class=\"na\">charset<\/span><span class=\"o\">=<\/span><span class=\"s\">&quot;UTF-8&quot;<\/span><span class=\"p\">&gt;<\/span>\n        <span class=\"p\">&lt;<\/span><span class=\"nt\">meta<\/span> <span class=\"na\">name<\/span><span class=\"o\">=<\/span><span class=\"s\">&quot;viewport&quot;<\/span> <span class=\"na\">content<\/span><span class=\"o\">=<\/span><span class=\"s\">&quot;width=device-width, user-scalable=no, initial-scale=1, maximum-scale=1, user-scalable=0&quot;<\/span> <span class=\"p\">\/&gt;<\/span>\n        <span class=\"p\">&lt;<\/span><span class=\"nt\">meta<\/span> <span class=\"na\">name<\/span><span class=\"o\">=<\/span><span class=\"s\">&quot;apple-mobile-web-app-capable&quot;<\/span> <span class=\"na\">content<\/span><span class=\"o\">=<\/span><span class=\"s\">&quot;yes&quot;<\/span> <span class=\"p\">\/&gt;<\/span>\n        <span class=\"p\">&lt;<\/span><span class=\"nt\">meta<\/span> <span class=\"na\">name<\/span><span class=\"o\">=<\/span><span class=\"s\">&quot;apple-mobile-web-app-status-bar-style&quot;<\/span> <span class=\"na\">content<\/span><span class=\"o\">=<\/span><span class=\"s\">&quot;black-translucent&quot;<\/span> <span class=\"p\">\/&gt;<\/span>\n        <span class=\"c\">&lt;!-- build:css --&gt;<\/span>\n        <span class=\"p\">&lt;<\/span><span class=\"nt\">link<\/span> <span class=\"na\">rel<\/span><span class=\"o\">=<\/span><span class=\"s\">&quot;stylesheet&quot;<\/span> <span class=\"na\">type<\/span><span class=\"o\">=<\/span><span class=\"s\">&quot;text\/css&quot;<\/span> <span class=\"na\">href<\/span><span class=\"o\">=<\/span><span class=\"s\">&quot;.\/css\/style.css&quot;<\/span> <span class=\"p\">\/&gt;<\/span>\n        <span class=\"c\">&lt;!-- endbuild --&gt;<\/span>\n    <span class=\"p\">&lt;\/<\/span><span class=\"nt\">head<\/span><span class=\"p\">&gt;<\/span>\n    <span class=\"p\">&lt;<\/span><span class=\"nt\">body<\/span><span class=\"p\">&gt;<\/span>\n        <span class=\"p\">&lt;<\/span><span class=\"nt\">h1<\/span><span class=\"p\">&gt;<\/span>JS13K Starter Project<span class=\"p\">&lt;\/<\/span><span class=\"nt\">h1<\/span><span class=\"p\">&gt;<\/span>\n        <span class=\"p\">&lt;<\/span><span class=\"nt\">h3<\/span><span class=\"p\">&gt;<\/span>Click to randomize!<span class=\"p\">&lt;\/<\/span><span class=\"nt\">h3<\/span><span class=\"p\">&gt;<\/span>\n        <span class=\"p\">&lt;<\/span><span class=\"nt\">canvas<\/span><span class=\"p\">&gt;&lt;\/<\/span><span class=\"nt\">canvas<\/span><span class=\"p\">&gt;<\/span>\n\n        <span class=\"c\">&lt;!-- build:js --&gt;<\/span>\n        <span class=\"p\">&lt;<\/span><span class=\"nt\">script<\/span> <span class=\"na\">src<\/span><span class=\"o\">=<\/span><span class=\"s\">&quot;.\/js\/game.js&quot;<\/span><span class=\"p\">&gt;&lt;\/<\/span><span class=\"nt\">script<\/span><span class=\"p\">&gt;<\/span>\n        <span class=\"p\">&lt;<\/span><span class=\"nt\">script<\/span> <span class=\"na\">src<\/span><span class=\"o\">=<\/span><span class=\"s\">&quot;.\/js\/draw.js&quot;<\/span><span class=\"p\">&gt;&lt;\/<\/span><span class=\"nt\">script<\/span><span class=\"p\">&gt;<\/span>\n        <span class=\"p\">&lt;<\/span><span class=\"nt\">script<\/span> <span class=\"na\">src<\/span><span class=\"o\">=<\/span><span class=\"s\">&quot;.\/js\/util.js&quot;<\/span><span class=\"p\">&gt;&lt;\/<\/span><span class=\"nt\">script<\/span><span class=\"p\">&gt;<\/span>\n        <span class=\"p\">&lt;<\/span><span class=\"nt\">script<\/span> <span class=\"na\">src<\/span><span class=\"o\">=<\/span><span class=\"s\">&quot;.\/js\/random_obj.js&quot;<\/span><span class=\"p\">&gt;&lt;\/<\/span><span class=\"nt\">script<\/span><span class=\"p\">&gt;<\/span>\n        <span class=\"c\">&lt;!-- endbuild --&gt;<\/span>\n    <span class=\"p\">&lt;\/<\/span><span class=\"nt\">body<\/span><span class=\"p\">&gt;<\/span>\n<span class=\"p\">&lt;\/<\/span><span class=\"nt\">html<\/span><span class=\"p\">&gt;<\/span>\n<\/pre><\/div>\n\n\n<h3>JS Code<\/h3>\n<p>As for the JS, one easy way to structure is it follow a couple of simple principles:<\/p>\n<p>Wrap your game code around a namespace\nSeparate code into different files based on their functionality\nA simple way to namespace your game is to have a global object for the game, I named it $ in this case as its short and jQuery is not being used in the project. Under that name space I'd attach different objects and functions, used across different files.<\/p>\n<p>For example, we can have reusable functions around drawing on canvas available under the <code>$.Draw<\/code> module inside the draw.js file which contains stuff like <code>$.Draw.rect<\/code>, <code>$.Draw.circle<\/code> or anything related to drawing on the screen.<\/p>\n<p>As for the logic of different characters or entities related to the game such as a meteor, hero, or a villain, those would have their own files and manage their own logic relative to the game and its loop. Those modules would have a class like structure with a constructor. That's because we want to be creating several instances of our game objects into the game such as several meteors, one ups, etc...<\/p>\n<p>The code for such a module would look something like this:<\/p>\n<div class=\"highlight\"><pre><span><\/span><span class=\"s2\">&quot;use strict&quot;<\/span><span class=\"p\">;<\/span>\n\n<span class=\"nx\">$<\/span><span class=\"p\">.<\/span><span class=\"nx\">RandomObj<\/span> <span class=\"o\">=<\/span> <span class=\"kd\">function<\/span> <span class=\"p\">()<\/span> <span class=\"p\">{<\/span>\n    <span class=\"k\">this<\/span><span class=\"p\">.<\/span><span class=\"nx\">x<\/span> <span class=\"o\">=<\/span> <span class=\"nx\">$<\/span><span class=\"p\">.<\/span><span class=\"nx\">util<\/span><span class=\"p\">.<\/span><span class=\"nx\">randomInRange<\/span><span class=\"p\">(<\/span><span class=\"mi\">0<\/span><span class=\"p\">,<\/span> <span class=\"nx\">$<\/span><span class=\"p\">.<\/span><span class=\"nx\">width<\/span><span class=\"p\">);<\/span>\n    <span class=\"k\">this<\/span><span class=\"p\">.<\/span><span class=\"nx\">y<\/span> <span class=\"o\">=<\/span> <span class=\"nx\">$<\/span><span class=\"p\">.<\/span><span class=\"nx\">util<\/span><span class=\"p\">.<\/span><span class=\"nx\">randomInRange<\/span><span class=\"p\">(<\/span><span class=\"mi\">0<\/span><span class=\"p\">,<\/span> <span class=\"nx\">$<\/span><span class=\"p\">.<\/span><span class=\"nx\">height<\/span><span class=\"p\">);<\/span>\n    <span class=\"k\">this<\/span><span class=\"p\">.<\/span><span class=\"nx\">dimension<\/span> <span class=\"o\">=<\/span> <span class=\"mi\">0<\/span><span class=\"p\">;<\/span>\n    <span class=\"k\">this<\/span><span class=\"p\">.<\/span><span class=\"nx\">targetDimension<\/span> <span class=\"o\">=<\/span> <span class=\"nx\">$<\/span><span class=\"p\">.<\/span><span class=\"nx\">util<\/span><span class=\"p\">.<\/span><span class=\"nx\">randomInRange<\/span><span class=\"p\">(<\/span><span class=\"mi\">50<\/span><span class=\"p\">,<\/span> <span class=\"mi\">70<\/span><span class=\"p\">);<\/span>\n    <span class=\"k\">this<\/span><span class=\"p\">.<\/span><span class=\"nx\">growthSpeed<\/span> <span class=\"o\">=<\/span> <span class=\"nx\">$<\/span><span class=\"p\">.<\/span><span class=\"nx\">util<\/span><span class=\"p\">.<\/span><span class=\"nx\">randomInRange<\/span><span class=\"p\">(<\/span><span class=\"mf\">0.5<\/span><span class=\"p\">,<\/span> <span class=\"mi\">2<\/span><span class=\"p\">);<\/span>\n    <span class=\"k\">this<\/span><span class=\"p\">.<\/span><span class=\"nx\">color<\/span> <span class=\"o\">=<\/span> <span class=\"nx\">$<\/span><span class=\"p\">.<\/span><span class=\"nx\">util<\/span><span class=\"p\">.<\/span><span class=\"nx\">pickRandomFromObject<\/span><span class=\"p\">(<\/span><span class=\"nx\">$<\/span><span class=\"p\">.<\/span><span class=\"nx\">colors<\/span><span class=\"p\">);<\/span>\n<span class=\"p\">};<\/span>\n\n<span class=\"nx\">$<\/span><span class=\"p\">.<\/span><span class=\"nx\">RandomObj<\/span><span class=\"p\">.<\/span><span class=\"nx\">prototype<\/span><span class=\"p\">.<\/span><span class=\"nx\">render<\/span> <span class=\"o\">=<\/span> <span class=\"kd\">function<\/span> <span class=\"p\">()<\/span> <span class=\"p\">{<\/span>\n    <span class=\"nx\">$<\/span><span class=\"p\">.<\/span><span class=\"nx\">Draw<\/span><span class=\"p\">.<\/span><span class=\"nx\">rect<\/span><span class=\"p\">(<\/span><span class=\"k\">this<\/span><span class=\"p\">.<\/span><span class=\"nx\">x<\/span><span class=\"p\">,<\/span> <span class=\"k\">this<\/span><span class=\"p\">.<\/span><span class=\"nx\">y<\/span><span class=\"p\">,<\/span> <span class=\"k\">this<\/span><span class=\"p\">.<\/span><span class=\"nx\">dimension<\/span><span class=\"p\">,<\/span> <span class=\"k\">this<\/span><span class=\"p\">.<\/span><span class=\"nx\">dimension<\/span><span class=\"p\">,<\/span> <span class=\"k\">this<\/span><span class=\"p\">.<\/span><span class=\"nx\">color<\/span><span class=\"p\">);<\/span>\n<span class=\"p\">};<\/span>\n\n<span class=\"nx\">$<\/span><span class=\"p\">.<\/span><span class=\"nx\">RandomObj<\/span><span class=\"p\">.<\/span><span class=\"nx\">prototype<\/span><span class=\"p\">.<\/span><span class=\"nx\">update<\/span> <span class=\"o\">=<\/span> <span class=\"kd\">function<\/span> <span class=\"p\">()<\/span> <span class=\"p\">{<\/span>\n    <span class=\"k\">if<\/span> <span class=\"p\">(<\/span><span class=\"k\">this<\/span><span class=\"p\">.<\/span><span class=\"nx\">dimension<\/span> <span class=\"o\">&lt;<\/span> <span class=\"k\">this<\/span><span class=\"p\">.<\/span><span class=\"nx\">targetDimension<\/span><span class=\"p\">)<\/span> <span class=\"p\">{<\/span>\n        <span class=\"k\">this<\/span><span class=\"p\">.<\/span><span class=\"nx\">dimension<\/span> <span class=\"o\">+=<\/span> <span class=\"k\">this<\/span><span class=\"p\">.<\/span><span class=\"nx\">growthSpeed<\/span><span class=\"p\">;<\/span>\n    <span class=\"p\">}<\/span>\n<span class=\"p\">};<\/span>\n<\/pre><\/div>\n\n\n<p>Creating an instance of our RandomObj would be something like <\/p>\n<div class=\"highlight\"><pre><span><\/span><span class=\"kd\">var<\/span> <span class=\"nx\">randomObjectInstance<\/span> <span class=\"o\">=<\/span> <span class=\"k\">new<\/span> <span class=\"nx\">$<\/span><span class=\"p\">.<\/span><span class=\"nx\">RandomObj<\/span><span class=\"p\">();<\/span>\n<\/pre><\/div>\n\n\n<p>which ideally would be instantiated from within our game loop. Each game object class would implement two essential functions, <strong>render<\/strong> and <strong>update<\/strong>.<\/p>\n<p>Each of those functions would be called automatically by our game loop on each iteration.<\/p>\n<p>Calling <code>randomObjectInstance.render()<\/code> would trigger the painting code for that specific instance of that specific game object.<\/p>\n<p>Calling <code>randomObjectInstance.update()<\/code> would trigger the change to be done on that instance's state such as growing in size + 1 when that game loop tick takes place.<\/p>\n<p>The game loop itself would be as simple as an infinite loop that triggers a render and update calls on each iteration on all the game objects available in the world. Example:<\/p>\n<div class=\"highlight\"><pre><span><\/span><span class=\"nx\">$<\/span><span class=\"p\">.<\/span><span class=\"nx\">loop<\/span> <span class=\"o\">=<\/span> <span class=\"kd\">function<\/span> <span class=\"p\">()<\/span> <span class=\"p\">{<\/span>\n    <span class=\"nx\">$<\/span><span class=\"p\">.<\/span><span class=\"nx\">render<\/span><span class=\"p\">();<\/span>\n    <span class=\"nx\">$<\/span><span class=\"p\">.<\/span><span class=\"nx\">update<\/span><span class=\"p\">();<\/span>\n\n    <span class=\"nb\">window<\/span><span class=\"p\">.<\/span><span class=\"nx\">requestAnimFrame<\/span><span class=\"p\">(<\/span><span class=\"nx\">$<\/span><span class=\"p\">.<\/span><span class=\"nx\">loop<\/span><span class=\"p\">);<\/span>\n<span class=\"p\">};<\/span>\n\n<span class=\"nx\">$<\/span><span class=\"p\">.<\/span><span class=\"nx\">update<\/span> <span class=\"o\">=<\/span> <span class=\"kd\">function<\/span> <span class=\"p\">()<\/span> <span class=\"p\">{<\/span>\n    <span class=\"k\">for<\/span> <span class=\"p\">(<\/span><span class=\"kd\">var<\/span> <span class=\"nx\">i<\/span> <span class=\"o\">=<\/span> <span class=\"mi\">0<\/span><span class=\"p\">;<\/span> <span class=\"nx\">i<\/span> <span class=\"o\">&lt;<\/span> <span class=\"nx\">$<\/span><span class=\"p\">.<\/span><span class=\"nx\">entities<\/span><span class=\"p\">.<\/span><span class=\"nx\">length<\/span><span class=\"p\">;<\/span> <span class=\"nx\">i<\/span><span class=\"o\">++<\/span><span class=\"p\">)<\/span> <span class=\"p\">{<\/span>\n        <span class=\"nx\">$<\/span><span class=\"p\">.<\/span><span class=\"nx\">entities<\/span><span class=\"p\">[<\/span><span class=\"nx\">i<\/span><span class=\"p\">].<\/span><span class=\"nx\">update<\/span><span class=\"p\">();<\/span>    \n    <span class=\"p\">}<\/span>\n<span class=\"p\">};<\/span>\n\n<span class=\"nx\">$<\/span><span class=\"p\">.<\/span><span class=\"nx\">render<\/span> <span class=\"o\">=<\/span> <span class=\"kd\">function<\/span> <span class=\"p\">()<\/span> <span class=\"p\">{<\/span>\n    <span class=\"nx\">$<\/span><span class=\"p\">.<\/span><span class=\"nx\">Draw<\/span><span class=\"p\">.<\/span><span class=\"nx\">clear<\/span><span class=\"p\">();<\/span>\n\n    <span class=\"k\">for<\/span> <span class=\"p\">(<\/span><span class=\"kd\">var<\/span> <span class=\"nx\">i<\/span> <span class=\"o\">=<\/span> <span class=\"mi\">0<\/span><span class=\"p\">;<\/span> <span class=\"nx\">i<\/span> <span class=\"o\">&lt;<\/span> <span class=\"nx\">$<\/span><span class=\"p\">.<\/span><span class=\"nx\">entities<\/span><span class=\"p\">.<\/span><span class=\"nx\">length<\/span><span class=\"p\">;<\/span> <span class=\"nx\">i<\/span><span class=\"o\">++<\/span><span class=\"p\">)<\/span> <span class=\"p\">{<\/span>\n        <span class=\"nx\">$<\/span><span class=\"p\">.<\/span><span class=\"nx\">entities<\/span><span class=\"p\">[<\/span><span class=\"nx\">i<\/span><span class=\"p\">].<\/span><span class=\"nx\">render<\/span><span class=\"p\">();<\/span>    \n    <span class=\"p\">}<\/span>\n<span class=\"p\">};<\/span>\n<\/pre><\/div>\n\n\n<p>The game loop is available in the <a href=\"https:\/\/github.com\/aymanfarhat\/js13k-starter\/blob\/master\/src\/js\/game.js\">game.js<\/a> file in the project.<\/p>\n<h2>Gulp build<\/h2>\n<p>The <a href=\"https:\/\/github.com\/aymanfarhat\/js13k-starter\/blob\/master\/gulpfile.js\">build script<\/a> is pretty straightforward, as mentioned earlier, its all about concatenating, minifying the source then outputting a final zip file. They're all divided to separate tasks <strong>serve<\/strong>, <strong>buildCSS<\/strong>, <strong>buildJS<\/strong>, <strong>buildIndex<\/strong>, and <strong>zipBuild<\/strong>. And finally, they're all grouped into one main <strong>build<\/strong> command which runs them in order.<\/p>\n<p>This runs every time changes are saved on the project thanks to the <strong>watch<\/strong> command. In addition, the game's entry point index.html via a serve command.<\/p>\n<p>To get everything up and running in one shot, just run gulp this will run an initial build, serve the app on the localhost and will run the watch command which will check for any changes and trigger a new build and zip.<\/p>\n<p>After each save, the gulp build will output the current size of the zipped project in order to stay alert on the size of the project. Something like <\/p>\n<div class=\"highlight\"><pre><span><\/span><span class=\"p\">[<\/span><span class=\"mi\">09<\/span><span class=\"p\">:<\/span><span class=\"mi\">32<\/span><span class=\"p\">:<\/span><span class=\"mi\">55<\/span><span class=\"p\">]<\/span> <span class=\"k\">Size<\/span> <span class=\"k\">of<\/span> <span class=\"n\">game<\/span><span class=\"p\">.<\/span><span class=\"n\">zip<\/span><span class=\"p\">:<\/span> <span class=\"mi\">2<\/span><span class=\"p\">.<\/span><span class=\"mi\">75<\/span> <span class=\"n\">KB<\/span>\n<\/pre><\/div>\n\n\n<p>You can find the starter project example on <a href=\"https:\/\/github.com\/aymanfarhat\/js13k-starter\">this link<\/a>.<\/p>\n<p>I hope you find this JS13K starter useful and I'm looking forward to participating and seeing all the awesome submissions this year. Would love to hear your feedback and suggestions on this post.<\/p>","category":[{"@attributes":{"term":"gulp"}},{"@attributes":{"term":"canvas"}},{"@attributes":{"term":"project"}},{"@attributes":{"term":"build"}},{"@attributes":{"term":"js13k"}},{"@attributes":{"term":"structure"}},{"@attributes":{"term":"html5"}}]},{"title":"Sudden S3 Boto authorisation error on server? Might need to sync up the time!","link":{"@attributes":{"href":"https:\/\/www.thecodeship.com\/gnu-linux\/sudden-s3-boto-auth-error-sync-up\/","rel":"alternate"}},"published":"2016-05-22T15:03:00+03:00","updated":"2016-05-22T15:03:00+03:00","author":{"name":"Ayman Farhat"},"id":"tag:www.thecodeship.com,2016-05-22:\/gnu-linux\/sudden-s3-boto-auth-error-sync-up\/","summary":"<p>I usually have a couple cron jobs scheduled, executing backup scripts between EC2 instances and store the data on S3 via Python's Boto library. Everything was running smoothly for several months until one day, the script was failing to authenticate with a 403 error from AWS S3, on one of the EC2 instances...<\/p>","content":"<p>I usually have a couple cron jobs scheduled, executing backup scripts between EC2 instances and store the data on S3 via Python's <a href=\"https:\/\/github.com\/boto\/boto\">Boto<\/a> library. Everything was running smoothly for several months until one day, the script was failing to authenticate with a 403 error from AWS S3, on a specific instance.<\/p>\n<p>Ports are open, credentials are okay and nothing has changed. What could possibly go wrong?<\/p>\n<p>Many hours of searching for possible sources for the problem ended. The last thing on my mind was a possible issue with time settings. After looking into that a bit more, it turned out that a recent update on <a href=\"https:\/\/github.com\/boto\/boto\">Boto<\/a>'s authenticate which made EC2 connections use hmac-v4 by default. Update referenced in <a href=\"https:\/\/github.com\/boto\/boto\/commit\/32abe3e99961069912b5004e1507d40e785300ae\">this commit<\/a>.<\/p>\n<p>The hmac-v4 method is more sensitive to clock <a href=\"https:\/\/en.wikipedia.org\/wiki\/Clock_skew#On_a_network\">skew<\/a>. If an instance's clock differs from the internet standard time servers by more than a couple minutes the authorisation will be rejected.<\/p>\n<p>All you really have to do is sync up the time on your server\nAll you have to do is setup the <a href=\"http:\/\/doc.ntp.org\/4.1.0\/ntpd.htm\">ntp daemon<\/a> on the server. It sets and maintains the system time of day in synchronism with Internet standard time servers.<\/p>\n<p>If you're on Ubuntu, you can just run these 2 commands and everything should be in place.<\/p>\n<p><code>sudo apt-get install ntp<\/code><\/p>\n<p><code>sudo ntp -u ntp:ntp<\/code><\/p>\n<p>It's amazing how some simple issues, get so much frustrating and time consuming to solve, mostly due to overlooking some details that can be taken for granted. Hope you found this post helpful!<\/p>","category":[{"@attributes":{"term":"s3"}},{"@attributes":{"term":"ntp"}},{"@attributes":{"term":"sync"}},{"@attributes":{"term":"boto"}},{"@attributes":{"term":"api"}},{"@attributes":{"term":"time"}}]},{"title":"A year of Vim - Beginner advice and lessons learned","link":{"@attributes":{"href":"https:\/\/www.thecodeship.com\/tools\/year-of-vim-beginner-advice-lessons-learned\/","rel":"alternate"}},"published":"2015-05-30T22:13:00+03:00","updated":"2015-05-30T22:13:00+03:00","author":{"name":"Ayman Farhat"},"id":"tag:www.thecodeship.com,2015-05-30:\/tools\/year-of-vim-beginner-advice-lessons-learned\/","summary":"<p>I've been using vim daily for almost a year now; it has been a wonderful (sometimes frustrating) experience. Today, a friend of mine was informing me about his interest in jumping into Vim. I thought I'd give him some advice from what I have learned over this period ...<\/p>","content":"<p>I've been using vim daily for almost a year now; it has been a wonderful (sometimes frustrating) experience. Today, a friend of mine was informing me about his interest in jumping into Vim. I thought I'd give him some advice from what I have learned over this period of time as a Vim beginner.<\/p>\n<h2>Why Vim?<\/h2>\n<p>If you're reading this, I assume that you're already sold on Vim and determined to get into it. Compared to other editors, Vim is hard to learn and use properly. It's easy to fall into using Vim just like you do with conventional editors like Sublime, and this might lead you to miss all of Vim's power and slow down your work.<\/p>\n<p>Here is a summary of the most important practices to adopt in order to improve your skills and reach the other side of the river.<\/p>\n<h2>Embrace the Vim mindset<\/h2>\n<p>Vim is tricky to learn and use properly compared to other common editors, and you shouldn't be using it like a common one. Since it provides a totally different experience, embracing Vim's principles is crucial in order for you to move forward.<\/p>\n<ul>\n<li>Understand and use Vim's modal editing system instead of doing everything while in insert mode<\/li>\n<li>Learn to move in Vim without using the arrow keys<\/li>\n<li>Understand command combinations and use them to speed up operations<\/li>\n<li>Learn to use Vim's search and replace features<\/li>\n<li>Leverage macros to save time and effort on repetitive patterns<\/li>\n<\/ul>\n<h2>Connect yourself with experienced users<\/h2>\n<p>Since there are different ways to do things, it is easy to adopt bad habits in Vim. One of the best ways to discover better practices is to connect with more experience Vim users. Watch them how they use the editor, ask them questions, pair program on Vim and let them help you out in improving your experience.<\/p>\n<p>As a matter of fact, most of the tips in this post are direct advice from my good friend (and Vim ninja) <a href=\"http:\/\/aboumrad.info\/\">Omar Abou Mrad<\/a>, who helped me a lot in improving my skills.<\/p>\n<p>You could also get some amazing help from the folks at <strong>#vim<\/strong> on Freenode.<\/p>\n<h2>Dropping the arrow keys<\/h2>\n<p>One of the important things to do early on, is to disable your arrow keys on all modes and not rely on them for navigation. This might sound like an odd start for beginners, but here is why I think you should:<\/p>\n<ul>\n<li>\n<p>The letter based navigation keys allow you to navigate the text without taking your hands away from standard typing configuration. This is more efficient and faster than moving your hand to hit the arrow keys every time you want to navigate text.<\/p>\n<\/li>\n<li>\n<p>In normal mode, Vim provides numerous efficient shortcuts for moving around beyond HJKL, such as by space, words, paragraphs, line search etc...<\/p>\n<\/li>\n<li>\n<p>You might say, \"What about navigating in insert mode?\" Well you shouldn't be moving around in insert mode, this mode is meant only for entering text.<\/p>\n<\/li>\n<\/ul>\n<p>So do yourself a favour and disable the arrow keys, by pasting this snippet into your vimrc:<\/p>\n<div class=\"highlight\"><pre><span><\/span><span class=\"n\">noremap<\/span>  <span class=\"o\">&lt;<\/span><span class=\"n\">Up<\/span><span class=\"o\">&gt;<\/span> <span class=\"ss\">&quot;&quot;<\/span>\n<span class=\"n\">noremap<\/span><span class=\"o\">!<\/span> <span class=\"o\">&lt;<\/span><span class=\"n\">Up<\/span><span class=\"o\">&gt;<\/span> <span class=\"o\">&lt;<\/span><span class=\"n\">Esc<\/span><span class=\"o\">&gt;<\/span>\n<span class=\"n\">noremap<\/span>  <span class=\"o\">&lt;<\/span><span class=\"n\">Down<\/span><span class=\"o\">&gt;<\/span> <span class=\"ss\">&quot;&quot;<\/span>\n<span class=\"n\">noremap<\/span><span class=\"o\">!<\/span> <span class=\"o\">&lt;<\/span><span class=\"n\">Down<\/span><span class=\"o\">&gt;<\/span> <span class=\"o\">&lt;<\/span><span class=\"n\">Esc<\/span><span class=\"o\">&gt;<\/span>\n<span class=\"n\">noremap<\/span>  <span class=\"o\">&lt;<\/span><span class=\"k\">Left<\/span><span class=\"o\">&gt;<\/span> <span class=\"ss\">&quot;&quot;<\/span>\n<span class=\"n\">noremap<\/span><span class=\"o\">!<\/span> <span class=\"o\">&lt;<\/span><span class=\"k\">Left<\/span><span class=\"o\">&gt;<\/span> <span class=\"o\">&lt;<\/span><span class=\"n\">Esc<\/span><span class=\"o\">&gt;<\/span>\n<span class=\"n\">noremap<\/span>  <span class=\"o\">&lt;<\/span><span class=\"k\">Right<\/span><span class=\"o\">&gt;<\/span> <span class=\"ss\">&quot;&quot;<\/span>\n<span class=\"n\">noremap<\/span><span class=\"o\">!<\/span> <span class=\"o\">&lt;<\/span><span class=\"k\">Right<\/span><span class=\"o\">&gt;<\/span> <span class=\"o\">&lt;<\/span><span class=\"n\">Esc<\/span><span class=\"o\">&gt;<\/span>\n<\/pre><\/div>\n\n\n<p>While it is crucial for a beginner to get rid of the arrow key habit, eventually you can configure and bind the arrow keys to serve other purposes that you might find useful to add.<\/p>\n<h2>Tabs is just another way to open files, and not your most efficient option<\/h2>\n<p>When I started using Vim, I came from a Sublime Text background. In the Sublime world, tabs are heavily used for opening and storing documents in the editor.<\/p>\n<p>When using Vim at first, one might expect that tabs would behave just like all the other editors or browser out there. Unfortunately, that is not the case with Vim.<\/p>\n<p>In Vim, tabs are not meant to be used as containers of documents and to navigate between them back and forth. Dealing with tabs on this premise will slow down your work.<\/p>\n<p>Instead, you should be looking at how to work with buffers in order to load files and navigate between them. Here is an <a href=\"https:\/\/joshldavis.com\/2014\/04\/05\/vim-tab-madness-buffers-vs-tabs\/\">excellent article<\/a> that highlights exactly this point and explains the use case of Tab, Window, and Buffer in Vim.<\/p>\n<h2>Practice is key<\/h2>\n<p>Speed in Vim depends a lot on muscle memory, and in order to develop that, you have to be using Vim constantly. The hard and preferred way is going into it head first and integrating it into your daily workflow as soon as you can.<\/p>\n<p>Getting out of your Sublime comfort zone is painful in the beginning but you have to keep going through, and as you learn more, the experience starts to feel much more sweet.<\/p>\n<p>When it comes to learning, one great resource to watch and learn about new Vim keys, tips and tricks is <a href=\"http:\/\/vimcasts.org\/\">Vimcasts<\/a>.<\/p>\n<p>In addition to that, constantly practicing through exercises will give your learning experience a huge boost. One highly recommended training resource is <a href=\"http:\/\/www.vimgolf.com\/\">Vim Golf<\/a>, make sure you check it out.<\/p>\n<h2>Ignore pre-loaded packages and fancy extensions<\/h2>\n<p>It is very important to understand Vim on a raw level before jumping into installing packages, add-ons and extensions. Installing bundles of extensions and pre-made configs early on, will pollute your Vim experience and distract you from focusing on Vim's core features.<\/p>\n<p>Try fighting the urge to replicate functionality in Vim that you're already used to in other editors. Vim is NOT these editors.<\/p>\n<h2>Slowly craft your own Vimrc<\/h2>\n<p>It is highly recommended that you craft your own .vimrc and know exactly what you're adding to it. Pasting large chunks of config code into it will eventually harm your Vim experience. In addition to that, understanding the basics of VimL will also help you know your Vimrc better and customise it as needed.<\/p>\n<h2>Keep an eye on repetitive patterns<\/h2>\n<p>Every time you find tedious editing tasks, look for a way to make them less tedious or repetitive. Always assume that there's a better way that can save you time.<\/p>\n<h2>Useful extensions to consider<\/h2>\n<p>Here is a list of recommended basic extensions, and that you might find useful to look into for your Vim journey:<\/p>\n<ul>\n<li>\n<p><a href=\"https:\/\/github.com\/kien\/ctrlp.vim\">Ctrlp<\/a>: Hitting Ctrl+P will allow you to use fuzzy search to find your files<\/p>\n<\/li>\n<li>\n<p><a href=\"https:\/\/github.com\/garbas\/vim-snipmate\">Snipmate<\/a>: Allows storing snippets and using them on the fly in your files<\/p>\n<\/li>\n<li>\n<p><a href=\"http:\/\/mattn.github.io\/emmet-vim\/\">Emmet<\/a>: Provides support for expanding abbreviations similar to <a href=\"http:\/\/emmet.io\/\">emmet<\/a><\/p>\n<\/li>\n<\/ul>\n<p>That was a summary of what I have learned so far about Vim, hopefully its clear enough for beginners to checkout out and learn a thing or two from it.<\/p>\n<p>Whether you are a beginner or an experienced Vim user, I would love to hear your feedback on this piece, make sure to share your thoughts in the comments below. Happy Vimming!<\/p>","category":[{"@attributes":{"term":"beginner"}},{"@attributes":{"term":"experience"}},{"@attributes":{"term":"vim"}},{"@attributes":{"term":"tips"}},{"@attributes":{"term":"advice"}}]},{"title":"The Hackathon Toolbox - Essential Tools and Practices","link":{"@attributes":{"href":"https:\/\/www.thecodeship.com\/general\/hackathon-toolbox-essential-tools-practices\/","rel":"alternate"}},"published":"2015-04-07T20:04:00+03:00","updated":"2015-04-07T20:04:00+03:00","author":{"name":"Ayman Farhat"},"id":"tag:www.thecodeship.com,2015-04-07:\/general\/hackathon-toolbox-essential-tools-practices\/","summary":"<p>In a couple of days I will be mentoring a team of students at the <a href=\"http:\/\/nyuad.nyu.edu\/en\/news-events\/conferences\/nyuad-hackathon.html\">NYUAD 2015 hackathon - Building Apps for Social Good in the Arab World<\/a>. Since I tend to participate often in development competitions and hackathons, I thought I'd compile a comprehensive list of essential tools needed ...<\/p>","content":"<p>In a couple of days I will be mentoring a team of students at the <a href=\"http:\/\/nyuad.nyu.edu\/en\/news-events\/conferences\/nyuad-hackathon.html\">NYUAD 2015 hackathon - Building Apps for Social Good in the Arab World<\/a>. Since I tend to participate often in development competitions and hackathons, I thought I'd compile a comprehensive list of essential tools needed for any Hackathon as a reference to get back to and share with team mates and friends.<\/p>\n<p>Being effective at a hackathon is all about efficiency. In this article I focus on different tools, tips and practices that will help your team be as efficient as possible in churning out a gorgeous product MVP in 3 or 4 days.<\/p>\n<p>Since a hackathon is not just about writing code, tools related to ideation, design, time management, resource sharing, and presentation are also discussed in detail.<\/p>\n<h2>Wireframes<\/h2>\n<p>Drawing quick wireframes for describing your UI and UX, is essential during the ideation phase. Prototyping your UI with such a tool will give you more clarity on the product and how it can look like.<\/p>\n<p>Furthermore, a good wireframe can guide your developers through the hackathon in implementing the desired UX while focusing on implementation. Here are some popular tools that can help:<\/p>\n<ul>\n<li><a href=\"https:\/\/moqups.com\/\">Moqups<\/a><\/li>\n<li><a href=\"http:\/\/www.balsamiq.com\/\">Basalmiq<\/a><\/li>\n<li><a href=\"http:\/\/www.invisionapp.com\/\">Invision<\/a><\/li>\n<li><a href=\"http:\/\/www.axure.com\/\">Axure<\/a><\/li>\n<li><a href=\"https:\/\/www.omnigroup.com\/omnigraffle\/\">OmniGraffle<\/a><\/li>\n<\/ul>\n<h2>File sharing<\/h2>\n<p>You will definitely run into the need to share files between your team members. Pick one good file sharing platform and make sure to have it set up prior to the hacking period. Ideally, do it at the end of the ideation day before the hacking starts.<\/p>\n<ul>\n<li><a href=\"https:\/\/www.dropbox.com\/\">Dropbox<\/a><\/li>\n<li><a href=\"https:\/\/www.google.com\/drive\/\">Google Drive<\/a><\/li>\n<\/ul>\n<h2>Project management<\/h2>\n<p>Clarity is power, the more you know exactly what you want to do, the easier it is to execute. Organizing your team's MVP goals into a simple and clear project plan can make a lot of difference.<\/p>\n<p>You can use a virtual <a href=\"http:\/\/en.wikipedia.org\/wiki\/Scrum_%28software_development%29\">Scrum board<\/a> for organizing, prioritizing and assigning tasks to the team. Here are some good tools to help in that:<\/p>\n<ul>\n<li><a href=\"https:\/\/trello.com\/\">Trello<\/a><\/li>\n<li><a href=\"https:\/\/plan.io\/sem\/en\/scrum-dashboard\/\">Plan.io<\/a><\/li>\n<li><a href=\"https:\/\/kanbanize.com\/\">Kanbanize<\/a><\/li>\n<li><a href=\"https:\/\/www.atlassian.com\/software\/jira\">Jira<\/a><\/li>\n<\/ul>\n<p>If you prefer a simple todo list approach, you can check <a href=\"https:\/\/asana.com\/\">Asana<\/a> or <a href=\"https:\/\/www.wunderlist.com\/\">Wunderlist<\/a><\/p>\n<h2>Image editing<\/h2>\n<p>You will probably need to edit images much more than you think. Tweaking that small icon or enhancing the logo at the last minute? Make sure to have a good image editor setup before the hacking starts.<\/p>\n<ul>\n<li><a href=\"http:\/\/www.adobe.com\/products\/photoshop.html\">Photoshop<\/a><\/li>\n<li><a href=\"http:\/\/www.gimp.org\/\">Gimp<\/a><\/li>\n<li><a href=\"http:\/\/www.adobe.com\/products\/illustrator.html\">Illustrator<\/a><\/li>\n<li><a href=\"https:\/\/inkscape.org\/en\/\">Inkscape<\/a><\/li>\n<li><a href=\"http:\/\/bohemiancoding.com\/sketch\/\">Sketch<\/a><\/li>\n<li><a href=\"http:\/\/www.pixelmator.com\/mac\/\">Pixelmator<\/a><\/li>\n<\/ul>\n<h2>Design<\/h2>\n<p>In the context of a hackathon, you can't afford spending time on polishing a static photoshop design and then build the UI. Design has to be part of the UI development process, so make sure your team and designer is comfortable with that. Teams with designers who know how to code, have a big advantage here.<\/p>\n<p>Here are some resources that can help in coming up with a decent design implementation:<\/p>\n<ul>\n<li><a href=\"http:\/\/webdesign.tutsplus.com\/articles\/9-quick-wins-for-halfway-decent-design--cms-19444\">9 Quick Wins for a Halfway Decent Design<\/a><\/li>\n<li><a href=\"http:\/\/bootsnipp.com\/\">Bootstrap Zero<\/a> - Free Bootstrap templates themes<\/li>\n<li><a href=\"http:\/\/html5up.net\/\">HTML5 UP<\/a><\/li>\n<li><a href=\"http:\/\/uxarchive.com\/\">UX Archive patterns<\/a> - UX and UI inspiration from top apps.<\/li>\n<\/ul>\n<h3>Fonts<\/h3>\n<ul>\n<li><a href=\"http:\/\/www.google.com\/fonts\">Google fonts<\/a><\/li>\n<li><a href=\"http:\/\/fontface.me\/\">Fontface.me<\/a> - Collection of gorgeous Arabic fonts available for free<\/li>\n<li><a href=\"http:\/\/fortawesome.github.io\/Font-Awesome\/icons\/\">Font Awesome Icons<\/a> - Collection of easy to use SVG icons available as CSS classes<\/li>\n<li><a href=\"http:\/\/fontpair.co\/\">Font Pair<\/a> - Helps you in pairing google fonts together<\/li>\n<\/ul>\n<h3>Backgrounds<\/h3>\n<ul>\n<li><a href=\"https:\/\/github.com\/btmills\/geopattern\">Geopattern<\/a> - Generate beautiful SVG patterns for backgrounds<\/li>\n<li><a href=\"https:\/\/github.com\/qrohlf\/trianglify\">Trianglify<\/a> - Javascript library for generating colorful triangle meshes that can be used as SVG images and CSS backgrounds<\/li>\n<li><a href=\"http:\/\/subtlepatterns.com\/\">Subtle Patterns<\/a><\/li>\n<\/ul>\n<h3>Colors<\/h3>\n<ul>\n<li><a href=\"http:\/\/clrs.cc\/\">Colors<\/a> - A nicer color palette for the web.<\/li>\n<li><a href=\"https:\/\/color.adobe.com\/\">Adobe Color CC<\/a> - Another palette generator<\/li>\n<\/ul>\n<h3>Cool CSS effects<\/h3>\n<ul>\n<li><a href=\"http:\/\/tympanus.net\/Development\/CreativeLinkEffects\/\">Creative Link Effects<\/a><\/li>\n<li><a href=\"http:\/\/codepen.io\/andreasstorm\/pen\/pyjEh\">Medium scrolling effect<\/a><\/li>\n<li><a href=\"http:\/\/tympanus.net\/Development\/CreativeButtons\/\">Creative button styles<\/a><\/li>\n<li><a href=\"http:\/\/tympanus.net\/Development\/IconHoverEffects\/\">Icon hover effects<\/a><\/li>\n<\/ul>\n<p>More UI design inspiration can be found on <a href=\"http:\/\/tympanus.net\/codrops\/\">Codrops<\/a><\/p>\n<h2>Version control<\/h2>\n<p>Git is an indispensable CVS that your programming team should be aware of. You can host your project for free on both <a href=\"https:\/\/github.com\/\">Github<\/a> and <a href=\"https:\/\/bitbucket.org\/\">Bitbucket<\/a>, then use whatever service your team is most comfortable with.<\/p>\n<p>Git presents lots of benefits including:<\/p>\n<ul>\n<li>Merging updates from different team members can be much faster and organized<\/li>\n<li>Deploying your product online is easier with git especially if you use a PaaS service such as Heroku (more on that below)<\/li>\n<li>Rolling back and forth between versions, this could be a lifesaver in some cases. Make sure to commit often.<\/li>\n<\/ul>\n<p>If you don't know how to use Git yet, learn about it right now with <a href=\"http:\/\/try.github.com\/levels\/1\/challenges\/1\">this getting started guide<\/a>. In order for your team to use Git effectively, they need to be already familiar with it and have used it in at least one project before the hackathon.<\/p>\n<h2>Web application hosting<\/h2>\n<p>Hosting your application, if required, is going to take considerable time. Don't even consider architecture and server setup best practices. Try to use services that abstract as much as possible from your deployment process. Such service include:<\/p>\n<ul>\n<li><a href=\"http:\/\/heroku.com\/\">Heroku<\/a><\/li>\n<li><a href=\"https:\/\/cloud.google.com\/appengine\/\">Google app engine<\/a><\/li>\n<li><a href=\"https:\/\/www.openshift.com\/\">Open Shift<\/a><\/li>\n<\/ul>\n<p>If you choose any of the above mentioned solutions, familiarity with them is important. Pick one and discover it as much as possible before the hackathon. In my opinion, the easiest one to get started with is Heroku, especially if you have some experience with Git.<\/p>\n<p>If you'd still want to go with a vanilla VPS solution, such as <a href=\"https:\/\/www.linode.com\/\">Linode<\/a>, <a href=\"http:\/\/aws.amazon.com\/ec2\/\">Amazon EC2<\/a> or <a href=\"https:\/\/www.digitalocean.com\/\">Digital Ocean<\/a>, make sure to have your server setup before the hacking starts. That includes installing all the required software and testing that deployments are working.<\/p>\n<h2>Documentation<\/h2>\n<p>As we all know, docs are vital for any developer to go on implementing anything. That is why it is very important to gather all your technical docs and organize them under bookmarks that you can access fast.<\/p>\n<p>An even better solution is to download and use them offline. Hackathons tend to be crowded events with everyone using the internet connection, you don't have to be completely dependent on the connection speed to access your docs.<\/p>\n<p>There are some interesting desktop apps that enable you to cache, read and search through your favorite docs. Two great apps are <a href=\"https:\/\/kapeli.com\/dash\">Dash<\/a> (Mac OS only) and <a href=\"http:\/\/zealdocs.org\/\">Zeal<\/a> (Cross-platform).<\/p>\n<h2>Code libraries and frameworks<\/h2>\n<p>Here is a list of the most popular libraries that you might want to include in your project to speed things up. Most of the recommendations are inclined towards web-based technologies, they can also be used within the context of a cross-platform HTML5 app running on top of something like Cordova.<\/p>\n<h3>Front-end web UI and Responsive Grid systems<\/h3>\n<ul>\n<li><a href=\"http:\/\/getbootstrap.com\/\">Bootstrap 3<\/a><\/li>\n<li><a href=\"http:\/\/foundation.zurb.com\/\">Foundation<\/a><\/li>\n<li><a href=\"http:\/\/materializecss.com\/\">Materialize<\/a> - Provides a full implementation of the Google material design, including a responsive grid.<\/li>\n<li><a href=\"http:\/\/getchopstick.com\/\">Chopstick<\/a><\/li>\n<\/ul>\n<p>Less fully-featured frameworks include:<\/p>\n<ul>\n<li><a href=\"http:\/\/getskeleton.com\/\">Skeleton<\/a><\/li>\n<li><a href=\"http:\/\/neat.bourbon.io\/\">Neat<\/a><\/li>\n<\/ul>\n<h3>Core Javascript libraries<\/h3>\n<ul>\n<li><a href=\"https:\/\/jquery.com\/\">jQuery<\/a><\/li>\n<li><a href=\"http:\/\/jqueryui.com\/\">jQuery UI<\/a><\/li>\n<li><a href=\"http:\/\/knockoutjs.com\/\">KnockoutJS<\/a><\/li>\n<\/ul>\n<h3>Javascript frameworks<\/h3>\n<p>If you're not already familiar with a Javascript framework, it might make a lot of sense to get into one. If you're experienced with any of the below mentioned frameworks, you'll be able to prototype your client-side functionality much faster than doing it in vanilla JS or utilizing simple libraries like jQuery.<\/p>\n<ul>\n<li><a href=\"https:\/\/angularjs.org\/\">AngularJS<\/a><\/li>\n<li><a href=\"http:\/\/emberjs.com\/\">EmberJS<\/a><\/li>\n<li><a href=\"https:\/\/www.meteor.com\/\">Meteor<\/a><\/li>\n<\/ul>\n<p>Don't plunge into a full-fledged framework unless you have around a month or two to get to experience it. If your hackathon is next week and you don't know any frameworks, you're better off exercising on libraries you already know and explore small libraries and plugins that can help for different use cases.<\/p>\n<h3>UI Control Libraries<\/h3>\n<ul>\n<li><a href=\"http:\/\/brianreavis.github.io\/selectize.js\/\">Selectize<\/a><\/li>\n<li><a href=\"http:\/\/github.hubspot.com\/drop\/docs\/welcome\/\">Drop.js<\/a> - The fast and capable dropdown library<\/li>\n<li><a href=\"http:\/\/scrollrevealjs.org\/\">Scroll Reveal<\/a> - Easily reveal elements as they enter the viewport<\/li>\n<li><a href=\"https:\/\/github.com\/rstacruz\/nprogress\">NProgress.js<\/a> - Slim progress bars like on YouTube and Medium.<\/li>\n<li><a href=\"https:\/\/github.com\/IanLunn\/Hover\">Hover<\/a> - Awesome CSS3 animations on mouse hover<\/li>\n<li><a href=\"http:\/\/wbotelhos.com\/raty\/\">jQuery Raty<\/a> - Star Rating Plugin<\/li>\n<li><a href=\"http:\/\/vitalets.github.io\/x-editable\/\">xEditable<\/a> - In-place editing with Twitter Bootstrap, jQuery UI or pure jQuery<\/li>\n<li><a href=\"http:\/\/fabien-d.github.io\/alertify.js\/\">Alertify<\/a> - Sweet looking alerts and browser dialogs<\/li>\n<li><a href=\"http:\/\/www.uibox.in\/\">UI Box<\/a> - Curated HTML, CSS, JS UI Component Library<\/li>\n<li><a href=\"http:\/\/masonry.desandro.com\/\">Masonry<\/a> - Cascading grid layout library<\/li>\n<\/ul>\n<h3>Charting libraries<\/h3>\n<ul>\n<li><a href=\"https:\/\/developers.google.com\/chart\/\">Google Charts<\/a><\/li>\n<li><a href=\"http:\/\/www.highcharts.com\/\">High Charts<\/a><\/li>\n<li><a href=\"http:\/\/canvasjs.com\/\">CanvasJS<\/a><\/li>\n<li><a href=\"http:\/\/gionkunz.github.io\/chartist-js\/\">Chartist.js<\/a><\/li>\n<li><a href=\"http:\/\/www.flotcharts.org\/\">FlotJS<\/a> Lightweight and comes as a jQuery plugin so integrates really well if you're using jQuery<\/li>\n<li><a href=\"http:\/\/www.amcharts.com\/\">AMCharts<\/a><\/li>\n<li><a href=\"http:\/\/d3js.org\/\">D3JS<\/a> - This is a low level (relatively) library with a significant learning curve, make sure you invest considerable time on it before using it.<\/li>\n<li>\n<p><a href=\"http:\/\/jtblin.github.io\/angular-chart.js\/\">Angular-chart<\/a> Mainly useful if you're already using Angular. What's really cool about it is that it allows you to build reactive charts.<\/p>\n<\/li>\n<li>\n<p><a href=\"http:\/\/addepar.github.io\/#\/ember-charts\/overview\">Ember Charts<\/a> If you're using Ember already as a framework of choice<\/p>\n<\/li>\n<\/ul>\n<h3>Mapping libraries<\/h3>\n<ul>\n<li><a href=\"https:\/\/developers.google.com\/maps\/\">Google maps API<\/a><\/li>\n<li><a href=\"https:\/\/www.mapbox.com\/\">Mapbox<\/a><\/li>\n<li><a href=\"http:\/\/www.openstreetmap.org\/\">Open Street Map<\/a><\/li>\n<li><a href=\"http:\/\/leafletjs.com\/\">LeafletJS<\/a><\/li>\n<li><a href=\"http:\/\/openlayers.org\/\">Open Layers<\/a><\/li>\n<li><a href=\"http:\/\/datamaps.github.io\/\">Data Maps<\/a><\/li>\n<li><a href=\"http:\/\/www.microsoft.com\/maps\/choose-your-bing-maps-API.aspx\">Bing Maps API<\/a><\/li>\n<li><a href=\"http:\/\/kartograph.org\/\">Kartograph<\/a><\/li>\n<\/ul>\n<h2>Backend as a service<\/h2>\n<p>I haven't included any backend frameworks or technologies above on purpose. You probably wont need to implement a custom backend within the scope of a hackathon.<\/p>\n<p>Don't allocate precious time building a perfect backend system nor architecting the perfect database. Since your prototype will be simple, its is more important to focus on the business logic, front-end UI and UX in order to make things happen in a short period of time. You can sacrifice building a perfect custom backend and API by consuming easier BaaS (Backend as a service) solutions.<\/p>\n<p>A good BaaS, will enable your team to:<\/p>\n<ul>\n<li>Have a CRUD enabled database, abstracted via a nicely documented API in no time<\/li>\n<li>Have a User management and authentication system baked into the backend<\/li>\n<li>Enable social media auth integration<\/li>\n<li>Get a push notifications system up and running<\/li>\n<li>Sync and store application and user data instantly across multiple platforms<\/li>\n<li>Save time on provisioning or managing server infrastructure<\/li>\n<\/ul>\n<p>All you have to do is consume the API from the application in order to do the common data persistence tasks.<\/p>\n<p>Here is list of popular solutions that meet the above criteria and more:<\/p>\n<ul>\n<li><a href=\"https:\/\/www.firebase.com\/\">Firebase<\/a><\/li>\n<li><a href=\"https:\/\/developers.facebook.com\/products\/parse\/\">Parse<\/a><\/li>\n<li><a href=\"https:\/\/backendless.com\/\">Backendless<\/a><\/li>\n<\/ul>\n<h2>Tools and setup<\/h2>\n<p>Make sure to setup your tools, code repository and frameworks before the official hacking starts. That way you'll be able to focus on implementation as soon as the hacking starts. Don't waste precious time for setting up environments or frameworks.<\/p>\n<h2>The Presentation<\/h2>\n<p>It would be sad to hustle for three days building something amazing and end screwing up the demo presentation. It happens a lot, a well polished and rehearsed presentation is essential to spend time on. Here are some tools and resources that can help:<\/p>\n<ul>\n<li><a href=\"http:\/\/www.google.com\/slides\/about\/\">Google Slides<\/a><\/li>\n<li><a href=\"http:\/\/prezi.com\/\">Prezi<\/a><\/li>\n<li><a href=\"http:\/\/www.apple.com\/ios\/keynote\/\">Keynote<\/a><\/li>\n<li><a href=\"http:\/\/www.sliderocket.com\/\">SlideRocket<\/a><\/li>\n<li><a href=\"https:\/\/github.com\/hakimel\/reveal.js\">RevealJS<\/a> - A framework for easily creating beautiful presentations using HTML.<\/li>\n<li><a href=\"http:\/\/techcrunch.com\/2014\/09\/01\/how-to-crush-your-hackathon-demo\/\">How to present a successful hackathon demo<\/a><\/li>\n<li><a href=\"https:\/\/djksar.wordpress.com\/2012\/10\/24\/top-5-tips-for-presenting-at-hackathons\/\">Top 5 tips for presenting at hackathons<\/a><\/li>\n<li><a href=\"http:\/\/www.lifehacker.com.au\/2013\/11\/top-10-ways-to-improve-hackathon-presentations\/\">Top 10 ways to improve hackathon presentations<\/a><\/li>\n<\/ul>\n<h2>Have fun!<\/h2>\n<p>First and foremost, a hackathon is about experimenting, meeting interesting people, learning something new and getting the opportunity to push your limits. Yes its a competition, but you don't have to stress yourself too much about winning. Make sure you have a good time, put your best effort and learn from your experience.<\/p>\n<p>That was my list for important tools to keep an eye on for your next hackathon. Do you think I have missed anything? Please let me know in the comments!<\/p>","category":[{"@attributes":{"term":"mvp"}},{"@attributes":{"term":"tricks"}},{"@attributes":{"term":"practices"}},{"@attributes":{"term":"tools"}},{"@attributes":{"term":"hackathon"}},{"@attributes":{"term":"tips"}}]},{"title":"A Developer's year in review","link":{"@attributes":{"href":"https:\/\/www.thecodeship.com\/general\/developer-year-in-review\/","rel":"alternate"}},"published":"2015-01-02T19:00:00+02:00","updated":"2015-01-02T19:00:00+02:00","author":{"name":"Ayman Farhat"},"id":"tag:www.thecodeship.com,2015-01-02:\/general\/developer-year-in-review\/","summary":"<p>As the end of every year approaches, we start to reflect back on it. As software developers, our questions can be many, What did I work on? What did I produce? What mistakes I have done, and what did we learn from them. Today, I'll seize this opportunity to ...<\/p>","content":"<p>As the end of every year approaches, we start to reflect back on it. As software developers, our questions can be many, What did I work on? What did I produce? What mistakes I have done, and what did we learn from them. Today, I'll seize this opportunity to write down my thoughts on 2014 and put a little direction for my goals in 2015.<\/p>\n<p>For me, this year was a transformational one on many levels. I joined a fantastic company(<a href=\"http:\/\/www.wamda.com\/\">wamda.com<\/a>), working with an amazing team, building promising products and enjoying the freedom of picking the right tools, the best practices and applying them to my workflow every day. Since we operate as a small team, it was not just about programming and churning out code. I was able to widen my horizon by getting involved more in planning, communication, operations, and the full agile development lifecycle. That entailed much more responsibility, and the learning experience was amazing. The early months of 2015 will start bearing the fruits of last year's hard work.<\/p>\n<p>It was a busy year for me, and that is reflected through the few posts on the blog. Despite this low activity from my end, the number of page views increased by a whopping 157%, number of sessions increased by 183%, and number of users increased by 186%, as compared to 2013. Almost all of the traffic was organic. The top 5 articles for this year were:<\/p>\n<ul>\n<li><a href=\"..\/..\/..\/..\/patterns\/guide-to-python-function-decorators\/\">A guide to Python's function decorators<\/a><\/li>\n<li><a href=\"..\/..\/..\/..\/deployment\/deploy-django-apache-virtualenv-and-mod_wsgi\/\">Deploy Django on Apache with Virtualenv and mod_wsgi<\/a><\/li>\n<li><a href=\"..\/..\/..\/..\/web-development\/methods-within-constructor-vs-prototype-in-javascript\/\">Methods Within Constructor vs Prototype in Javascript<\/a><\/li>\n<li><a href=\"..\/..\/..\/..\/web-development\/alternative-to-javascript-evil-setinterval\/\">An alternative to Javascript's evil setInterval<\/a><\/li>\n<li><a href=\"..\/..\/..\/..\/web-development\/javascript-url-object\/\">Javascript URL Object<\/a><\/li>\n<\/ul>\n<h2>Technologies, frameworks and tools<\/h2>\n<h3>Laravel 4<\/h3>\n<p>I made the shift from using Code Igniter to <a href=\"http:\/\/laravel.com\/\">Laravel 4<\/a> when doing PHP development. It was a much needed one, I'm using this framework mostly for work, and really enjoying its power and elegance.<\/p>\n<h3>Angular JS<\/h3>\n<p>The world is shifting to building fat client web applications, and demand for Single Page Apps is increasing. Investing time in learning an SPA framework, was a no-brainer. <a href=\"https:\/\/angularjs.org\/\">Angular<\/a> can give you real super powers once you get the hang out of it, building complex client side applications becomes much more structured, maintainable and fun. I was leaning more towards EmberJS yet chose Angular as it seemed friendlier and better documented to start with. In 2015, I'm definitely evaluating Ember and focusing on a single client side framework from there.<\/p>\n<h3>Neo4J<\/h3>\n<p>In one of our projects, our data was insanely unstructured and pretty dense in relations. The initial architecture was done with MySQL, and then we started thinking that a relational database might not be the right choice for modeling an unstructured graph. That's where I discovered and shifted to Neo4J as our database for that project. The shift was pretty smooth, the project was still in early stages, and we saved a lot of lines of code. I did though underestimate the time it would take us to make the shift, which got reflected into a delay in the development of some features. The advantages that a graph database gave us were probably worth the time, faster development, more expressive queries, and the opportunity to dig deeper into our data in order to extract smarter information. For a better overview on graph databases and <a href=\"http:\/\/neo4j.com\/\">Neo4J<\/a>, I suggest you check out this great summary on <a href=\"http:\/\/www.slideshare.net\/emileifrem\/nosql-east-a-nosql-overview-and-the-benefits-of-graph-databases\">The Benefits Of Graph Databases<\/a><\/p>\n<h3>Elastic Search<\/h3>\n<p>Getting into <a href=\"http:\/\/www.elasticsearch.org\/\">Elastic Search<\/a> was an important experience for me too. Despite only scratching the surface of what this search engine can do, it was a pretty good introduction to pave the way for digging deeper into implementing search features into projects with it.<\/p>\n<h3>Other improvements added to my web development workflow<\/h3>\n<ul>\n<li>Writing <a href=\"http:\/\/sass-lang.com\/\">SASS<\/a> instead of CSS in larger projects  <\/li>\n<li>Managing dependencies with <a href=\"http:\/\/bower.io\/\">Bower<\/a>  <\/li>\n<li>Automating tasks with <a href=\"http:\/\/gruntjs.com\/\">Grunt<\/a><\/li>\n<\/ul>\n<h2>Projects<\/h2>\n<p>When it came to side projects, it was mainly unfinished experiments here and there, either to test or learn something new. I participated in <a target=\"_blank\">Js13kGames<\/a> competition and managed to get the 11th place in the mobile category with <a href=\"http:\/\/jumbit.co\">Jumbit<\/a>. Jumbit is my second vanilla Javascript game after <a href=\"http:\/\/aymanfarhat.com\/game\">The Game<\/a>.<\/p>\n<p>One very important thing I learned this year about projects, is that if you want to accomplish things, you have to get into the habbit of finishing what you start. Instead of pushing 2 small projects online I could have done much more with all the half projects I built, the problem is that I did not commit to getting them online. It doesn't matter how your perception or interest changes towards a project, just finish it and put it online before moving on to the next idea. You'll never know the real value of an idea if no one else sees it or gives you feedback on it.<\/p>\n<p>In 2015 I will do my best to finish what I start, before moving on to the next thing.<\/p>\n<h2>Books Read<\/h2>\n<p>This year was pretty rich when it came to reading. Introducing audio books to my daily commutes enabled me to get a better understanding of certain books before reading them, this can help a lot in stirring interest to read some books.<\/p>\n<p>I was able to get into the following books:  <\/p>\n<ul>\n<li>\n<p><a href=\"http:\/\/www.amazon.com\/gp\/product\/0374533555\/ref=as_li_tl?ie=UTF8&amp;camp=1789&amp;creative=390957&amp;creativeASIN=0374533555&amp;linkCode=as2&amp;tag=thcosh00-20&amp;linkId=XS4I4XRK33GL7BWL\">Thinking fast and slow<\/a>  <\/p>\n<\/li>\n<li>\n<p><a href=\"http:\/\/www.amazon.com\/gp\/product\/0307887898\/ref=as_li_tl?ie=UTF8&amp;camp=1789&amp;creative=390957&amp;creativeASIN=0307887898&amp;linkCode=as2&amp;tag=thcosh00-20&amp;linkId=VETV56PFI4VBBIVC\">The Lean Startup<\/a>  <\/p>\n<\/li>\n<li><a href=\"http:\/\/www.amazon.com\/gp\/product\/B00FFBOGDQ\/ref=as_li_tl?ie=UTF8&amp;camp=1789&amp;creative=390957&amp;creativeASIN=B00FFBOGDQ&amp;linkCode=as2&amp;tag=thcosh00-20&amp;linkId=DQ2RHFNX65EL5KA2\">The Genius in all of us<\/a>  <\/li>\n<li><a href=\"http:\/\/www.amazon.com\/gp\/product\/1118441540\/ref=as_li_tl?ie=UTF8&amp;camp=1789&amp;creative=390957&amp;creativeASIN=1118441540&amp;linkCode=as2&amp;tag=thcosh00-20&amp;linkId=KZ4VXQUIJNM7KDO6\">Startup communities<\/a>  <\/li>\n<li><a href=\"http:\/\/www.amazon.com\/gp\/product\/0684832674\/ref=as_li_tl?ie=UTF8&amp;camp=1789&amp;creative=390957&amp;creativeASIN=0684832674&amp;linkCode=as2&amp;tag=thcosh00-20&amp;linkId=XPO7PJCK2ZBHEVQR\">Where Wizards Stay Up Late: The Origins Of The Internet<\/a>  <\/li>\n<li><a href=\"http:\/\/www.amazon.com\/gp\/product\/0316204366\/ref=as_li_tl?ie=UTF8&amp;camp=1789&amp;creative=390957&amp;creativeASIN=0316204366&amp;linkCode=as2&amp;tag=thcosh00-20&amp;linkId=UJUPZ76BNSZAPSQO\">David and Goliath: Underdogs, Misfits, and the Art of Battling Giants<\/a>  <\/li>\n<li><a href=\"..\/add\/\">Javascript The Good Parts<\/a>  <\/li>\n<li><a href=\"http:\/\/www.amazon.com\/gp\/product\/193398869X\/ref=as_li_tl?ie=UTF8&amp;camp=1789&amp;creative=390957&amp;creativeASIN=193398869X&amp;linkCode=as2&amp;tag=thcosh00-20&amp;linkId=YXO6DHCYTIKKGJMS\">Secrets of the Javascript Ninja (In progress)<\/a>  <\/li>\n<li><a href=\"http:\/\/www.amazon.com\/gp\/product\/0735611319\/ref=as_li_tl?ie=UTF8&amp;camp=1789&amp;creative=390957&amp;creativeASIN=0735611319&amp;linkCode=as2&amp;tag=thcosh00-20&amp;linkId=QVBMQEKPK7DC2ZS6\">CODE: Hidden Language of computer software and hardware (In progress)<\/a><\/li>\n<\/ul>\n<h2>Health<\/h2>\n<p>This year, I got more serious about running; in summer I was training 3 times a week. In November, I participated in the local 10 Km marathon and was able to complete it in around 1 hour. It does not sound much, but definitely an improvement in endurance and fitness over that of the previous year. Exercising has helped me a lot in feeling better, happier and becoming more productive, I would recommend to every programmer out there, to commit to a cardio training habit, running is a life changing habit. In 2015 I'm planning on continuing the training across the whole year, and my target time for the 10K marathon this time is 45 minutes.<\/p>\n<h2>What about the resolutions for the new year?<\/h2>\n<p>As I compared what I wanted to do in 2014 with what I actually did, I noticed that I did meet some of the goals, while I totally forgot other ones, and instead did different things. I think year-wide goals are just too broad, during a year we develop and change a lot. Views, interests, perception and opinions change throughout the year, sometimes we reach the end with a completely different perspective than the one we started with. That's why I think, planning in terms of quarters might be a much more effective approach. Planning goals for 3 months or a quarter ahead gives much better control and focus over them, not to mention that new goals every 3 months would directly be related to a much more refined image on how the year is going and what's happening in it. I am going to try doing that this year, and see how it goes. As a matter of fact I am currently developing a tool to aid me just in that (details to be posted soon).<\/p>\n<p>That was a brief summary on one of the best years of my life, and I'm really looking forward to the next one. Hopefully, more valuable projects, more productivity and more articles on this blog. What about you? How was your 2014 as a developer? And what are you looking forward to in 2015? Would love to hear your thoughts in the comments. Happy new year!<\/p>","category":[{"@attributes":{"term":"review"}},{"@attributes":{"term":"developer"}},{"@attributes":{"term":"2014"}}]},{"title":"Common pitfalls when working with Javascript Arrays","link":{"@attributes":{"href":"https:\/\/www.thecodeship.com\/web-development\/common-pitfalls-when-working-with-javascript-arrays\/","rel":"alternate"}},"published":"2014-03-27T15:00:00+02:00","updated":"2014-03-27T15:00:00+02:00","author":{"name":"Ayman Farhat"},"id":"tag:www.thecodeship.com,2014-03-27:\/web-development\/common-pitfalls-when-working-with-javascript-arrays\/","summary":"<p>In most programming languages, an array is a data structure represented by an arrangement of items at equally spaced addresses in computer memory. Unfortunately, that is not the case with Javascript. A Javascript array is simply an object with array like characteristics reflected through certain methods. Apart from being much \u2026<\/p>","content":"<p>In most programming languages, an array is a data structure represented by an arrangement of items at equally spaced addresses in computer memory. Unfortunately, that is not the case with Javascript. A Javascript array is simply an object with array like characteristics reflected through certain methods. Apart from being much ...<\/p>\n<p>In most programming languages, an array is a data structure represented by an arrangement of items at equally spaced addresses in computer memory. Unfortunately, that is not the case with Javascript. A Javascript array is simply an object with array like characteristics reflected through certain methods.<\/p>\n<p>Apart from being much slower than regular arrays, they can be quite problematic when misused. I will be summarizing some of the pitfalls that I have come across, along with tips to avoid them.<\/p>\n<h2>Length<\/h2>\n<p>The length of an array is computed as the largest integer property in the object plus one, and that is not necessarily the number of objects in the array! Lets take a simple example:<\/p>\n<div class=\"highlight\"><pre><span><\/span><span class=\"kd\">var<\/span> <span class=\"nx\">myArray<\/span> <span class=\"o\">=<\/span> <span class=\"p\">[];<\/span>\n<span class=\"nx\">console<\/span><span class=\"p\">.<\/span><span class=\"nx\">log<\/span><span class=\"p\">(<\/span><span class=\"nx\">myArray<\/span><span class=\"p\">.<\/span><span class=\"nx\">length<\/span><span class=\"p\">);<\/span> <span class=\"c1\">\/\/ outputs 0<\/span>\n<span class=\"nx\">myArray<\/span><span class=\"p\">[<\/span><span class=\"mi\">100<\/span><span class=\"p\">]<\/span> <span class=\"o\">=<\/span> <span class=\"s1\">&#39;some value&#39;<\/span><span class=\"p\">;<\/span>\n<span class=\"nx\">console<\/span><span class=\"p\">.<\/span><span class=\"nx\">log<\/span><span class=\"p\">(<\/span><span class=\"nx\">myArray<\/span><span class=\"p\">.<\/span><span class=\"nx\">length<\/span><span class=\"p\">);<\/span> <span class=\"c1\">\/\/ outputs 101 despite containing one element<\/span>\n<\/pre><\/div>\n\n\n<p>That is why it is advised to use <a href=\"https:\/\/developer.mozilla.org\/en-US\/docs\/Web\/JavaScript\/Reference\/Global_Objects\/Array\/push\">Array.prototype.push<\/a> for all insertions and refrain from explicitly setting index values. The push method will insert the value with the right key\/index.<\/p>\n<h2>Removing elements<\/h2>\n<p>The <a href=\"https:\/\/developer.mozilla.org\/en-US\/docs\/Web\/JavaScript\/Reference\/Operators\/delete\">delete operator<\/a> can be used to remove a property from any object. Arrays are objects so if we apply that we'll end up with a hole in the array as the indexes(keys) will not shift automatically to reflect that change. Example:<\/p>\n<div class=\"highlight\"><pre><span><\/span><span class=\"kd\">var<\/span> <span class=\"nx\">myArray<\/span> <span class=\"o\">=<\/span> <span class=\"p\">[<\/span><span class=\"s1\">&#39;a&#39;<\/span><span class=\"p\">,<\/span> <span class=\"s1\">&#39;b&#39;<\/span><span class=\"p\">,<\/span> <span class=\"s1\">&#39;c&#39;<\/span><span class=\"p\">,<\/span> <span class=\"s1\">&#39;d&#39;<\/span><span class=\"p\">];<\/span>\n<span class=\"k\">delete<\/span> <span class=\"nx\">myArray<\/span><span class=\"p\">[<\/span><span class=\"mi\">1<\/span><span class=\"p\">];<\/span>\n<span class=\"nx\">console<\/span><span class=\"p\">.<\/span><span class=\"nx\">log<\/span><span class=\"p\">(<\/span><span class=\"nx\">myArray<\/span><span class=\"p\">[<\/span><span class=\"mi\">1<\/span><span class=\"p\">]);<\/span> <span class=\"c1\">\/\/ outputs undefined while the expected value was c<\/span>\n<\/pre><\/div>\n\n\n<p>Bottom line, using <strong>delete<\/strong> on arrays should be avoided.<\/p>\n<p>The proper method for removing an element on a certain index would be <a href=\"https:\/\/developer.mozilla.org\/en-US\/docs\/Web\/JavaScript\/Reference\/Global_Objects\/Array\/splice\">Array.prototype.splice<\/a> Other safe methods worth mentioning: <a href=\"https:\/\/developer.mozilla.org\/en-US\/docs\/Web\/JavaScript\/Reference\/Global_Objects\/Array\/pop\">Array.prototype.pop<\/a> - add\/remove elements from the end of the array <a href=\"https:\/\/developer.mozilla.org\/en-US\/docs\/Web\/JavaScript\/Reference\/Global_Objects\/Array\/shift\">Array.prototype.shift<\/a> - add\/remove elements from the beginning of the array<\/p>\n<h2>Sorting<\/h2>\n<p><a href=\"https:\/\/developer.mozilla.org\/en\/docs\/Web\/JavaScript\/Reference\/Global_Objects\/Array\/sort\">Array.prototype.sort()<\/a> is the go to method for sorting arrays, but there are a couple of issues we need to be aware of.<\/p>\n<p>The sorting order is by default <a href=\"http:\/\/en.wikipedia.org\/wiki\/Lexicographical_order\">lexicographic<\/a> and not numeric regardless of the types of values in the array. Even if the array is all numbers, all values will be converted to string and sorted lexicographically. Here is an example:<\/p>\n<div class=\"highlight\"><pre><span><\/span><span class=\"kd\">var<\/span> <span class=\"nx\">myarray<\/span> <span class=\"o\">=<\/span> <span class=\"p\">[<\/span><span class=\"mi\">80<\/span><span class=\"p\">,<\/span> <span class=\"mi\">9<\/span><span class=\"p\">,<\/span> <span class=\"mi\">34<\/span><span class=\"p\">,<\/span> <span class=\"mi\">23<\/span><span class=\"p\">,<\/span> <span class=\"mi\">5<\/span><span class=\"p\">,<\/span> <span class=\"mi\">1<\/span><span class=\"p\">];<\/span>\n<span class=\"nx\">myarray<\/span><span class=\"p\">.<\/span><span class=\"nx\">sort<\/span><span class=\"p\">();<\/span>\n<span class=\"nx\">console<\/span><span class=\"p\">.<\/span><span class=\"nx\">log<\/span><span class=\"p\">(<\/span><span class=\"nx\">myarray<\/span><span class=\"p\">);<\/span> <span class=\"c1\">\/\/ outputs: [1, 23, 34, 5, 80, 9] <\/span>\n<\/pre><\/div>\n\n\n<p>While the correct output for the above should be <code>[ 1, 5, 9, 23, 34, 80 ]<\/code><\/p>\n<p>Fortunately, the sort method takes an optional compare function in order to specify the sort order. When the compare function is supplied no type will take place. The order depends on the return value of the compare function. To learn more about the rules I suggest checking out the <a href=\"https:\/\/developer.mozilla.org\/en\/docs\/Web\/JavaScript\/Reference\/Global_Objects\/Array\/sort#Description\">description of compareFunction on MDN<\/a><\/p>\n<p>Back to our last example, the proper way to sort an array of numbers would be:<\/p>\n<div class=\"highlight\"><pre><span><\/span><span class=\"nx\">myarray<\/span><span class=\"p\">.<\/span><span class=\"nx\">sort<\/span><span class=\"p\">(<\/span><span class=\"kd\">function<\/span><span class=\"p\">(<\/span><span class=\"nx\">a<\/span><span class=\"p\">,<\/span> <span class=\"nx\">b<\/span><span class=\"p\">){<\/span>\n    <span class=\"k\">if<\/span> <span class=\"p\">(<\/span><span class=\"nx\">a<\/span> <span class=\"o\">&lt;<\/span> <span class=\"nx\">b<\/span><span class=\"p\">)<\/span>\n        <span class=\"k\">return<\/span> <span class=\"o\">-<\/span><span class=\"mi\">1<\/span><span class=\"p\">;<\/span>\n    <span class=\"k\">if<\/span> <span class=\"p\">(<\/span><span class=\"nx\">a<\/span> <span class=\"o\">&gt;<\/span> <span class=\"nx\">b<\/span><span class=\"p\">)<\/span>\n        <span class=\"k\">return<\/span> <span class=\"mi\">1<\/span><span class=\"p\">;<\/span>\n    <span class=\"k\">else<\/span>\n        <span class=\"k\">return<\/span> <span class=\"mi\">0<\/span><span class=\"p\">;<\/span>\n<span class=\"p\">});<\/span>\n<\/pre><\/div>\n\n\n<h2>Type Checking<\/h2>\n<p>Javascript's <a href=\"https:\/\/developer.mozilla.org\/en-US\/docs\/Web\/JavaScript\/Reference\/Operators\/typeof\">typeof operator<\/a> does not distinguish between an array and any other object.<\/p>\n<div class=\"highlight\"><pre><span><\/span><span class=\"nx\">console<\/span><span class=\"p\">.<\/span><span class=\"nx\">log<\/span><span class=\"p\">(<\/span><span class=\"k\">typeof<\/span> <span class=\"p\">[]);<\/span> <span class=\"c1\">\/\/ outputs: object<\/span>\n<span class=\"nx\">console<\/span><span class=\"p\">.<\/span><span class=\"nx\">log<\/span><span class=\"p\">(<\/span><span class=\"k\">typeof<\/span> <span class=\"p\">{});<\/span> <span class=\"c1\">\/\/ outputs: object<\/span>\n<\/pre><\/div>\n\n\n<p>In order to identify an array type there are a couple of ways. One thing we can do is check <a href=\"https:\/\/developer.mozilla.org\/en-US\/docs\/Web\/JavaScript\/Reference\/Global_Objects\/Object\/constructor\">Object.prototype.constructor<\/a> method which returns a reference to the object function that created the instance's prototype.<\/p>\n<div class=\"highlight\"><pre><span><\/span><span class=\"kd\">function<\/span> <span class=\"nx\">check_array<\/span> <span class=\"p\">(<\/span><span class=\"nx\">value<\/span><span class=\"p\">)<\/span> <span class=\"p\">{<\/span>\n    <span class=\"k\">return<\/span> <span class=\"nx\">value<\/span> <span class=\"o\">&amp;&amp;<\/span> <span class=\"k\">typeof<\/span> <span class=\"nx\">value<\/span> <span class=\"o\">===<\/span> <span class=\"s1\">&#39;object&#39;<\/span> <span class=\"o\">&amp;&amp;<\/span> <span class=\"nx\">value<\/span><span class=\"p\">.<\/span><span class=\"nx\">constructor<\/span> <span class=\"o\">===<\/span> <span class=\"nb\">Array<\/span><span class=\"p\">;<\/span>\n<span class=\"p\">}<\/span>\n\n<span class=\"nx\">console<\/span><span class=\"p\">.<\/span><span class=\"nx\">log<\/span><span class=\"p\">(<\/span><span class=\"nx\">check_array<\/span><span class=\"p\">([]));<\/span> <span class=\"c1\">\/\/ outputs: true<\/span>\n<span class=\"nx\">console<\/span><span class=\"p\">.<\/span><span class=\"nx\">log<\/span><span class=\"p\">(<\/span><span class=\"nx\">check_array<\/span><span class=\"p\">({}));<\/span> <span class=\"c1\">\/\/ outputs: false<\/span>\n<\/pre><\/div>\n\n\n<p>Another would be through applying <a href=\"https:\/\/developer.mozilla.org\/en-US\/docs\/Web\/JavaScript\/Reference\/Global_Objects\/Object\/toString\">Object.prototype.toString method<\/a> to get the class.<\/p>\n<div class=\"highlight\"><pre><span><\/span><span class=\"kd\">function<\/span> <span class=\"nx\">check_array<\/span> <span class=\"p\">(<\/span><span class=\"nx\">value<\/span><span class=\"p\">)<\/span> <span class=\"p\">{<\/span>\n    <span class=\"k\">return<\/span> <span class=\"nb\">Object<\/span><span class=\"p\">.<\/span><span class=\"nx\">prototype<\/span><span class=\"p\">.<\/span><span class=\"nx\">toString<\/span><span class=\"p\">.<\/span><span class=\"nx\">apply<\/span><span class=\"p\">(<\/span><span class=\"nx\">value<\/span><span class=\"p\">)<\/span> <span class=\"o\">===<\/span> <span class=\"s1\">&#39;[object Array]&#39;<\/span><span class=\"p\">;<\/span>\n<span class=\"p\">}<\/span>\n\n<span class=\"nx\">console<\/span><span class=\"p\">.<\/span><span class=\"nx\">log<\/span><span class=\"p\">(<\/span><span class=\"nx\">check_array<\/span><span class=\"p\">([]));<\/span> <span class=\"c1\">\/\/ outputs: true<\/span>\n<span class=\"nx\">console<\/span><span class=\"p\">.<\/span><span class=\"nx\">log<\/span><span class=\"p\">(<\/span><span class=\"nx\">check_array<\/span><span class=\"p\">({}));<\/span> <span class=\"c1\">\/\/ outputs: false<\/span>\n<\/pre><\/div>\n\n\n<h2>for-in loops<\/h2>\n<p>The <a href=\"https:\/\/developer.mozilla.org\/en-US\/docs\/Web\/JavaScript\/Reference\/Statements\/for...in\">for-in loop<\/a> seems convenient at times but it is meant to iterate over an object properties rather than an array structure. There are two main reasons for that:<\/p>\n<h3>It does not guarantee order<\/h3>\n<p>The use of for-in makes no guarantee over the order of the properties. The order of enumeration depends on the implementation which might differ between browsers. According to this <a href=\"http:\/\/ejohn.org\/blog\/javascript-in-chrome\/\">article<\/a>, all modern implementations iterate through object properties in the order in which they were defined.<\/p>\n<h3>Unexpected properties from the prototype chain<\/h3>\n<p>for-in does not only iterate over properties attached to the object itself but also over its prototypes, and that might result in extra unexpected results. Here is an example<\/p>\n<div class=\"highlight\"><pre><span><\/span><span class=\"c1\">\/\/ Somewhere else in our program we decided to add an extra property for all arrays<\/span>\n<span class=\"nb\">Array<\/span><span class=\"p\">.<\/span><span class=\"nx\">prototype<\/span><span class=\"p\">.<\/span><span class=\"nx\">foo<\/span> <span class=\"o\">=<\/span> <span class=\"s2\">&quot;bar&quot;<\/span><span class=\"p\">;<\/span>\n\n<span class=\"kd\">var<\/span> <span class=\"nx\">myArray<\/span> <span class=\"o\">=<\/span> <span class=\"p\">[<\/span><span class=\"mi\">1<\/span><span class=\"p\">,<\/span><span class=\"mi\">2<\/span><span class=\"p\">,<\/span><span class=\"mi\">3<\/span><span class=\"p\">];<\/span>\n<span class=\"kd\">var<\/span> <span class=\"nx\">result<\/span> <span class=\"o\">=<\/span> <span class=\"p\">[];<\/span>\n\n<span class=\"k\">for<\/span> <span class=\"p\">(<\/span><span class=\"kd\">var<\/span> <span class=\"nx\">key<\/span> <span class=\"k\">in<\/span> <span class=\"nx\">myArray<\/span><span class=\"p\">){<\/span>\n    <span class=\"nx\">console<\/span><span class=\"p\">.<\/span><span class=\"nx\">log<\/span><span class=\"p\">(<\/span><span class=\"nx\">key<\/span><span class=\"p\">);<\/span>\n<span class=\"p\">}<\/span>\n\n<span class=\"c1\">\/\/ Outputs:<\/span>\n<span class=\"c1\">\/\/ 0<\/span>\n<span class=\"c1\">\/\/ 1<\/span>\n<span class=\"c1\">\/\/ 2<\/span>\n<span class=\"c1\">\/\/ foo<\/span>\n<\/pre><\/div>\n\n\n<p>In the above example, <strong>foo<\/strong> becomes part of every array and will show up in a for-in. This can be avoided by checking that property does not belong to the prototype of the object on each iteration via <a href=\"https:\/\/developer.mozilla.org\/en-US\/docs\/Web\/JavaScript\/Reference\/Global_Objects\/Object\/hasOwnProperty\">hasOwnProperty<\/a>, but is all that worth it?<\/p>\n<p>And thats why it is advised to stick with the conventional for statement in order to avoid those problems. As long as the indexes of the array are correct it should work fine.<\/p>\n<h2>The associative array confusion<\/h2>\n<p>Associative arrays similar to those existing in other languages such as PHP do not exist in Javascript. An example such as this would \"work\" but is not correct:<\/p>\n<div class=\"highlight\"><pre><span><\/span><span class=\"kd\">var<\/span> <span class=\"nx\">myAssocArray<\/span> <span class=\"o\">=<\/span> <span class=\"p\">[];<\/span>\n<span class=\"nx\">myAssocArray<\/span><span class=\"p\">[<\/span><span class=\"s2\">&quot;foo&quot;<\/span><span class=\"p\">]<\/span> <span class=\"o\">=<\/span> <span class=\"s2\">&quot;bar&quot;<\/span><span class=\"p\">;<\/span>\n<\/pre><\/div>\n\n\n<p>The above example is not an associative array but rather an abused array object. The above code works simply because arrays are objects too. As a matter of fact the same thing can be applied to any other object in Javascript. <code>myAssocArray[\"foo\"] = \"bar\"<\/code> is equivalent to <code>myAssocArray.foo = \"bar\"<\/code>, simply adding a new property to the current array object.<\/p>\n<p>For a key\/value pairs structure using vanilla Javascript objects is the right option.<\/p>\n<h2>Did I miss anything?<\/h2>\n<p>I hope you enjoyed this post and found it useful in one way or another. If you think I missed anything, please let me know in the comments. Have a good day!<\/p>","category":[{"@attributes":{"term":"javascript"}},{"@attributes":{"term":"arrays"}},{"@attributes":{"term":"tips"}},{"@attributes":{"term":"pitfall"}}]},{"title":"A guide to Python's function decorators","link":{"@attributes":{"href":"https:\/\/www.thecodeship.com\/patterns\/guide-to-python-function-decorators\/","rel":"alternate"}},"published":"2014-01-06T12:00:00+02:00","updated":"2014-01-06T12:00:00+02:00","author":{"name":"Ayman Farhat"},"id":"tag:www.thecodeship.com,2014-01-06:\/patterns\/guide-to-python-function-decorators\/","summary":"<p>Python is rich with powerful features and expressive syntax. One of my favorites is decorators. In the context of design patterns, decorators dynamically alter the functionality of a function, method or class without having to directly use subclasses. This is ideal when you need to extend the functionality of functions ...<\/p>","content":"<p>Python is rich with powerful features and expressive syntax. One of my favorites is decorators. In the context of design patterns, decorators dynamically alter the functionality of a function, method or class without having to directly use subclasses. This is ideal when you need to extend the functionality of functions that you don't want to modify. We can implement the decorator pattern anywhere, but Python facilitates the implementation by providing much more expressive features and syntax for that.<\/p>\n<p>In this post I will be discussing Python's function decorators in depth, accompanied by a bunch of examples on the way to clear up the concepts. All examples are in Python 2.7 but the same concepts should apply to Python 3 with some change in the syntax.<\/p>\n<p>Essentially, decorators work as wrappers, modifying the behavior of the code before and after a target function execution, without the need to modify the function itself, augmenting the original functionality, thus decorating it.<\/p>\n<h2>What you need to know about functions<\/h2>\n<p>Before diving in, there are some prerequisites that should be clear. In Python, functions are first class citizens, they are objects and that means we can do a lot of useful stuff with them.<\/p>\n<h3>Assign functions to variables<\/h3>\n<div class=\"highlight\"><pre><span><\/span><span class=\"k\">def<\/span> <span class=\"nf\">greet<\/span><span class=\"p\">(<\/span><span class=\"n\">name<\/span><span class=\"p\">):<\/span>\n    <span class=\"k\">return<\/span> <span class=\"s2\">&quot;hello &quot;<\/span><span class=\"o\">+<\/span><span class=\"n\">name<\/span>\n\n<span class=\"n\">greet_someone<\/span> <span class=\"o\">=<\/span> <span class=\"n\">greet<\/span>\n<span class=\"k\">print<\/span><span class=\"p\">(<\/span><span class=\"n\">greet_someone<\/span><span class=\"p\">(<\/span><span class=\"s2\">&quot;John&quot;<\/span><span class=\"p\">))<\/span>\n\n<span class=\"c1\"># Outputs: hello John<\/span>\n<\/pre><\/div>\n\n\n<h3>Define functions inside other functions<\/h3>\n<div class=\"highlight\"><pre><span><\/span><span class=\"k\">def<\/span> <span class=\"nf\">greet<\/span><span class=\"p\">(<\/span><span class=\"n\">name<\/span><span class=\"p\">):<\/span>\n    <span class=\"k\">def<\/span> <span class=\"nf\">get_message<\/span><span class=\"p\">():<\/span>\n        <span class=\"k\">return<\/span> <span class=\"s2\">&quot;Hello &quot;<\/span>\n\n    <span class=\"n\">result<\/span> <span class=\"o\">=<\/span> <span class=\"n\">get_message<\/span><span class=\"p\">()<\/span><span class=\"o\">+<\/span><span class=\"n\">name<\/span>\n    <span class=\"k\">return<\/span> <span class=\"n\">result<\/span>\n\n<span class=\"k\">print<\/span><span class=\"p\">(<\/span><span class=\"n\">greet<\/span><span class=\"p\">(<\/span><span class=\"s2\">&quot;John&quot;<\/span><span class=\"p\">))<\/span>\n\n<span class=\"c1\"># Outputs: Hello John<\/span>\n<\/pre><\/div>\n\n\n<h3>Functions can be passed as parameters to other functions<\/h3>\n<div class=\"highlight\"><pre><span><\/span><span class=\"k\">def<\/span> <span class=\"nf\">greet<\/span><span class=\"p\">(<\/span><span class=\"n\">name<\/span><span class=\"p\">):<\/span>\n   <span class=\"k\">return<\/span> <span class=\"s2\">&quot;Hello &quot;<\/span> <span class=\"o\">+<\/span> <span class=\"n\">name<\/span> \n\n<span class=\"k\">def<\/span> <span class=\"nf\">call_func<\/span><span class=\"p\">(<\/span><span class=\"n\">func<\/span><span class=\"p\">):<\/span>\n    <span class=\"n\">other_name<\/span> <span class=\"o\">=<\/span> <span class=\"s2\">&quot;John&quot;<\/span>\n    <span class=\"k\">return<\/span> <span class=\"n\">func<\/span><span class=\"p\">(<\/span><span class=\"n\">other_name<\/span><span class=\"p\">)<\/span>  \n\n<span class=\"k\">print<\/span><span class=\"p\">(<\/span><span class=\"n\">call_func<\/span><span class=\"p\">(<\/span><span class=\"n\">greet<\/span><span class=\"p\">))<\/span>\n\n<span class=\"c1\"># Outputs: Hello John<\/span>\n<\/pre><\/div>\n\n\n<h3>Functions can return other functions<\/h3>\n<p>In other words, functions generating other functions.<\/p>\n<div class=\"highlight\"><pre><span><\/span><span class=\"k\">def<\/span> <span class=\"nf\">compose_greet_func<\/span><span class=\"p\">():<\/span>\n    <span class=\"k\">def<\/span> <span class=\"nf\">get_message<\/span><span class=\"p\">():<\/span>\n        <span class=\"k\">return<\/span> <span class=\"s2\">&quot;Hello there!&quot;<\/span>\n\n    <span class=\"k\">return<\/span> <span class=\"n\">get_message<\/span>\n\n<span class=\"n\">greet<\/span> <span class=\"o\">=<\/span> <span class=\"n\">compose_greet_func<\/span><span class=\"p\">()<\/span>\n<span class=\"k\">print<\/span><span class=\"p\">(<\/span><span class=\"n\">greet<\/span><span class=\"p\">())<\/span>\n\n<span class=\"c1\"># Outputs: Hello there!<\/span>\n<\/pre><\/div>\n\n\n<h3>Inner functions have access to the enclosing scope<\/h3>\n<p>More commonly known as a <a href=\"http:\/\/www.shutupandship.com\/2012\/01\/python-closures-explained.html\">closure<\/a>. A very powerful pattern that we will come across while building decorators. Another thing to note, Python only allows <a href=\"http:\/\/www.tech-thoughts-blog.com\/2013\/07\/writing-closure-in-python.html\">read access to the outer scope<\/a> and not assignment. Notice how we modified the example above to read a \"name\" argument from the enclosing scope of the inner function and return the new function.<\/p>\n<div class=\"highlight\"><pre><span><\/span><span class=\"k\">def<\/span> <span class=\"nf\">compose_greet_func<\/span><span class=\"p\">(<\/span><span class=\"n\">name<\/span><span class=\"p\">):<\/span>\n    <span class=\"k\">def<\/span> <span class=\"nf\">get_message<\/span><span class=\"p\">():<\/span>\n        <span class=\"k\">return<\/span> <span class=\"s2\">&quot;Hello there &quot;<\/span><span class=\"o\">+<\/span><span class=\"n\">name<\/span><span class=\"o\">+<\/span><span class=\"s2\">&quot;!&quot;<\/span>\n\n    <span class=\"k\">return<\/span> <span class=\"n\">get_message<\/span>\n\n<span class=\"n\">greet<\/span> <span class=\"o\">=<\/span> <span class=\"n\">compose_greet_func<\/span><span class=\"p\">(<\/span><span class=\"s2\">&quot;John&quot;<\/span><span class=\"p\">)<\/span>\n<span class=\"k\">print<\/span><span class=\"p\">(<\/span><span class=\"n\">greet<\/span><span class=\"p\">())<\/span>\n\n<span class=\"c1\"># Outputs: Hello there John!<\/span>\n<\/pre><\/div>\n\n\n<h2>Composition of Decorators<\/h2>\n<p>Function decorators are simply wrappers to existing functions. Putting the ideas mentioned above together, we can build a decorator. In this example let's consider a function that wraps the string output of another function by <strong>p<\/strong> tags.<\/p>\n<div class=\"highlight\"><pre><span><\/span><span class=\"k\">def<\/span> <span class=\"nf\">get_text<\/span><span class=\"p\">(<\/span><span class=\"n\">name<\/span><span class=\"p\">):<\/span>\n   <span class=\"k\">return<\/span> <span class=\"s2\">&quot;lorem ipsum, {0} dolor sit amet&quot;<\/span><span class=\"o\">.<\/span><span class=\"n\">format<\/span><span class=\"p\">(<\/span><span class=\"n\">name<\/span><span class=\"p\">)<\/span>\n\n<span class=\"k\">def<\/span> <span class=\"nf\">p_decorate<\/span><span class=\"p\">(<\/span><span class=\"n\">func<\/span><span class=\"p\">):<\/span>\n   <span class=\"k\">def<\/span> <span class=\"nf\">func_wrapper<\/span><span class=\"p\">(<\/span><span class=\"n\">name<\/span><span class=\"p\">):<\/span>\n       <span class=\"k\">return<\/span> <span class=\"s2\">&quot;&lt;p&gt;{0}&lt;\/p&gt;&quot;<\/span><span class=\"o\">.<\/span><span class=\"n\">format<\/span><span class=\"p\">(<\/span><span class=\"n\">func<\/span><span class=\"p\">(<\/span><span class=\"n\">name<\/span><span class=\"p\">))<\/span>\n   <span class=\"k\">return<\/span> <span class=\"n\">func_wrapper<\/span>\n\n<span class=\"n\">my_get_text<\/span> <span class=\"o\">=<\/span> <span class=\"n\">p_decorate<\/span><span class=\"p\">(<\/span><span class=\"n\">get_text<\/span><span class=\"p\">)<\/span>\n\n<span class=\"k\">print<\/span><span class=\"p\">(<\/span><span class=\"n\">my_get_text<\/span><span class=\"p\">(<\/span><span class=\"s2\">&quot;John&quot;<\/span><span class=\"p\">))<\/span>\n\n<span class=\"c1\"># &lt;p&gt;Outputs lorem ipsum, John dolor sit amet&lt;\/p&gt;<\/span>\n<\/pre><\/div>\n\n\n<p>That was our first decorator. A function that takes another function as an argument, generates a new function, augmenting the work of the original function, and returning the generated function so we can use it anywhere. To have get_text itself be decorated by p_decorate, we just have to assign get_text to the result of p_decorate.<\/p>\n<div class=\"highlight\"><pre><span><\/span><span class=\"n\">get_text<\/span> <span class=\"o\">=<\/span> <span class=\"n\">p_decorate<\/span><span class=\"p\">(<\/span><span class=\"n\">get_text<\/span><span class=\"p\">)<\/span>\n\n<span class=\"k\">print<\/span><span class=\"p\">(<\/span><span class=\"n\">get_text<\/span><span class=\"p\">(<\/span><span class=\"s2\">&quot;John&quot;<\/span><span class=\"p\">))<\/span>\n\n<span class=\"c1\"># &lt;p&gt;Outputs lorem ipsum, John dolor sit amet&lt;\/p&gt;<\/span>\n<\/pre><\/div>\n\n\n<p>Another thing to notice is that our decorated function takes a name argument. All what we had to do in the decorator is to let the wrapper of get_text pass that argument.<\/p>\n<h2>Python's Decorator Syntax<\/h2>\n<p>Python makes creating and using decorators a bit cleaner and nicer for the programmer through some <a href=\"http:\/\/en.wikipedia.org\/wiki\/Syntactic_sugar\">syntactic sugar<\/a> To decorate get_text we don't have to <code>get_text = p_decorator(get_text)<\/code> There is a neat shortcut for that, which is to mention the name of the decorating function before the function to be decorated. The name of the decorator should be perpended with an @ symbol.<\/p>\n<div class=\"highlight\"><pre><span><\/span><span class=\"k\">def<\/span> <span class=\"nf\">p_decorate<\/span><span class=\"p\">(<\/span><span class=\"n\">func<\/span><span class=\"p\">):<\/span>\n   <span class=\"k\">def<\/span> <span class=\"nf\">func_wrapper<\/span><span class=\"p\">(<\/span><span class=\"n\">name<\/span><span class=\"p\">):<\/span>\n       <span class=\"k\">return<\/span> <span class=\"s2\">&quot;&lt;p&gt;{0}&lt;\/p&gt;&quot;<\/span><span class=\"o\">.<\/span><span class=\"n\">format<\/span><span class=\"p\">(<\/span><span class=\"n\">func<\/span><span class=\"p\">(<\/span><span class=\"n\">name<\/span><span class=\"p\">))<\/span>\n   <span class=\"k\">return<\/span> <span class=\"n\">func_wrapper<\/span>\n\n<span class=\"nd\">@p_decorate<\/span>\n<span class=\"k\">def<\/span> <span class=\"nf\">get_text<\/span><span class=\"p\">(<\/span><span class=\"n\">name<\/span><span class=\"p\">):<\/span>\n   <span class=\"k\">return<\/span> <span class=\"s2\">&quot;lorem ipsum, {0} dolor sit amet&quot;<\/span><span class=\"o\">.<\/span><span class=\"n\">format<\/span><span class=\"p\">(<\/span><span class=\"n\">name<\/span><span class=\"p\">)<\/span>\n\n<span class=\"k\">print<\/span><span class=\"p\">(<\/span><span class=\"n\">get_text<\/span><span class=\"p\">(<\/span><span class=\"s2\">&quot;John&quot;<\/span><span class=\"p\">))<\/span>\n\n<span class=\"c1\"># Outputs &lt;p&gt;lorem ipsum, John dolor sit amet&lt;\/p&gt;<\/span>\n<\/pre><\/div>\n\n\n<p>Now let's consider we wanted to decorate our get_text function by 2 other functions to wrap a div and strong tag around the string output.<\/p>\n<div class=\"highlight\"><pre><span><\/span><span class=\"k\">def<\/span> <span class=\"nf\">p_decorate<\/span><span class=\"p\">(<\/span><span class=\"n\">func<\/span><span class=\"p\">):<\/span>\n   <span class=\"k\">def<\/span> <span class=\"nf\">func_wrapper<\/span><span class=\"p\">(<\/span><span class=\"n\">name<\/span><span class=\"p\">):<\/span>\n       <span class=\"k\">return<\/span> <span class=\"s2\">&quot;&lt;p&gt;{0}&lt;\/p&gt;&quot;<\/span><span class=\"o\">.<\/span><span class=\"n\">format<\/span><span class=\"p\">(<\/span><span class=\"n\">func<\/span><span class=\"p\">(<\/span><span class=\"n\">name<\/span><span class=\"p\">))<\/span>\n   <span class=\"k\">return<\/span> <span class=\"n\">func_wrapper<\/span>\n\n<span class=\"k\">def<\/span> <span class=\"nf\">strong_decorate<\/span><span class=\"p\">(<\/span><span class=\"n\">func<\/span><span class=\"p\">):<\/span>\n    <span class=\"k\">def<\/span> <span class=\"nf\">func_wrapper<\/span><span class=\"p\">(<\/span><span class=\"n\">name<\/span><span class=\"p\">):<\/span>\n        <span class=\"k\">return<\/span> <span class=\"s2\">&quot;&lt;strong&gt;{0}&lt;\/strong&gt;&quot;<\/span><span class=\"o\">.<\/span><span class=\"n\">format<\/span><span class=\"p\">(<\/span><span class=\"n\">func<\/span><span class=\"p\">(<\/span><span class=\"n\">name<\/span><span class=\"p\">))<\/span>\n    <span class=\"k\">return<\/span> <span class=\"n\">func_wrapper<\/span>\n\n<span class=\"k\">def<\/span> <span class=\"nf\">div_decorate<\/span><span class=\"p\">(<\/span><span class=\"n\">func<\/span><span class=\"p\">):<\/span>\n    <span class=\"k\">def<\/span> <span class=\"nf\">func_wrapper<\/span><span class=\"p\">(<\/span><span class=\"n\">name<\/span><span class=\"p\">):<\/span>\n        <span class=\"k\">return<\/span> <span class=\"s2\">&quot;&lt;div&gt;{0}&lt;\/div&gt;&quot;<\/span><span class=\"o\">.<\/span><span class=\"n\">format<\/span><span class=\"p\">(<\/span><span class=\"n\">func<\/span><span class=\"p\">(<\/span><span class=\"n\">name<\/span><span class=\"p\">))<\/span>\n    <span class=\"k\">return<\/span> <span class=\"n\">func_wrapper<\/span>\n<\/pre><\/div>\n\n\n<p>With the basic approach, decorating get_text would be along the lines of<\/p>\n<div class=\"highlight\"><pre><span><\/span><span class=\"n\">get_text<\/span> <span class=\"o\">=<\/span> <span class=\"n\">div_decorate<\/span><span class=\"p\">(<\/span><span class=\"n\">p_decorate<\/span><span class=\"p\">(<\/span><span class=\"n\">strong_decorate<\/span><span class=\"p\">(<\/span><span class=\"n\">get_text<\/span><span class=\"p\">)))<\/span>\n<\/pre><\/div>\n\n\n<p>With Python's decorator syntax, same thing can be achieved with much more expressive power.<\/p>\n<div class=\"highlight\"><pre><span><\/span><span class=\"nd\">@div_decorate<\/span>\n<span class=\"nd\">@p_decorate<\/span>\n<span class=\"nd\">@strong_decorate<\/span>\n<span class=\"k\">def<\/span> <span class=\"nf\">get_text<\/span><span class=\"p\">(<\/span><span class=\"n\">name<\/span><span class=\"p\">):<\/span>\n   <span class=\"k\">return<\/span> <span class=\"s2\">&quot;lorem ipsum, {0} dolor sit amet&quot;<\/span><span class=\"o\">.<\/span><span class=\"n\">format<\/span><span class=\"p\">(<\/span><span class=\"n\">name<\/span><span class=\"p\">)<\/span>\n\n<span class=\"k\">print<\/span><span class=\"p\">(<\/span><span class=\"n\">get_text<\/span><span class=\"p\">(<\/span><span class=\"s2\">&quot;John&quot;<\/span><span class=\"p\">))<\/span>\n\n<span class=\"c1\"># Outputs &lt;div&gt;&lt;p&gt;&lt;strong&gt;lorem ipsum, John dolor sit amet&lt;\/strong&gt;&lt;\/p&gt;&lt;\/div&gt;<\/span>\n<\/pre><\/div>\n\n\n<p>One important thing to notice here is that the order of setting our decorators matters. If the order was different in the example above, the output would have been different.<\/p>\n<h2>Decorating Methods<\/h2>\n<p>In Python, methods are functions that expect their first parameter to be a reference to the current object. We can build decorators for methods the same way, while taking <strong>self<\/strong> into consideration in the wrapper function.<\/p>\n<div class=\"highlight\"><pre><span><\/span><span class=\"k\">def<\/span> <span class=\"nf\">p_decorate<\/span><span class=\"p\">(<\/span><span class=\"n\">func<\/span><span class=\"p\">):<\/span>\n   <span class=\"k\">def<\/span> <span class=\"nf\">func_wrapper<\/span><span class=\"p\">(<\/span><span class=\"bp\">self<\/span><span class=\"p\">):<\/span>\n       <span class=\"k\">return<\/span> <span class=\"s2\">&quot;&lt;p&gt;{0}&lt;\/p&gt;&quot;<\/span><span class=\"o\">.<\/span><span class=\"n\">format<\/span><span class=\"p\">(<\/span><span class=\"n\">func<\/span><span class=\"p\">(<\/span><span class=\"bp\">self<\/span><span class=\"p\">))<\/span>\n   <span class=\"k\">return<\/span> <span class=\"n\">func_wrapper<\/span>\n\n<span class=\"k\">class<\/span> <span class=\"nc\">Person<\/span><span class=\"p\">(<\/span><span class=\"nb\">object<\/span><span class=\"p\">):<\/span>\n    <span class=\"k\">def<\/span> <span class=\"fm\">__init__<\/span><span class=\"p\">(<\/span><span class=\"bp\">self<\/span><span class=\"p\">):<\/span>\n        <span class=\"bp\">self<\/span><span class=\"o\">.<\/span><span class=\"n\">name<\/span> <span class=\"o\">=<\/span> <span class=\"s2\">&quot;John&quot;<\/span>\n        <span class=\"bp\">self<\/span><span class=\"o\">.<\/span><span class=\"n\">family<\/span> <span class=\"o\">=<\/span> <span class=\"s2\">&quot;Doe&quot;<\/span>\n\n    <span class=\"nd\">@p_decorate<\/span>\n    <span class=\"k\">def<\/span> <span class=\"nf\">get_fullname<\/span><span class=\"p\">(<\/span><span class=\"bp\">self<\/span><span class=\"p\">):<\/span>\n        <span class=\"k\">return<\/span> <span class=\"bp\">self<\/span><span class=\"o\">.<\/span><span class=\"n\">name<\/span><span class=\"o\">+<\/span><span class=\"s2\">&quot; &quot;<\/span><span class=\"o\">+<\/span><span class=\"bp\">self<\/span><span class=\"o\">.<\/span><span class=\"n\">family<\/span>\n\n<span class=\"n\">my_person<\/span> <span class=\"o\">=<\/span> <span class=\"n\">Person<\/span><span class=\"p\">()<\/span>\n<span class=\"k\">print<\/span><span class=\"p\">(<\/span><span class=\"n\">my_person<\/span><span class=\"o\">.<\/span><span class=\"n\">get_fullname<\/span><span class=\"p\">())<\/span>\n<\/pre><\/div>\n\n\n<p>A much better approach would be to make our decorator useful for functions and methods alike. This can be done by putting <a href=\"http:\/\/docs.python.org\/2\/tutorial\/controlflow.html#arbitrary-argument-lists\"><em>args and <\/em>*kwargs<\/a> as parameters for the wrapper, then it can accept any arbitrary number of arguments and keyword arguments.<\/p>\n<div class=\"highlight\"><pre><span><\/span><span class=\"k\">def<\/span> <span class=\"nf\">p_decorate<\/span><span class=\"p\">(<\/span><span class=\"n\">func<\/span><span class=\"p\">):<\/span>\n   <span class=\"k\">def<\/span> <span class=\"nf\">func_wrapper<\/span><span class=\"p\">(<\/span><span class=\"o\">*<\/span><span class=\"n\">args<\/span><span class=\"p\">,<\/span> <span class=\"o\">**<\/span><span class=\"n\">kwargs<\/span><span class=\"p\">):<\/span>\n       <span class=\"k\">return<\/span> <span class=\"s2\">&quot;&lt;p&gt;{0}&lt;\/p&gt;&quot;<\/span><span class=\"o\">.<\/span><span class=\"n\">format<\/span><span class=\"p\">(<\/span><span class=\"n\">func<\/span><span class=\"p\">(<\/span><span class=\"o\">*<\/span><span class=\"n\">args<\/span><span class=\"p\">,<\/span> <span class=\"o\">**<\/span><span class=\"n\">kwargs<\/span><span class=\"p\">))<\/span>\n   <span class=\"k\">return<\/span> <span class=\"n\">func_wrapper<\/span>\n\n<span class=\"k\">class<\/span> <span class=\"nc\">Person<\/span><span class=\"p\">(<\/span><span class=\"nb\">object<\/span><span class=\"p\">):<\/span>\n    <span class=\"k\">def<\/span> <span class=\"fm\">__init__<\/span><span class=\"p\">(<\/span><span class=\"bp\">self<\/span><span class=\"p\">):<\/span>\n        <span class=\"bp\">self<\/span><span class=\"o\">.<\/span><span class=\"n\">name<\/span> <span class=\"o\">=<\/span> <span class=\"s2\">&quot;John&quot;<\/span>\n        <span class=\"bp\">self<\/span><span class=\"o\">.<\/span><span class=\"n\">family<\/span> <span class=\"o\">=<\/span> <span class=\"s2\">&quot;Doe&quot;<\/span>\n\n    <span class=\"nd\">@p_decorate<\/span>\n    <span class=\"k\">def<\/span> <span class=\"nf\">get_fullname<\/span><span class=\"p\">(<\/span><span class=\"bp\">self<\/span><span class=\"p\">):<\/span>\n        <span class=\"k\">return<\/span> <span class=\"bp\">self<\/span><span class=\"o\">.<\/span><span class=\"n\">name<\/span><span class=\"o\">+<\/span><span class=\"s2\">&quot; &quot;<\/span><span class=\"o\">+<\/span><span class=\"bp\">self<\/span><span class=\"o\">.<\/span><span class=\"n\">family<\/span>\n\n<span class=\"n\">my_person<\/span> <span class=\"o\">=<\/span> <span class=\"n\">Person<\/span><span class=\"p\">()<\/span>\n\n<span class=\"k\">print<\/span><span class=\"p\">(<\/span><span class=\"n\">my_person<\/span><span class=\"o\">.<\/span><span class=\"n\">get_fullname<\/span><span class=\"p\">())<\/span>\n<\/pre><\/div>\n\n\n<h2>Passing arguments to decorators<\/h2>\n<p>Looking back at the example before the one above, you can notice how redundant the decorators in the example are. 3 decorators(div_decorate, p_decorate, strong_decorate) each with the same functionality but wrapping the string with different tags. We can definitely do much better than that. Why not have a more general implementation for one that takes the tag to wrap with as a string? Yes please!<\/p>\n<div class=\"highlight\"><pre><span><\/span><span class=\"k\">def<\/span> <span class=\"nf\">tags<\/span><span class=\"p\">(<\/span><span class=\"n\">tag_name<\/span><span class=\"p\">):<\/span>\n    <span class=\"k\">def<\/span> <span class=\"nf\">tags_decorator<\/span><span class=\"p\">(<\/span><span class=\"n\">func<\/span><span class=\"p\">):<\/span>\n        <span class=\"k\">def<\/span> <span class=\"nf\">func_wrapper<\/span><span class=\"p\">(<\/span><span class=\"n\">name<\/span><span class=\"p\">):<\/span>\n            <span class=\"k\">return<\/span> <span class=\"s2\">&quot;&lt;{0}&gt;{1}&lt;\/{0}&gt;&quot;<\/span><span class=\"o\">.<\/span><span class=\"n\">format<\/span><span class=\"p\">(<\/span><span class=\"n\">tag_name<\/span><span class=\"p\">,<\/span> <span class=\"n\">func<\/span><span class=\"p\">(<\/span><span class=\"n\">name<\/span><span class=\"p\">))<\/span>\n        <span class=\"k\">return<\/span> <span class=\"n\">func_wrapper<\/span>\n    <span class=\"k\">return<\/span> <span class=\"n\">tags_decorator<\/span>\n\n<span class=\"nd\">@tags<\/span><span class=\"p\">(<\/span><span class=\"s2\">&quot;p&quot;<\/span><span class=\"p\">)<\/span>\n<span class=\"k\">def<\/span> <span class=\"nf\">get_text<\/span><span class=\"p\">(<\/span><span class=\"n\">name<\/span><span class=\"p\">):<\/span>\n    <span class=\"k\">return<\/span> <span class=\"s2\">&quot;Hello &quot;<\/span><span class=\"o\">+<\/span><span class=\"n\">name<\/span>\n\n<span class=\"k\">print<\/span><span class=\"p\">(<\/span><span class=\"n\">get_text<\/span><span class=\"p\">(<\/span><span class=\"s2\">&quot;John&quot;<\/span><span class=\"p\">))<\/span>\n\n<span class=\"c1\"># Outputs &lt;p&gt;Hello John&lt;\/p&gt;<\/span>\n<\/pre><\/div>\n\n\n<p>It took a bit more work in this case. Decorators expect to receive a function as an argument, that is why we will have to build a function that takes those extra arguments and generate our decorator on the fly. In the example above <strong>tags<\/strong>, is our decorator generator.<\/p>\n<h2>Debugging decorated functions<\/h2>\n<p>At the end of the day decorators are just wrapping our functions, in case of debugging that can be problematic since the wrapper function does not carry the name, module and docstring of the original function. Based on the example above if we do:<\/p>\n<div class=\"highlight\"><pre><span><\/span><span class=\"k\">print<\/span><span class=\"p\">(<\/span><span class=\"n\">get_text<\/span><span class=\"o\">.<\/span><span class=\"vm\">__name__<\/span><span class=\"p\">)<\/span>\n<span class=\"c1\"># Outputs func_wrapper<\/span>\n<\/pre><\/div>\n\n\n<p>The output was expected to be <strong>get_text<\/strong> yet, the attributes <code>__name__<\/code>, <code>__doc__<\/code>, and <code>__module__<\/code> of <strong>get_text<\/strong> got overridden by those of the wrapper(func_wrapper). Obviously we can reset them within func_wrapper but Python provides a much nicer way.<\/p>\n<h3>Functools to the rescue<\/h3>\n<p>Fortunately Python (as of version 2.5) includes the <a href=\"http:\/\/docs.python.org\/2\/library\/functools.html\">functools<\/a> module which contains <a href=\"http:\/\/docs.python.org\/2\/library\/functools.html#functools.wraps\">functools.wraps<\/a>. Wraps is a decorator for updating the attributes of the wrapping function(func_wrapper) to those of the original function(get_text). This is as simple as decorating func_wrapper by @wraps(func). Here is the updated example:<\/p>\n<div class=\"highlight\"><pre><span><\/span><span class=\"kn\">from<\/span> <span class=\"nn\">functools<\/span> <span class=\"kn\">import<\/span> <span class=\"n\">wraps<\/span>\n\n<span class=\"k\">def<\/span> <span class=\"nf\">tags<\/span><span class=\"p\">(<\/span><span class=\"n\">tag_name<\/span><span class=\"p\">):<\/span>\n    <span class=\"k\">def<\/span> <span class=\"nf\">tags_decorator<\/span><span class=\"p\">(<\/span><span class=\"n\">func<\/span><span class=\"p\">):<\/span>\n        <span class=\"nd\">@wraps<\/span><span class=\"p\">(<\/span><span class=\"n\">func<\/span><span class=\"p\">)<\/span>\n        <span class=\"k\">def<\/span> <span class=\"nf\">func_wrapper<\/span><span class=\"p\">(<\/span><span class=\"n\">name<\/span><span class=\"p\">):<\/span>\n            <span class=\"k\">return<\/span> <span class=\"s2\">&quot;&lt;{0}&gt;{1}&lt;\/{0}&gt;&quot;<\/span><span class=\"o\">.<\/span><span class=\"n\">format<\/span><span class=\"p\">(<\/span><span class=\"n\">tag_name<\/span><span class=\"p\">,<\/span> <span class=\"n\">func<\/span><span class=\"p\">(<\/span><span class=\"n\">name<\/span><span class=\"p\">))<\/span>\n        <span class=\"k\">return<\/span> <span class=\"n\">func_wrapper<\/span>\n    <span class=\"k\">return<\/span> <span class=\"n\">tags_decorator<\/span>\n\n<span class=\"nd\">@tags<\/span><span class=\"p\">(<\/span><span class=\"s2\">&quot;p&quot;<\/span><span class=\"p\">)<\/span>\n<span class=\"k\">def<\/span> <span class=\"nf\">get_text<\/span><span class=\"p\">(<\/span><span class=\"n\">name<\/span><span class=\"p\">):<\/span>\n    <span class=\"sd\">&quot;&quot;&quot;returns some text&quot;&quot;&quot;<\/span>\n    <span class=\"k\">return<\/span> <span class=\"s2\">&quot;Hello &quot;<\/span><span class=\"o\">+<\/span><span class=\"n\">name<\/span>\n\n<span class=\"k\">print<\/span><span class=\"p\">(<\/span><span class=\"n\">get_text<\/span><span class=\"o\">.<\/span><span class=\"vm\">__name__<\/span><span class=\"p\">)<\/span> <span class=\"c1\"># get_text<\/span>\n<span class=\"k\">print<\/span><span class=\"p\">(<\/span><span class=\"n\">get_text<\/span><span class=\"o\">.<\/span><span class=\"vm\">__doc__<\/span><span class=\"p\">)<\/span> <span class=\"c1\"># returns some text<\/span>\n<span class=\"k\">print<\/span><span class=\"p\">(<\/span><span class=\"n\">get_text<\/span><span class=\"o\">.<\/span><span class=\"vm\">__module__<\/span><span class=\"p\">)<\/span> <span class=\"c1\"># __main__<\/span>\n<\/pre><\/div>\n\n\n<p>You can notice from the output that the attributes of get_text are the correct ones now.<\/p>\n<h2>Where to use decorators<\/h2>\n<p>The examples in this post are pretty simple relative to how much you can do with decorators. They can give so much power and elegance to your program. In general, decorators are ideal for extending the behavior of functions that we don't want to modify. For a great list of useful decorators I suggest you check out the <a href=\"https:\/\/wiki.python.org\/moin\/PythonDecoratorLibrary\">Python Decorator Library<\/a><\/p>\n<h2>More reading resources<\/h2>\n<p>Here is a list of other resources worth checking out:<\/p>\n<ul>\n<li><a href=\"https:\/\/wiki.python.org\/moin\/PythonDecorators#What_is_a_Decorator\">What is a decorator?<\/a><\/li>\n<li><a href=\"http:\/\/www.artima.com\/weblogs\/viewpost.jsp?thread=240808\">Decorators I: Introduction to Python Decorators<\/a><\/li>\n<li><a href=\"http:\/\/www.artima.com\/weblogs\/viewpost.jsp?thread=240845\">Python Decorators II: Decorator Arguments<\/a><\/li>\n<li><a href=\"http:\/\/www.artima.com\/weblogs\/viewpost.jsp?thread=241209\">Python Decorators III: A Decorator-Based Build System<\/a><\/li>\n<li><a href=\"http:\/\/www.amazon.com\/gp\/product\/B006ZHJSIM\/ref=as_li_ss_tl?ie=UTF8&amp;camp=1789&amp;creative=390957&amp;creativeASIN=B006ZHJSIM&amp;linkCode=as2&amp;tag=thcosh00-20\">Guide to: Learning Python Decorators by Matt Harrison<\/a><img alt=\"\" src=\"http:\/\/ir-na.amazon-adsystem.com\/e\/ir?t=thcosh00-20&amp;l=as2&amp;o=1&amp;a=B006ZHJSIM\"><\/li>\n<\/ul>\n<h2>Phew!<\/h2>\n<p>That was an introduction to the concept of decorators in Python. I hope that you found this post helpful, if you have any suggestions or questions please do share them in the comments. Happy coding!<\/p>","category":[{"@attributes":{"term":"python"}},{"@attributes":{"term":"patterns"}},{"@attributes":{"term":"decorators"}},{"@attributes":{"term":"syntax"}},{"@attributes":{"term":"functions"}}]},{"title":"An alternative to Javascript's evil setInterval","link":{"@attributes":{"href":"https:\/\/www.thecodeship.com\/web-development\/alternative-to-javascript-evil-setinterval\/","rel":"alternate"}},"published":"2013-11-03T12:00:00+02:00","updated":"2013-11-03T12:00:00+02:00","author":{"name":"Ayman Farhat"},"id":"tag:www.thecodeship.com,2013-11-03:\/web-development\/alternative-to-javascript-evil-setinterval\/","summary":"<p>If you use Javascript often you might have come across the need for delaying the execution of code within intervals of time with repetition. This is were the native setInterval function comes to use. Despite being useful in many applications, setInterval's shortcomings could blow up your application in some ...<\/p>","content":"<p>If you use Javascript often you might have come across the need for delaying the execution of code within intervals of time with repetition. This is were the native setInterval function comes to use. Despite being useful in many applications, setInterval's shortcomings could blow up your application in some circumstances.<\/p>\n<h2>Why is setInterval evil?<\/h2>\n<h3>Doesn't care whether the callback is still running or not<\/h3>\n<p>In some cases, the function might need longer than the interval time to finish execution. Consider using setInterval to poll a remote server every 5 seconds, network latency, unresponsive server and other variables might prevent the request from completing on time. What would happen is that you'll end up with a bunch of queued requests that may not necessarily return in order.<\/p>\n<h3>Ignores errors<\/h3>\n<p>If for some reason, an error occurs in part of the code that is called by setInterval, it wouldn't break and keeps executing the faulty code.<\/p>\n<h3>Isn't that flexible<\/h3>\n<p>On top of the previously mentioned shortcomings, I'd really appreciate if setInterval had an optional parameter for setting the number of repetitions instead of repeating infinitely.<\/p>\n<h2>A better implementation<\/h2>\n<p>So I implemented my own setInterval function which tackles the above mentioned shortcomings. It supports the expected setInterval parameters with an extra optional one for specifying the number of repetitions to be done.<\/p>\n<div class=\"highlight\"><pre><span><\/span><span class=\"kd\">function<\/span> <span class=\"nx\">interval<\/span><span class=\"p\">(<\/span><span class=\"nx\">func<\/span><span class=\"p\">,<\/span> <span class=\"nx\">wait<\/span><span class=\"p\">,<\/span> <span class=\"nx\">times<\/span><span class=\"p\">){<\/span>\n    <span class=\"kd\">var<\/span> <span class=\"nx\">interv<\/span> <span class=\"o\">=<\/span> <span class=\"kd\">function<\/span><span class=\"p\">(<\/span><span class=\"nx\">w<\/span><span class=\"p\">,<\/span> <span class=\"nx\">t<\/span><span class=\"p\">){<\/span>\n        <span class=\"k\">return<\/span> <span class=\"kd\">function<\/span><span class=\"p\">(){<\/span>\n            <span class=\"k\">if<\/span><span class=\"p\">(<\/span><span class=\"k\">typeof<\/span> <span class=\"nx\">t<\/span> <span class=\"o\">===<\/span> <span class=\"s2\">&quot;undefined&quot;<\/span> <span class=\"o\">||<\/span> <span class=\"nx\">t<\/span><span class=\"o\">--<\/span> <span class=\"o\">&gt;<\/span> <span class=\"mi\">0<\/span><span class=\"p\">){<\/span>\n                <span class=\"nx\">setTimeout<\/span><span class=\"p\">(<\/span><span class=\"nx\">interv<\/span><span class=\"p\">,<\/span> <span class=\"nx\">w<\/span><span class=\"p\">);<\/span>\n                <span class=\"k\">try<\/span><span class=\"p\">{<\/span>\n                    <span class=\"nx\">func<\/span><span class=\"p\">.<\/span><span class=\"nx\">call<\/span><span class=\"p\">(<\/span><span class=\"kc\">null<\/span><span class=\"p\">);<\/span>\n                <span class=\"p\">}<\/span>\n                <span class=\"k\">catch<\/span><span class=\"p\">(<\/span><span class=\"nx\">e<\/span><span class=\"p\">){<\/span>\n                    <span class=\"nx\">t<\/span> <span class=\"o\">=<\/span> <span class=\"mi\">0<\/span><span class=\"p\">;<\/span>\n                    <span class=\"k\">throw<\/span> <span class=\"nx\">e<\/span><span class=\"p\">.<\/span><span class=\"nx\">toString<\/span><span class=\"p\">();<\/span>\n                <span class=\"p\">}<\/span>\n            <span class=\"p\">}<\/span>\n        <span class=\"p\">};<\/span>\n    <span class=\"p\">}(<\/span><span class=\"nx\">wait<\/span><span class=\"p\">,<\/span> <span class=\"nx\">times<\/span><span class=\"p\">);<\/span>\n\n    <span class=\"nx\">setTimeout<\/span><span class=\"p\">(<\/span><span class=\"nx\">interv<\/span><span class=\"p\">,<\/span> <span class=\"nx\">wait<\/span><span class=\"p\">);<\/span>\n<span class=\"p\">};<\/span>\n<\/pre><\/div>\n\n\n<p>The interval function has an internal function called interv which gets invoked automatically via setTimeout, within <strong>interv<\/strong> is a <a href=\"https:\/\/developer.mozilla.org\/en-US\/docs\/Web\/JavaScript\/Guide\/Closures\">closure<\/a> that checks the the repetition times, invokes the callback and invokes interv again via setTimeout. In case an exception bubbles up from the callback call the interval calling will stop and the exception will be thrown.<\/p>\n<p>This pattern does not guarantee execution on a fixed interval of course, yet it does guarantee that the previous interval has completed before recursing, which I think is much more important.<\/p>\n<h2>Usage<\/h2>\n<p>So to execute a piece of code 5 times with an interval or 10 seconds between each you would do something like this:<\/p>\n<div class=\"highlight\"><pre><span><\/span><span class=\"nx\">interval<\/span><span class=\"p\">(<\/span><span class=\"kd\">function<\/span><span class=\"p\">(){<\/span>\n        <span class=\"c1\">\/\/ Code block goes here<\/span>\n    <span class=\"p\">},<\/span> <span class=\"mi\">1000<\/span><span class=\"p\">,<\/span> <span class=\"mi\">10<\/span><span class=\"p\">);<\/span>\n<\/pre><\/div>\n\n\n<p>Hope you find this snippet useful, feel free to leave your comments below.<\/p>\n<h2>References<\/h2>\n<ul>\n<li><a href=\"http:\/\/ejohn.org\/blog\/how-javascript-timers-work\/\">How Javascript Timers Work<\/a><\/li>\n<li><a href=\"https:\/\/developer.mozilla.org\/en-US\/docs\/Web\/API\/window.setInterval\">window.setInterval<\/a><\/li>\n<li><a href=\"https:\/\/developer.mozilla.org\/en-US\/docs\/Web\/API\/window.setTimeout\">window.setTimeout<\/a><\/li>\n<\/ul>","category":[{"@attributes":{"term":"javascript"}},{"@attributes":{"term":"improvement"}},{"@attributes":{"term":"setinterval"}},{"@attributes":{"term":"settimeout"}}]},{"title":"Understanding the apt-cache depends Output","link":{"@attributes":{"href":"https:\/\/www.thecodeship.com\/gnu-linux\/understanding-apt-cache-depends-output\/","rel":"alternate"}},"published":"2013-08-18T05:00:00+03:00","updated":"2013-08-18T05:00:00+03:00","author":{"name":"Ayman Farhat"},"id":"tag:www.thecodeship.com,2013-08-18:\/gnu-linux\/understanding-apt-cache-depends-output\/","summary":"<p>I was using apt-cache in Ubuntu to get a list of dependencies for a certain package and parse the output programmatically, eventually I wanted to programatically download and package them within an archive for offline installs later on. I was not really sure about the exact meanings of the output ...<\/p>","content":"<p>I was using apt-cache in Ubuntu to get a list of dependencies for a certain package and parse the output programmatically, eventually I wanted to programatically download and package them within an archive for offline installs later on. I was not really sure about the exact meanings of the output given that as I have never really used \"apt-cache depends\" before.<\/p>\n<p>Example output for Python:<\/p>\n<div class=\"highlight\"><pre><span><\/span>$ apt-cache depends python\n\npython\n  Depends: python2.7\n  Depends: python-minimal\n  Suggests: python-doc\n  Suggests: python-tk\n  Conflicts: python-central\n  Breaks: \n  Breaks: \n  Breaks: \n  Breaks: update-manager-core\n  Replaces: python-dev\n<\/pre><\/div>\n\n\n<p>Another example output for Node:<\/p>\n<div class=\"highlight\"><pre><span><\/span>$ apt-cache depends node\n\nnode\n  Depends: libax25\n  Depends: libc6\n  Depends: zlib1g\n  Depends: update-inetd\n <span class=\"p\">|<\/span>Depends: openbsd-inetd\n  Depends: \n    openbsd-inetd\n    rlinetd\n    xinetd\n    inetutils-inetd\n<\/pre><\/div>\n\n\n<p>So I did some reading on the <a href=\"http:\/\/www.debian.org\/doc\/debian-policy\/\">debian policy manual<\/a> and got some answers. Here is a brief summary of what sinked in:<\/p>\n<p><strong>Recommends<\/strong> fields denote packages that are usually found with the target pacakage most of the time unless in unusual installations. Those packages are installed by apt by default with your target.<\/p>\n<p><strong>Suggests<\/strong> fields denote packages that may be useful with your target package but not necessarily required, for example the Python doc is not required for the functioning of Python on your machine but could be useful to you in one way or another.<\/p>\n<p><strong>Depends<\/strong> fields denote packages that are absolutely needed for the successful installation of your target package, they need to be installed and configured prior to your package installation.<\/p>\n<p><strong>Breaks<\/strong> fields denote packages that are already installed on your system yet \"break\" the process of your installation as dpkg will refuse upacking and installing your target packages unless the broken package gets deconfigured first. This usually happens in case of package incompatibility with a certain version specification. So the automatic solution would be to upgrade(deconfigure) all the packages listed within this field.<\/p>\n<p><strong>Conflicts<\/strong> fields denote packages that are already installed on your machine and declare an absolute incompatibility, those packages need to be uninstalled before installing the target package.<\/p>\n<p><strong>Replaces<\/strong> fields denote packages that are already installed on your machine and will have some of their files replaced by the target package to be installed.<\/p>\n<p>As for the |(pipe flag), between two packages in the list it denotes an \"or\" relationship between packages following each other. For example in the second example:<\/p>\n<div class=\"highlight\"><pre><span><\/span> <span class=\"n\">Depends<\/span><span class=\"p\">:<\/span> <span class=\"k\">update<\/span><span class=\"o\">-<\/span><span class=\"n\">inetd<\/span>\n <span class=\"o\">|<\/span><span class=\"n\">Depends<\/span><span class=\"p\">:<\/span> <span class=\"n\">openbsd<\/span><span class=\"o\">-<\/span><span class=\"n\">inetd<\/span>\n<\/pre><\/div>\n\n\n<p>update-inetd and openbsd-inetd are alternatives to each other yet the first one(update-inetd) is preferred.<\/p>\n<p>As for the package names within brackets those are known as virtual packages. A virtual package is a generic name that applies to a group of packages which provide similar basic functionality.For example if you check the second output example when listing the dependencies of Node we notice the following:<\/p>\n<div class=\"highlight\"><pre><span><\/span><span class=\"n\">Depends<\/span><span class=\"o\">:<\/span> \n    <span class=\"n\">openbsd<\/span><span class=\"o\">-<\/span><span class=\"n\">inetd<\/span>\n    <span class=\"n\">rlinetd<\/span>\n    <span class=\"n\">xinetd<\/span>\n    <span class=\"n\">inetutils<\/span><span class=\"o\">-<\/span><span class=\"n\">inetd<\/span>\n<\/pre><\/div>\n\n\n<p>In this case, \"inet-superserver\" is name of a virtual package(which isn't a real package) under which falls a list of real packages which equally satisfy the \u201cInternet Super-Server\u201d functionality\/dependency that is needed for Node to work properly. Thus installing any of those options would be enough.<\/p>\n<p>I hope you have found this simple summary helpful. So much more information can be found within the <a href=\"http:\/\/www.debian.org\/doc\/debian-policy\/ch-relationships.html\">Debian policy manual chapter 7<\/a><\/p>","category":[{"@attributes":{"term":"output"}},{"@attributes":{"term":"cache"}},{"@attributes":{"term":"apt"}},{"@attributes":{"term":"linux"}}]},{"title":"Proxypy: Cross Domain Javascript Requests with Python","link":{"@attributes":{"href":"https:\/\/www.thecodeship.com\/web-development\/proxypy-cross-domain-javascript-requests-python\/","rel":"alternate"}},"published":"2013-06-08T17:00:00+03:00","updated":"2013-06-08T17:00:00+03:00","author":{"name":"Ayman Farhat"},"id":"tag:www.thecodeship.com,2013-06-08:\/web-development\/proxypy-cross-domain-javascript-requests-python\/","summary":"<p>When working with web applications you might run into the need to fetch remote content via Ajax where that content isn't necessarily JSON, such as HTML or config file, or even request JSON content from a remote server that doesn't support JSONP. This is when setting up a ...<\/p>","content":"<p>When working with web applications you might run into the need to fetch remote content via Ajax where that content isn't necessarily JSON, such as HTML or config file, or even request JSON content from a remote server that doesn't support JSONP. This is when setting up a proxy service can be a good idea.<\/p>\n<p>A proxy can act as an intermediary between your Javascript and the remote data, eliminating all the cross domain limitations imposed on the client. The pattern in principle is simple:<\/p>\n<ol>\n<li>Assign to your web application's server some view to receive a request with a parameter of the desired remote content address.<\/li>\n<li>Let the server fetch the content from the remote server via an HTTP request.<\/li>\n<li>Wrap the results into a JSON object and return them back.<\/li>\n<\/ol>\n<p>I was working with Python so I wrote a simple proxy module that can work with any Python web application framework. You can check out the <a href=\"https:\/\/github.com\/aymanfarhat\/proxypy\/blob\/master\/proxypy.py\">source of the module<\/a> on <a href=\"https:\/\/github.com\/aymanfarhat\/proxypy\">Github<\/a> along with an example of a <a href=\"https:\/\/github.com\/aymanfarhat\/proxypy\/tree\/master\/example\">Flask application<\/a> using it.<\/p>\n<p>Include the Proxypy module in your project and import it then assign a view to receive the request and call <code>proxypy.<\/code>get<code>(query_string)<\/code> the method takes a single argument that is the query string received in the request. The query string would work on the following parameters:<\/p>\n<ul>\n<li><strong>url<\/strong> which is the address of the page with the target content to fetch.<\/li>\n<li><strong>headers<\/strong>(optional) if you wish to return the reply headers of the request.<\/li>\n<li><strong>callback<\/strong>(optional) if your request against the proxy service itself is from a different domain.<\/li>\n<\/ul>\n<p>Here is a simple demonstration of usage with <a href=\"http:\/\/flask.pocoo.org\/\">Flask<\/a>, the \"crossdomain\" view receives the request and calls proxypy's get method to fetch the remote content based on the supplied parameters and then returns the JSON string containing the wrapped reply from the remote.<\/p>\n<div class=\"highlight\"><pre><span><\/span><span class=\"kn\">from<\/span> <span class=\"nn\">flask<\/span> <span class=\"kn\">import<\/span> <span class=\"n\">Flask<\/span><span class=\"p\">,<\/span> <span class=\"n\">request<\/span><span class=\"p\">,<\/span> <span class=\"n\">Response<\/span><span class=\"p\">,<\/span> <span class=\"n\">render_template<\/span>\n<span class=\"kn\">import<\/span> <span class=\"nn\">json<\/span>\n<span class=\"kn\">import<\/span> <span class=\"nn\">proxypy<\/span>\n\n<span class=\"n\">app<\/span> <span class=\"o\">=<\/span> <span class=\"n\">Flask<\/span><span class=\"p\">(<\/span><span class=\"vm\">__name__<\/span><span class=\"p\">)<\/span>\n\n<span class=\"nd\">@app.route<\/span><span class=\"p\">(<\/span><span class=\"s2\">&quot;\/&quot;<\/span><span class=\"p\">)<\/span>\n<span class=\"k\">def<\/span> <span class=\"nf\">index<\/span><span class=\"p\">():<\/span>\n    <span class=\"k\">return<\/span> <span class=\"n\">render_template<\/span><span class=\"p\">(<\/span><span class=\"s2\">&quot;index.html&quot;<\/span><span class=\"p\">)<\/span>\n\n<span class=\"nd\">@app.route<\/span><span class=\"p\">(<\/span><span class=\"s2\">&quot;\/crossdomain&quot;<\/span><span class=\"p\">)<\/span>\n<span class=\"k\">def<\/span> <span class=\"nf\">crossdom<\/span><span class=\"p\">():<\/span>\n    <span class=\"n\">reply<\/span> <span class=\"o\">=<\/span> <span class=\"n\">proxypy<\/span><span class=\"o\">.<\/span><span class=\"n\">get<\/span><span class=\"p\">(<\/span><span class=\"n\">request<\/span><span class=\"o\">.<\/span><span class=\"n\">query_string<\/span><span class=\"p\">)<\/span>\n    <span class=\"k\">return<\/span> <span class=\"n\">Response<\/span><span class=\"p\">(<\/span><span class=\"n\">reply<\/span><span class=\"p\">,<\/span><span class=\"n\">status<\/span><span class=\"o\">=<\/span><span class=\"mi\">200<\/span><span class=\"p\">,<\/span><span class=\"n\">mimetype<\/span><span class=\"o\">=<\/span><span class=\"s1\">&#39;application\/json&#39;<\/span><span class=\"p\">)<\/span>\n\n<span class=\"k\">if<\/span> <span class=\"vm\">__name__<\/span> <span class=\"o\">==<\/span> <span class=\"s2\">&quot;__main__&quot;<\/span><span class=\"p\">:<\/span>\n    <span class=\"n\">app<\/span><span class=\"o\">.<\/span><span class=\"n\">run<\/span><span class=\"p\">(<\/span><span class=\"n\">debug<\/span><span class=\"o\">=<\/span><span class=\"bp\">True<\/span><span class=\"p\">)<\/span>\n<\/pre><\/div>\n\n\n<p>You can view a full example in action <a href=\"http:\/\/proxypy.herokuapp.com\/\">here<\/a> and check the full source of the script on <a href=\"https:\/\/github.com\/aymanfarhat\/proxypy\">Github<\/a>.<\/p>\n<p>You might have noticed a couple of limitations in terms of features in this proxy namely, no support for HTTP methods other than GET and this is because I haven't really came across a need for that plus allowing more methods such as POST may increase the possibilities of exploitation.<\/p>\n<p>I hope that you find this module useful in one way or another, would love to see your feedback about this topic in the comments. Have a nice weekend!<\/p>","category":[{"@attributes":{"term":"javascript"}},{"@attributes":{"term":"python"}},{"@attributes":{"term":"cross-domain"}}]},{"title":"Javascript URL Object","link":{"@attributes":{"href":"https:\/\/www.thecodeship.com\/web-development\/javascript-url-object\/","rel":"alternate"}},"published":"2013-05-20T11:00:00+03:00","updated":"2013-05-20T11:00:00+03:00","author":{"name":"Ayman Farhat"},"id":"tag:www.thecodeship.com,2013-05-20:\/web-development\/javascript-url-object\/","summary":"<p>I've been doing some operations on URLs in Javascript where I needed to extract different parts from a string URL, such as the domain, port, and most importantly the parameter values. I really wished there was some function which takes a string URL and breaks it down into an ...<\/p>","content":"<p>I've been doing some operations on URLs in Javascript where I needed to extract different parts from a string URL, such as the domain, port, and most importantly the parameter values. I really wished there was some function which takes a string URL and breaks it down into an object of accessible properties automatically. What I particularly wanted in such a data structure is to have easy access to the URL's parameter values as key value pairs and store a parameters multiple values automatically into an array too.<\/p>\n<p>So I ended up writing my own function for generating such a structure for a string URL. In this implementation, there are 3 optional parameters:<\/p>\n<ul>\n<li>The URL string, if not supplied the value defaults to the current window's location.<\/li>\n<li>An unescape option to remove any escaping(for example space escape) from the parameter values, is true by default.<\/li>\n<li>A convert to number option, for converting any numeric values in the parameters from string to int or float, it true by default.<\/li>\n<\/ul>\n<p>Getting most of the URL values was done through a nice trick which is to add a hyperlink element to the DOM and theN access the values via the hyperlink object by calling its generated attributes, namely: protocol, hostname, port, hash, pathname, and search. Calling the search attribute of the URL would return a string of the parameters part of the URL, from there the job was to parse that string and its values and generate a set of key(parameter name) value(parameter value) pairs representing it. <\/p>\n<p>Here is the full implementation:<\/p>\n<div class=\"highlight\"><pre><span><\/span><span class=\"kd\">function<\/span> <span class=\"nx\">urlObject<\/span><span class=\"p\">(<\/span><span class=\"nx\">options<\/span><span class=\"p\">)<\/span> <span class=\"p\">{<\/span>\n    <span class=\"s2\">&quot;use strict&quot;<\/span><span class=\"p\">;<\/span>\n    <span class=\"cm\">\/*global window, document*\/<\/span>\n\n    <span class=\"kd\">var<\/span> <span class=\"nx\">url_search_arr<\/span><span class=\"p\">,<\/span>\n        <span class=\"nx\">option_key<\/span><span class=\"p\">,<\/span>\n        <span class=\"nx\">i<\/span><span class=\"p\">,<\/span>\n        <span class=\"nx\">urlObj<\/span><span class=\"p\">,<\/span>\n        <span class=\"nx\">get_param<\/span><span class=\"p\">,<\/span>\n        <span class=\"nx\">key<\/span><span class=\"p\">,<\/span>\n        <span class=\"nx\">val<\/span><span class=\"p\">,<\/span>\n        <span class=\"nx\">url_query<\/span><span class=\"p\">,<\/span>\n        <span class=\"nx\">url_get_params<\/span> <span class=\"o\">=<\/span> <span class=\"p\">{},<\/span>\n        <span class=\"nx\">a<\/span> <span class=\"o\">=<\/span> <span class=\"nb\">document<\/span><span class=\"p\">.<\/span><span class=\"nx\">createElement<\/span><span class=\"p\">(<\/span><span class=\"s1\">&#39;a&#39;<\/span><span class=\"p\">),<\/span>\n        <span class=\"nx\">default_options<\/span> <span class=\"o\">=<\/span> <span class=\"p\">{<\/span>\n            <span class=\"s1\">&#39;url&#39;<\/span><span class=\"o\">:<\/span> <span class=\"nb\">window<\/span><span class=\"p\">.<\/span><span class=\"nx\">location<\/span><span class=\"p\">.<\/span><span class=\"nx\">href<\/span><span class=\"p\">,<\/span>\n            <span class=\"s1\">&#39;unescape&#39;<\/span><span class=\"o\">:<\/span> <span class=\"kc\">true<\/span><span class=\"p\">,<\/span>\n            <span class=\"s1\">&#39;convert_num&#39;<\/span><span class=\"o\">:<\/span> <span class=\"kc\">true<\/span>\n        <span class=\"p\">};<\/span>\n\n    <span class=\"k\">if<\/span> <span class=\"p\">(<\/span><span class=\"k\">typeof<\/span> <span class=\"nx\">options<\/span> <span class=\"o\">!==<\/span> <span class=\"s2\">&quot;object&quot;<\/span><span class=\"p\">)<\/span> <span class=\"p\">{<\/span>\n        <span class=\"nx\">options<\/span> <span class=\"o\">=<\/span> <span class=\"nx\">default_options<\/span><span class=\"p\">;<\/span>\n    <span class=\"p\">}<\/span> <span class=\"k\">else<\/span> <span class=\"p\">{<\/span>\n        <span class=\"k\">for<\/span> <span class=\"p\">(<\/span><span class=\"nx\">option_key<\/span> <span class=\"k\">in<\/span> <span class=\"nx\">default_options<\/span><span class=\"p\">)<\/span> <span class=\"p\">{<\/span>\n            <span class=\"k\">if<\/span> <span class=\"p\">(<\/span><span class=\"nx\">default_options<\/span><span class=\"p\">.<\/span><span class=\"nx\">hasOwnProperty<\/span><span class=\"p\">(<\/span><span class=\"nx\">option_key<\/span><span class=\"p\">))<\/span> <span class=\"p\">{<\/span>\n                <span class=\"k\">if<\/span> <span class=\"p\">(<\/span><span class=\"nx\">options<\/span><span class=\"p\">[<\/span><span class=\"nx\">option_key<\/span><span class=\"p\">]<\/span> <span class=\"o\">===<\/span> <span class=\"kc\">undefined<\/span><span class=\"p\">)<\/span> <span class=\"p\">{<\/span>\n                    <span class=\"nx\">options<\/span><span class=\"p\">[<\/span><span class=\"nx\">option_key<\/span><span class=\"p\">]<\/span> <span class=\"o\">=<\/span> <span class=\"nx\">default_options<\/span><span class=\"p\">[<\/span><span class=\"nx\">option_key<\/span><span class=\"p\">];<\/span>\n                <span class=\"p\">}<\/span>\n            <span class=\"p\">}<\/span>\n        <span class=\"p\">}<\/span>\n    <span class=\"p\">}<\/span>\n\n    <span class=\"nx\">a<\/span><span class=\"p\">.<\/span><span class=\"nx\">href<\/span> <span class=\"o\">=<\/span> <span class=\"nx\">options<\/span><span class=\"p\">.<\/span><span class=\"nx\">url<\/span><span class=\"p\">;<\/span>\n    <span class=\"nx\">url_query<\/span> <span class=\"o\">=<\/span> <span class=\"nx\">a<\/span><span class=\"p\">.<\/span><span class=\"nx\">search<\/span><span class=\"p\">.<\/span><span class=\"nx\">substring<\/span><span class=\"p\">(<\/span><span class=\"mi\">1<\/span><span class=\"p\">);<\/span>\n    <span class=\"nx\">url_search_arr<\/span> <span class=\"o\">=<\/span> <span class=\"nx\">url_query<\/span><span class=\"p\">.<\/span><span class=\"nx\">split<\/span><span class=\"p\">(<\/span><span class=\"s1\">&#39;&amp;&#39;<\/span><span class=\"p\">);<\/span>\n\n    <span class=\"k\">if<\/span> <span class=\"p\">(<\/span><span class=\"nx\">url_search_arr<\/span><span class=\"p\">[<\/span><span class=\"mi\">0<\/span><span class=\"p\">].<\/span><span class=\"nx\">length<\/span> <span class=\"o\">&gt;<\/span> <span class=\"mi\">1<\/span><span class=\"p\">)<\/span> <span class=\"p\">{<\/span>\n        <span class=\"k\">for<\/span> <span class=\"p\">(<\/span><span class=\"nx\">i<\/span> <span class=\"o\">=<\/span> <span class=\"mi\">0<\/span><span class=\"p\">;<\/span> <span class=\"nx\">i<\/span> <span class=\"o\">&lt;<\/span> <span class=\"nx\">url_search_arr<\/span><span class=\"p\">.<\/span><span class=\"nx\">length<\/span><span class=\"p\">;<\/span> <span class=\"nx\">i<\/span> <span class=\"o\">+=<\/span> <span class=\"mi\">1<\/span><span class=\"p\">)<\/span> <span class=\"p\">{<\/span>\n            <span class=\"nx\">get_param<\/span> <span class=\"o\">=<\/span> <span class=\"nx\">url_search_arr<\/span><span class=\"p\">[<\/span><span class=\"nx\">i<\/span><span class=\"p\">].<\/span><span class=\"nx\">split<\/span><span class=\"p\">(<\/span><span class=\"s2\">&quot;=&quot;<\/span><span class=\"p\">);<\/span>\n\n            <span class=\"k\">if<\/span> <span class=\"p\">(<\/span><span class=\"nx\">options<\/span><span class=\"p\">.<\/span><span class=\"nx\">unescape<\/span><span class=\"p\">)<\/span> <span class=\"p\">{<\/span>\n                <span class=\"nx\">key<\/span> <span class=\"o\">=<\/span> <span class=\"nb\">decodeURI<\/span><span class=\"p\">(<\/span><span class=\"nx\">get_param<\/span><span class=\"p\">[<\/span><span class=\"mi\">0<\/span><span class=\"p\">]);<\/span>\n                <span class=\"nx\">val<\/span> <span class=\"o\">=<\/span> <span class=\"nb\">decodeURI<\/span><span class=\"p\">(<\/span><span class=\"nx\">get_param<\/span><span class=\"p\">[<\/span><span class=\"mi\">1<\/span><span class=\"p\">]);<\/span>\n            <span class=\"p\">}<\/span> <span class=\"k\">else<\/span> <span class=\"p\">{<\/span>\n                <span class=\"nx\">key<\/span> <span class=\"o\">=<\/span> <span class=\"nx\">get_param<\/span><span class=\"p\">[<\/span><span class=\"mi\">0<\/span><span class=\"p\">];<\/span>\n                <span class=\"nx\">val<\/span> <span class=\"o\">=<\/span> <span class=\"nx\">get_param<\/span><span class=\"p\">[<\/span><span class=\"mi\">1<\/span><span class=\"p\">];<\/span>\n            <span class=\"p\">}<\/span>\n\n            <span class=\"k\">if<\/span> <span class=\"p\">(<\/span><span class=\"nx\">options<\/span><span class=\"p\">.<\/span><span class=\"nx\">convert_num<\/span><span class=\"p\">)<\/span> <span class=\"p\">{<\/span>\n                <span class=\"k\">if<\/span> <span class=\"p\">(<\/span><span class=\"nx\">val<\/span><span class=\"p\">.<\/span><span class=\"nx\">match<\/span><span class=\"p\">(<\/span><span class=\"sr\">\/^\\d+$\/<\/span><span class=\"p\">))<\/span> <span class=\"p\">{<\/span>\n                    <span class=\"nx\">val<\/span> <span class=\"o\">=<\/span> <span class=\"nb\">parseInt<\/span><span class=\"p\">(<\/span><span class=\"nx\">val<\/span><span class=\"p\">,<\/span> <span class=\"mi\">10<\/span><span class=\"p\">);<\/span>\n                <span class=\"p\">}<\/span> <span class=\"k\">else<\/span> <span class=\"k\">if<\/span> <span class=\"p\">(<\/span><span class=\"nx\">val<\/span><span class=\"p\">.<\/span><span class=\"nx\">match<\/span><span class=\"p\">(<\/span><span class=\"sr\">\/^\\d+\\.\\d+$\/<\/span><span class=\"p\">))<\/span> <span class=\"p\">{<\/span>\n                    <span class=\"nx\">val<\/span> <span class=\"o\">=<\/span> <span class=\"nb\">parseFloat<\/span><span class=\"p\">(<\/span><span class=\"nx\">val<\/span><span class=\"p\">);<\/span>\n                <span class=\"p\">}<\/span>\n            <span class=\"p\">}<\/span>\n\n            <span class=\"k\">if<\/span> <span class=\"p\">(<\/span><span class=\"nx\">url_get_params<\/span><span class=\"p\">[<\/span><span class=\"nx\">key<\/span><span class=\"p\">]<\/span> <span class=\"o\">===<\/span> <span class=\"kc\">undefined<\/span><span class=\"p\">)<\/span> <span class=\"p\">{<\/span>\n                <span class=\"nx\">url_get_params<\/span><span class=\"p\">[<\/span><span class=\"nx\">key<\/span><span class=\"p\">]<\/span> <span class=\"o\">=<\/span> <span class=\"nx\">val<\/span><span class=\"p\">;<\/span>\n            <span class=\"p\">}<\/span> <span class=\"k\">else<\/span> <span class=\"k\">if<\/span> <span class=\"p\">(<\/span><span class=\"k\">typeof<\/span> <span class=\"nx\">url_get_params<\/span><span class=\"p\">[<\/span><span class=\"nx\">key<\/span><span class=\"p\">]<\/span> <span class=\"o\">===<\/span> <span class=\"s2\">&quot;string&quot;<\/span><span class=\"p\">)<\/span> <span class=\"p\">{<\/span>\n                <span class=\"nx\">url_get_params<\/span><span class=\"p\">[<\/span><span class=\"nx\">key<\/span><span class=\"p\">]<\/span> <span class=\"o\">=<\/span> <span class=\"p\">[<\/span><span class=\"nx\">url_get_params<\/span><span class=\"p\">[<\/span><span class=\"nx\">key<\/span><span class=\"p\">],<\/span> <span class=\"nx\">val<\/span><span class=\"p\">];<\/span>\n            <span class=\"p\">}<\/span> <span class=\"k\">else<\/span> <span class=\"p\">{<\/span>\n                <span class=\"nx\">url_get_params<\/span><span class=\"p\">[<\/span><span class=\"nx\">key<\/span><span class=\"p\">].<\/span><span class=\"nx\">push<\/span><span class=\"p\">(<\/span><span class=\"nx\">val<\/span><span class=\"p\">);<\/span>\n            <span class=\"p\">}<\/span>\n\n            <span class=\"nx\">get_param<\/span> <span class=\"o\">=<\/span> <span class=\"p\">[];<\/span>\n        <span class=\"p\">}<\/span>\n    <span class=\"p\">}<\/span>\n\n    <span class=\"nx\">urlObj<\/span> <span class=\"o\">=<\/span> <span class=\"p\">{<\/span>\n        <span class=\"nx\">protocol<\/span><span class=\"o\">:<\/span> <span class=\"nx\">a<\/span><span class=\"p\">.<\/span><span class=\"nx\">protocol<\/span><span class=\"p\">,<\/span>\n        <span class=\"nx\">hostname<\/span><span class=\"o\">:<\/span> <span class=\"nx\">a<\/span><span class=\"p\">.<\/span><span class=\"nx\">hostname<\/span><span class=\"p\">,<\/span>\n        <span class=\"nx\">host<\/span><span class=\"o\">:<\/span> <span class=\"nx\">a<\/span><span class=\"p\">.<\/span><span class=\"nx\">host<\/span><span class=\"p\">,<\/span>\n        <span class=\"nx\">port<\/span><span class=\"o\">:<\/span> <span class=\"nx\">a<\/span><span class=\"p\">.<\/span><span class=\"nx\">port<\/span><span class=\"p\">,<\/span>\n        <span class=\"nx\">hash<\/span><span class=\"o\">:<\/span> <span class=\"nx\">a<\/span><span class=\"p\">.<\/span><span class=\"nx\">hash<\/span><span class=\"p\">.<\/span><span class=\"nx\">substr<\/span><span class=\"p\">(<\/span><span class=\"mi\">1<\/span><span class=\"p\">),<\/span>\n        <span class=\"nx\">pathname<\/span><span class=\"o\">:<\/span> <span class=\"nx\">a<\/span><span class=\"p\">.<\/span><span class=\"nx\">pathname<\/span><span class=\"p\">,<\/span>\n        <span class=\"nx\">search<\/span><span class=\"o\">:<\/span> <span class=\"nx\">a<\/span><span class=\"p\">.<\/span><span class=\"nx\">search<\/span><span class=\"p\">,<\/span>\n        <span class=\"nx\">parameters<\/span><span class=\"o\">:<\/span> <span class=\"nx\">url_get_params<\/span>\n    <span class=\"p\">};<\/span>\n\n    <span class=\"k\">return<\/span> <span class=\"nx\">urlObj<\/span><span class=\"p\">;<\/span>\n<span class=\"p\">}<\/span>\n<\/pre><\/div>\n\n\n<p>And here is sample usage with a sample output:<\/p>\n<div class=\"highlight\"><pre><span><\/span><span class=\"nt\">urlObject<\/span><span class=\"o\">(<\/span><span class=\"p\">{<\/span><span class=\"err\">&#39;url&#39;:&#39;<\/span><span class=\"n\">http<\/span><span class=\"p\">:<\/span><span class=\"o\">\/\/<\/span><span class=\"n\">localhost<\/span><span class=\"o\">.<\/span><span class=\"n\">test<\/span><span class=\"o\">?<\/span><span class=\"n\">name<\/span><span class=\"o\">=<\/span><span class=\"n\">ayman<\/span><span class=\"o\">&amp;<\/span><span class=\"n\">age<\/span><span class=\"o\">=<\/span><span class=\"mi\">22<\/span><span class=\"o\">&amp;<\/span><span class=\"n\">gpa<\/span><span class=\"o\">=<\/span><span class=\"mf\">3.5<\/span><span class=\"o\">&amp;<\/span><span class=\"n\">course<\/span><span class=\"o\">=<\/span><span class=\"n\">programming<\/span><span class=\"o\">&amp;<\/span><span class=\"n\">course<\/span><span class=\"o\">=<\/span><span class=\"n\">mathematics<\/span><span class=\"o\">&amp;<\/span><span class=\"n\">course<\/span><span class=\"o\">=<\/span><span class=\"n\">algorithms<\/span><span class=\"err\">&#39;<\/span><span class=\"p\">}<\/span><span class=\"o\">);<\/span>\n<\/pre><\/div>\n\n\n<div class=\"highlight\"><pre><span><\/span><span class=\"err\">{<\/span>\n  <span class=\"ss\">&quot;protocol&quot;<\/span><span class=\"p\">:<\/span> <span class=\"ss\">&quot;http:&quot;<\/span><span class=\"p\">,<\/span>\n  <span class=\"ss\">&quot;hostname&quot;<\/span><span class=\"p\">:<\/span> <span class=\"ss\">&quot;localhost.test&quot;<\/span><span class=\"p\">,<\/span>\n  <span class=\"ss\">&quot;host&quot;<\/span><span class=\"p\">:<\/span> <span class=\"ss\">&quot;localhost.test&quot;<\/span><span class=\"p\">,<\/span>\n  <span class=\"ss\">&quot;port&quot;<\/span><span class=\"p\">:<\/span> <span class=\"ss\">&quot;&quot;<\/span><span class=\"p\">,<\/span>\n  <span class=\"ss\">&quot;hash&quot;<\/span><span class=\"p\">:<\/span> <span class=\"ss\">&quot;&quot;<\/span><span class=\"p\">,<\/span>\n  <span class=\"ss\">&quot;pathname&quot;<\/span><span class=\"p\">:<\/span> <span class=\"ss\">&quot;\/&quot;<\/span><span class=\"p\">,<\/span>\n  <span class=\"ss\">&quot;search&quot;<\/span><span class=\"p\">:<\/span> <span class=\"ss\">&quot;?name=ayman&amp;age=22&amp;gpa=3.5&amp;course=programming&amp;course=mathematics&amp;course=algorithms&quot;<\/span><span class=\"p\">,<\/span>\n  <span class=\"ss\">&quot;parameters&quot;<\/span><span class=\"p\">:<\/span> <span class=\"err\">{<\/span>\n  <span class=\"ss\">&quot;name&quot;<\/span><span class=\"p\">:<\/span> <span class=\"ss\">&quot;ayman&quot;<\/span><span class=\"p\">,<\/span>\n  <span class=\"ss\">&quot;age&quot;<\/span><span class=\"p\">:<\/span> <span class=\"mi\">22<\/span><span class=\"p\">,<\/span>\n  <span class=\"ss\">&quot;gpa&quot;<\/span><span class=\"p\">:<\/span> <span class=\"mi\">3<\/span><span class=\"p\">.<\/span><span class=\"mi\">5<\/span><span class=\"p\">,<\/span>\n  <span class=\"ss\">&quot;course&quot;<\/span><span class=\"p\">:<\/span> <span class=\"p\">[<\/span>\n     <span class=\"ss\">&quot;programming&quot;<\/span><span class=\"p\">,<\/span>\n     <span class=\"ss\">&quot;mathematics&quot;<\/span><span class=\"p\">,<\/span>\n     <span class=\"ss\">&quot;algorithms&quot;<\/span><span class=\"p\">]<\/span>\n  <span class=\"err\">}<\/span>\n<span class=\"err\">}<\/span>\n<\/pre><\/div>\n\n\n<p>The code is also available as a Gist <a href=\"https:\/\/gist.github.com\/aymanfarhat\/5608517\">here<\/a>.<\/p>\n<p>I hope that you find this code snippet helpful in your work, would love to hear any suggestions and comments from your side about the code too. Have a good day!<\/p>\n<h2>Update (Jan 12, 2014)<\/h2>\n<p>I just updated the code above with a refactored and unit tested version of the old function. Fixed some bugs and style problems. The code conforms to <a href=\"http:\/\/www.ecma-international.org\/publications\/standards\/Ecma-262.htm\">Third Edition of the ECMAScript Programming Language Standard<\/a> via <a href=\"http:\/\/www.jslint.com\/lint.html\">JSLint<\/a>. The <a href=\"https:\/\/gist.github.com\/aymanfarhat\/5608517\">gist<\/a> is up to date and it is also available on <a href=\"http:\/\/jsfiddle.net\/aymanfarhat\/ut5kd\/1\/\">JSFiddle for testing<\/a>.<\/p>","category":[{"@attributes":{"term":"javascript"}},{"@attributes":{"term":"parameters"}},{"@attributes":{"term":"urls"}},{"@attributes":{"term":"parsing"}}]},{"title":"Serializing a Function's Arguments in Javascript","link":{"@attributes":{"href":"https:\/\/www.thecodeship.com\/web-development\/serializing-function-arguments-in-javascript\/","rel":"alternate"}},"published":"2013-05-13T01:00:00+03:00","updated":"2013-05-13T01:00:00+03:00","author":{"name":"Ayman Farhat"},"id":"tag:www.thecodeship.com,2013-05-13:\/web-development\/serializing-function-arguments-in-javascript\/","summary":"<p>At many instances when doing Ajax requests in Javascript, a serialize function comes in handy for creating a string in standard URL-encoded notation for the request parameters. The most widely used <a href=\"http:\/\/api.jquery.com\/serialize\/\">serialize function<\/a> is that of jQuery which works great on forms. But what if the request is not based ...<\/p>","content":"<p>At many instances when doing Ajax requests in Javascript, a serialize function comes in handy for creating a string in standard URL-encoded notation for the request parameters. The most widely used <a href=\"http:\/\/api.jquery.com\/serialize\/\">serialize function<\/a> is that of jQuery which works great on forms. But what if the request is not based on values of a form?<\/p>\n<h2>Some Background<\/h2>\n<p>Consider a function who's job is to take some arguments, and perform a request with the supplied values. Also consider that there is no form involved, the first thing that comes to mind is to build the data string through concatenating the values of the arguments and we'd end up doing something like this:<\/p>\n<div class=\"highlight\"><pre><span><\/span><span class=\"kd\">function<\/span> <span class=\"nx\">requesEmployee<\/span><span class=\"p\">(<\/span><span class=\"nx\">name<\/span><span class=\"p\">,<\/span><span class=\"nx\">middle<\/span><span class=\"p\">,<\/span><span class=\"nx\">family<\/span><span class=\"p\">,<\/span><span class=\"nx\">income<\/span><span class=\"p\">)<\/span> <span class=\"p\">{<\/span>\n   <span class=\"kd\">var<\/span> <span class=\"nx\">params<\/span> <span class=\"o\">=<\/span> <span class=\"s2\">&quot;name=&quot;<\/span><span class=\"o\">+<\/span><span class=\"nx\">name<\/span><span class=\"o\">+<\/span><span class=\"s2\">&quot;&amp;middle=&quot;<\/span><span class=\"o\">+<\/span><span class=\"nx\">middle<\/span><span class=\"o\">+<\/span><span class=\"s2\">&quot;&amp;family=&quot;<\/span><span class=\"o\">+<\/span><span class=\"nx\">family<\/span><span class=\"o\">+<\/span><span class=\"s2\">&quot;&amp;income=&quot;<\/span><span class=\"o\">+<\/span><span class=\"nx\">income<\/span><span class=\"p\">;<\/span>\n\n   <span class=\"nx\">$<\/span><span class=\"p\">.<\/span><span class=\"nx\">ajax<\/span><span class=\"p\">({<\/span>\n      <span class=\"nx\">url<\/span><span class=\"o\">:<\/span><span class=\"s2\">&quot;someurl&quot;<\/span><span class=\"p\">,<\/span>\n      <span class=\"nx\">type<\/span><span class=\"o\">:<\/span><span class=\"s2\">&quot;GET&quot;<\/span><span class=\"p\">,<\/span>\n      <span class=\"nx\">data<\/span><span class=\"o\">:<\/span> <span class=\"nx\">params<\/span>\n   <span class=\"p\">});<\/span>\n\n   <span class=\"c1\">\/\/ etc...<\/span>\n<span class=\"p\">}<\/span>\n<\/pre><\/div>\n\n\n<p>The params string looks fine but what if there were much more parameters like 10, 15 or 20? Concatenating them would be annoying and wont look much elegant even when using something like <a href=\"http:\/\/www.diveintojavascript.com\/projects\/javascript-sprintf\">sprintf<\/a>. The ability to build such a string right from the function's argument values and names would be very powerful and that's what we're going to do.<\/p>\n<h2>Serializing through the Function's Arguments<\/h2>\n<p>As an alternative we can write a function to do the serialization for us. After all, a list of argument values on a function is easily accessible and so are the parameter names, all what we have to do after obtaining the two lists is glue them together into a string. Here is an example:<\/p>\n<div class=\"highlight\"><pre><span><\/span><span class=\"kd\">function<\/span> <span class=\"nx\">serializeArgs<\/span><span class=\"p\">(<\/span><span class=\"nx\">func<\/span><span class=\"p\">,<\/span><span class=\"nx\">args<\/span><span class=\"p\">,<\/span><span class=\"nx\">exclude<\/span><span class=\"p\">)<\/span> <span class=\"p\">{<\/span>\n   <span class=\"nx\">exclude<\/span> <span class=\"o\">=<\/span> <span class=\"nx\">exclude<\/span> <span class=\"o\">||<\/span> <span class=\"p\">[];<\/span>\n\n   <span class=\"kd\">var<\/span> <span class=\"nx\">args<\/span> <span class=\"o\">=<\/span> <span class=\"nb\">Array<\/span><span class=\"p\">.<\/span><span class=\"nx\">prototype<\/span><span class=\"p\">.<\/span><span class=\"nx\">slice<\/span><span class=\"p\">.<\/span><span class=\"nx\">call<\/span><span class=\"p\">(<\/span><span class=\"nx\">args<\/span><span class=\"p\">,<\/span><span class=\"mi\">0<\/span><span class=\"p\">);<\/span>\n   <span class=\"kd\">var<\/span> <span class=\"nx\">funstr<\/span> <span class=\"o\">=<\/span> <span class=\"nx\">func<\/span><span class=\"p\">.<\/span><span class=\"nx\">toString<\/span><span class=\"p\">();<\/span>\n   <span class=\"kd\">var<\/span> <span class=\"nx\">params<\/span> <span class=\"o\">=<\/span> <span class=\"nx\">funstr<\/span><span class=\"p\">.<\/span><span class=\"nx\">slice<\/span><span class=\"p\">(<\/span><span class=\"nx\">funstr<\/span><span class=\"p\">.<\/span><span class=\"nx\">indexOf<\/span><span class=\"p\">(<\/span><span class=\"s1\">&#39;(&#39;<\/span><span class=\"p\">)<\/span><span class=\"o\">+<\/span><span class=\"mi\">1<\/span><span class=\"p\">,<\/span> <span class=\"nx\">funstr<\/span><span class=\"p\">.<\/span><span class=\"nx\">indexOf<\/span><span class=\"p\">(<\/span><span class=\"s1\">&#39;)&#39;<\/span><span class=\"p\">)).<\/span><span class=\"nx\">match<\/span><span class=\"p\">(<\/span><span class=\"sr\">\/([^\\s,]+)\/g<\/span><span class=\"p\">);<\/span>\n\n   <span class=\"kd\">var<\/span> <span class=\"nx\">serialized<\/span> <span class=\"o\">=<\/span> <span class=\"s2\">&quot;&quot;<\/span><span class=\"p\">;<\/span>\n\n   <span class=\"k\">for<\/span><span class=\"p\">(<\/span><span class=\"kd\">var<\/span> <span class=\"nx\">i<\/span> <span class=\"k\">in<\/span> <span class=\"nx\">args<\/span><span class=\"p\">)<\/span> <span class=\"p\">{<\/span>\n      <span class=\"k\">if<\/span><span class=\"p\">(<\/span><span class=\"o\">!<\/span><span class=\"nx\">contains<\/span><span class=\"p\">(<\/span><span class=\"nx\">exclude<\/span><span class=\"p\">,<\/span><span class=\"nx\">params<\/span><span class=\"p\">[<\/span><span class=\"nx\">i<\/span><span class=\"p\">]))<\/span> <span class=\"p\">{<\/span>\n         <span class=\"nx\">append<\/span> <span class=\"o\">=<\/span> <span class=\"p\">(<\/span><span class=\"nx\">i<\/span> <span class=\"o\">&lt;<\/span> <span class=\"nx\">args<\/span><span class=\"p\">.<\/span><span class=\"nx\">length<\/span><span class=\"o\">-<\/span><span class=\"mi\">1<\/span><span class=\"p\">)<\/span><span class=\"o\">?<\/span><span class=\"s2\">&quot;&amp;&quot;<\/span><span class=\"o\">:<\/span><span class=\"s2\">&quot;&quot;<\/span><span class=\"p\">;<\/span>\n         <span class=\"nx\">serialized<\/span> <span class=\"o\">+=<\/span> <span class=\"nx\">params<\/span><span class=\"p\">[<\/span><span class=\"nx\">i<\/span><span class=\"p\">]<\/span><span class=\"o\">+<\/span><span class=\"s2\">&quot;=&quot;<\/span><span class=\"o\">+<\/span><span class=\"nx\">args<\/span><span class=\"p\">[<\/span><span class=\"nx\">i<\/span><span class=\"p\">]<\/span><span class=\"o\">+<\/span><span class=\"nx\">append<\/span><span class=\"p\">;<\/span>\n      <span class=\"p\">}<\/span>\n   <span class=\"p\">}<\/span>\n   <span class=\"k\">return<\/span> <span class=\"nx\">serialized<\/span><span class=\"p\">;<\/span>\n<span class=\"p\">}<\/span>\n\n<span class=\"kd\">function<\/span> <span class=\"nx\">contains<\/span><span class=\"p\">(<\/span><span class=\"nx\">arr<\/span><span class=\"p\">,<\/span> <span class=\"nx\">obj<\/span><span class=\"p\">)<\/span> <span class=\"p\">{<\/span>\n   <span class=\"kd\">var<\/span> <span class=\"nx\">i<\/span> <span class=\"o\">=<\/span> <span class=\"nx\">arr<\/span><span class=\"p\">.<\/span><span class=\"nx\">length<\/span><span class=\"p\">;<\/span>\n\n   <span class=\"k\">while<\/span><span class=\"p\">(<\/span><span class=\"nx\">i<\/span><span class=\"o\">--<\/span><span class=\"p\">)<\/span> <span class=\"p\">{<\/span>\n      <span class=\"k\">if<\/span><span class=\"p\">(<\/span><span class=\"nx\">arr<\/span><span class=\"p\">[<\/span><span class=\"nx\">i<\/span><span class=\"p\">]<\/span> <span class=\"o\">===<\/span> <span class=\"nx\">obj<\/span><span class=\"p\">)<\/span>\n         <span class=\"k\">return<\/span> <span class=\"kc\">true<\/span><span class=\"p\">;<\/span>\n   <span class=\"p\">}<\/span>\n\n   <span class=\"k\">return<\/span> <span class=\"kc\">false<\/span><span class=\"p\">;<\/span>\n<span class=\"p\">}<\/span>\n<\/pre><\/div>\n\n\n<p>Sample usage:<\/p>\n<div class=\"highlight\"><pre><span><\/span><span class=\"kd\">function<\/span> <span class=\"nx\">request<\/span><span class=\"p\">(<\/span><span class=\"nx\">name<\/span><span class=\"p\">,<\/span><span class=\"nx\">middle<\/span><span class=\"p\">,<\/span><span class=\"nx\">family<\/span><span class=\"p\">,<\/span><span class=\"nx\">income<\/span><span class=\"p\">)<\/span> <span class=\"p\">{<\/span>\n   <span class=\"kd\">var<\/span> <span class=\"nx\">data<\/span> <span class=\"o\">=<\/span> <span class=\"nx\">serializeArgs<\/span><span class=\"p\">(<\/span><span class=\"nx\">test<\/span><span class=\"p\">,<\/span><span class=\"nx\">arguments<\/span><span class=\"p\">,[<\/span><span class=\"s1\">&#39;two&#39;<\/span><span class=\"p\">]);<\/span>\n   <span class=\"cm\">\/* Result: name=john&amp;family=Doe&amp;income=9000 *\/<\/span>\n\n   <span class=\"cm\">\/** Rest of the code **\/<\/span>\n<span class=\"p\">}<\/span>\n\n<span class=\"nx\">request<\/span><span class=\"p\">(<\/span><span class=\"s2\">&quot;John&quot;<\/span><span class=\"p\">,<\/span><span class=\"s2\">&quot;Jim&quot;<\/span><span class=\"p\">,<\/span><span class=\"s2\">&quot;Doe&quot;<\/span><span class=\"p\">,<\/span><span class=\"mi\">9000<\/span><span class=\"p\">);<\/span>\n<\/pre><\/div>\n\n\n<p>the serializeArgs function takes 3 arguments, the called function, argument values, and an optional array of excluded parameters that we don't want to include within the serialized string. The argument values we're getting them for \"free\" as within any function exists a local array-like object(arguments) holding the values. As for obtaining a list of the names I converted the function into a string and sliced my way to obtain them, pretty hackish yet works well on any browser and does the job. After obtaining those two lists, building the string is a trivial, all we have to do is loop and concatenating on each index.<\/p>\n<p>The contains function is optional if you're using some library which provides that functionality, such as <a href=\"http:\/\/jquery.com\/\">jQuery<\/a> or <a href=\"http:\/\/underscorejs.org\/\">underscore<\/a>.<\/p>\n<p>I hope you find this technique useful in any of your work, would love to hear your feedback and remarks about this post, have a good day!<\/p>","category":[{"@attributes":{"term":"javascript"}},{"@attributes":{"term":"serialization"}},{"@attributes":{"term":"functions"}},{"@attributes":{"term":"parameters"}}]},{"title":"Quality Computer Science Education for Free? Yes Please!","link":{"@attributes":{"href":"https:\/\/www.thecodeship.com\/general\/quality-computer-science-education-free-yes-please\/","rel":"alternate"}},"published":"2013-05-03T13:57:00+03:00","updated":"2013-05-03T13:57:00+03:00","author":{"name":"Ayman Farhat"},"id":"tag:www.thecodeship.com,2013-05-03:\/general\/quality-computer-science-education-free-yes-please\/","summary":"<p>Recently I have been following a couple of online courses on Coursera and udacity, and I must say the quality of the material is really good. Browsing through several programming and computer science courses you can notice that the quality of the material is at least as good as many...<\/p>","content":"<p>Recently I have been following a couple of online courses on Coursera and udacity, and I must say the quality of the material is really good. Browsing through several programming and computer science courses you can notice that the quality of the material is at least as good as many paid physical universities if not better. After all, most of those courses are given by professors from top universities around the world! The number of courses available on the current <a href=\"http:\/\/en.wikipedia.org\/wiki\/MOOC\">MOOC<\/a> websites is increasing year after year which is opening up the doors to new fields of study online.<\/p>\n<p>Computer science, programming and maths courses are widely available on all of the MOOC websites out there. With that many courses available, that gave me the idea to look into devising a complete 4 year curriculum for a free quality Computer Science education. Of course there isn't a \"real\" degree associated with all that, but after all education is about the knowledge that one attains and not a signed fancy paper.<\/p>\n<p>The suggested curriculum can go over 4 years, each year is divided into 2 semesters and in each semester the student would take 4 required major courses(Computer Science, Programming and Maths) and maybe one liberal arts elective that is not necessarily related to the major.<\/p>\n<p>A typical undergraduate CS program would give the student background in the following areas:<\/p>\n<ul>\n<li>Computer Programming<\/li>\n<li>Data Structures<\/li>\n<li>Object Oriented Programming<\/li>\n<li>Algorithms<\/li>\n<li>Programming Language Theory<\/li>\n<li>Software Engineering<\/li>\n<li>Computer Networking<\/li>\n<li>Human Computer Interaction<\/li>\n<li>Operating Systems<\/li>\n<li>Web Programming<\/li>\n<li>Database Systems<\/li>\n<li>Mathematics<ul>\n<li>Discrete Mathematics<\/li>\n<li>Calculus<\/li>\n<li>Statistics and Probability<\/li>\n<li>Linear Algebra<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<p>The curriculum basically assumes a high school education preferably focused on the sciences. And here it goes:<\/p>\n<h3>Year 1, 1st Semester:<\/h3>\n<ul>\n<li><a href=\"https:\/\/www.coursera.org\/course\/cs101\">Computer Science 101<\/a>: Teaches the essential ideas of Computer Science for a zero-prior-experience audience.<\/li>\n<li><a href=\"https:\/\/www.coursera.org\/course\/programming1\">Programming 1 (Fundamentals of Programming)<\/a>: Introduces the fundamental building blocks of programming and teaches you how to write fun and useful programs using the Python language.<\/li>\n<li><a href=\"https:\/\/www.coursera.org\/course\/calcsing\">Single Variable Calculus<\/a>: Discusses differential and integral calculus, with an emphasis on conceptual understanding and applications to the engineering, physical, and social sciences. An equivalent course is available at <a href=\"http:\/\/ocw.mit.edu\/courses\/mathematics\/18-01sc-single-variable-calculus-fall-2010\/index.htm\">MIT Open Course ware<\/a><\/li>\n<li><a href=\"https:\/\/www.coursera.org\/course\/stats1\">Statistics 1<\/a>: Comprehensive and friendly introduction to statistics including an introduction to R programming and its applications within statistics.<\/li>\n<\/ul>\n<h3>Year 1, 2nd Semester:<\/h3>\n<ul>\n<li><a href=\"https:\/\/www.coursera.org\/course\/programming2\">Programming 2(Crafting Quality Code)<\/a>: Focused on writing quality code that runs correctly and efficiently, Teaches how to design, code and validate programs and how to compare programs that are addressing the same task.<\/li>\n<li><a href=\"http:\/\/www.coursera.org\/course\/programdesign\">Introduction to Systematic Program Design<\/a>: Building programs that are elegant, well tested and easy to maintain. Different tracks with different popular programming languages.<\/li>\n<li><a href=\"http:\/\/www.aduni.org\/courses\/discrete\/\">Discrete Mathematics<\/a>: Introduces a collection of mathematical topics that are directly related to theoretical Computer Science, each of which is a domain in mathematics on its own.<\/li>\n<li><a href=\"https:\/\/www.coursera.org\/course\/sciwrite\">Writing in the Sciences<\/a>: Teaches science students to become more effective writers. Topics include: principles of good writing, tricks for writing faster, the format of a scientific manuscript, and issues in publication and peer review.<\/li>\n<\/ul>\n<h3>Year 2, 1st Semester<\/h3>\n<ul>\n<li><a href=\"https:\/\/www.coursera.org\/course\/matrix\">Linear Algebra<\/a>: Learn the concepts and methods of linear algebra, and how to use them to think about computational problems arising in computer science.<\/li>\n<li><a href=\"http:\/\/www.coursera.org\/course\/db\">Introduction to Databases<\/a>: Covers database design and the use of database management systems for applications.<\/li>\n<li><a href=\"https:\/\/www.coursera.org\/course\/algo\">Algorithm Design and Analysis 1<\/a>: Fundamental principles of algorithm design: divide-and-conquer methods, graph algorithms, practical data structures (heaps, hash tables, search trees), randomized algorithms, and more.<\/li>\n<li><a href=\"https:\/\/www.udacity.com\/course\/cs259\">Software Debugging<\/a>: Teaches systematic debugging, how to automate debugging and build several functional debugging tools in Python.<\/li>\n<\/ul>\n<h3>Year 2, 2nd Semester<\/h3>\n<ul>\n<li><a href=\"https:\/\/www.coursera.org\/course\/algo2\">Algorithm Design and Analysis 2<\/a>: Fundamental principles of advanced algorithm design: greedy algorithms and applications; dynamic programming and applications; NP-completeness and what it means for the algorithm designer; the design and analysis of heuristics; and more.<\/li>\n<li><a href=\"https:\/\/www.udacity.com\/course\/cs258\">Software Testing<\/a>: Basics of Software Testing<\/li>\n<li><a href=\"http:\/\/www.thecodeship.com\/admin\/blog\/post\/add\/\">Computer Networks<\/a>: Learn how networks are designed and work. Topics covered: Networking layers, data transmission, protocols, DNS, HTTP, and network security while learning how the Internet works.<\/li>\n<li><a href=\"https:\/\/www.coursera.org\/course\/crypto\">Cryptography 1<\/a>: Learn about the inner workings of cryptographic primitives and how to apply this knowledge in real-world applications.<\/li>\n<\/ul>\n<h3>Year 3, 1st Semester<\/h3>\n<ul>\n<li><a href=\"https:\/\/www.coursera.org\/course\/crypto2\">Cryptography 2<\/a>: Continuation of the topics discussed in Cryptography 1.<\/li>\n<li><a href=\"https:\/\/www.coursera.org\/course\/hci\">Human - Computer Interaction<\/a>: Build human-centered design skills, so that you have the principles and methods to create excellent interfaces with any technology.<\/li>\n<li><a href=\"https:\/\/www.coursera.org\/course\/proglang\">Programming Languages<\/a>: Investigate the basic concepts behind programming languages, with a strong emphasis on the techniques and benefits of functional programming.<\/li>\n<li><a href=\"https:\/\/www.udacity.com\/course\/cs271\">Introduction to Artificial Intelligence<\/a>: Basics of Artificial Intelligence, which includes machine learning, probabilistic reasoning, robotics, and natural language processing.<\/li>\n<\/ul>\n<h3>Year 3, 2nd Semester<\/h3>\n<ul>\n<li><a href=\"https:\/\/www.udacity.com\/course\/cs253\">Web Development<\/a>:Teaches the basics of Web Development including building a blog web application from scratch and how to make it scale.<\/li>\n<li><a href=\"https:\/\/www.coursera.org\/course\/compilers\">Compilers<\/a>: Discusses the major ideas used today in the implementation of programming language compilers. You will learn how a program written in a high-level language designed for humans is systematically translated into a program written in low-level assembly more suited to machines.<\/li>\n<li><a href=\"https:\/\/www.coursera.org\/course\/datasci\">Introduction to Data Science<\/a>: Introduce yourself to the basics of data science and leave armed with practical experience extracting value from big data.<\/li>\n<li><a href=\"https:\/\/www.coursera.org\/course\/automata\">Automata<\/a>: Covers finite automata, context-free grammars, Turing machines, undecidable problems, and intractable problems (NP-completeness).<\/li>\n<\/ul>\n<h3>Year 4, 1st Semester<\/h3>\n<ul>\n<li><a href=\"https:\/\/www.udacity.com\/course\/cs344\">Introduction to Parallel Programming<\/a>: Master the fundamentals of massively parallel computing by using CUDA C\/C++ to program modern GPUs.<\/li>\n<li><a href=\"https:\/\/www.coursera.org\/course\/ml\">Machine Learning<\/a>: Learn about the most effective machine learning techniques, and gain practice implementing them and getting them to work for yourself.<\/li>\n<li><a href=\"https:\/\/www.coursera.org\/course\/images\">Image and Video Processing<\/a>: Look behind the scenes of image and video processing, from the basic and classical tools to the most modern and advanced algorithms.<\/li>\n<li><a href=\"https:\/\/www.coursera.org\/course\/startup\">Startup Engineering<\/a>: Learn the engineering skills needed to build a technology startup from the ground up.<\/li>\n<\/ul>\n<h3>Year 4, 2nd Semester<\/h3>\n<ul>\n<li><a href=\"https:\/\/www.coursera.org\/course\/nlp\">Natural Language Processing<\/a>: Learn fundamental algorithms and mathematical models for processing natural language, and how these can be used to solve practical problems.<\/li>\n<li><a href=\"https:\/\/www.coursera.org\/course\/pgm\">Probabilistic Graphical Models<\/a>: Learn the basics of the PGM representation and how to construct them, using both human knowledge and machine learning techniques.<\/li>\n<li><a href=\"https:\/\/www.udacity.com\/course\/cs373\">Artificial Intelligence for Robotics<\/a>: Probabilistic inference, planning and search, localization, tracking and control, all with a focus on robotics.<\/li>\n<li><a href=\"http:\/\/www.coursera.org\/course\/digitalmedia\">Creative Programming for Digital Media &amp; Mobile Apps<\/a>: Apply technical skills to creative work ranging from video games to art installations to interactive music, and also for artists who would like to use programming in their artistic practice.<\/li>\n<\/ul>\n<p>And there you have it, a 4 year curriculum for a full Computer Science education that contains a mix of theoretical and practical topics. What I really love about all those courses is that almost all of them focus on discussing where and how to apply the theory you're learning in the real world and in programming. And this in my opinion is what makes such courses really effective in adding value to the learner, and makes them more ready to the job market.<\/p>\n<p>Platforms such as Coursera and Udacity are revolutionizing the way we learn to the better. The age where virtually anyone in the world regardless of their economic status, color, race and gender can have access to a decent free education is already here. Freedom not necessarily in the monetary sense but also freedom in time, choosing topics of interest, and learning speed. Freedom from the handcuffs of classic educational institutions who's main goal is to make profits while delivering a one size fits all education.<\/p>\n<p>What I would really like to see one day is a new style of educational institutions that are affiliated with with MOOC platforms and operate within different areas. For a certain reasonable tuition they'd host the MOOC courses locally and organize them into full undergraduate programs, they provide the students with lecture material to study, and administer the exams. At the end of the curriculum, successful students would receive a degree that is officially approved by the MOOC platform and that is equivalent to a degree from any other classical education institution.<\/p>\n<p>Do you think such model for tech related education(such as applied Computer Science) can really work? Would love to hear your comments.<\/p>","category":[{"@attributes":{"term":"science"}},{"@attributes":{"term":"coursera"}},{"@attributes":{"term":"free"}},{"@attributes":{"term":"udacity"}},{"@attributes":{"term":"computer"}},{"@attributes":{"term":"education"}}]},{"title":"New Version of the Blog is Out","link":{"@attributes":{"href":"https:\/\/www.thecodeship.com\/general\/new-version-of-the-blog-is-out\/","rel":"alternate"}},"published":"2013-05-02T15:23:00+03:00","updated":"2013-05-02T15:23:00+03:00","author":{"name":"Ayman Farhat"},"id":"tag:www.thecodeship.com,2013-05-02:\/general\/new-version-of-the-blog-is-out\/","summary":"<p>Hello everyone, as you can see I have just rolled out a new design for this blog, the old one was nice yet had some evident problems in terms of UX, layout and colors. So I decided to revamp it into this. The new blog design is simpler, the layout ...<\/p>","content":"<p>Hello everyone, as you can see I have just rolled out a new design for this blog, the old one was nice yet had some evident problems in terms of UX, layout and colors. So I decided to revamp it into this. The new blog design is simpler, the layout is responsive and built on top of <a href=\"http:\/\/www.getskeleton.com\/\">skeleton<\/a>, better syntax highlighting including inline code, and several fixes for bugs that were present within the old one. I'd really love to read your comments and suggestions regarding the new design. Some nice new features will be added gradually soon too.<\/p>\n<p>This blog is built with Python's Django, it is open source, released under <a href=\"http:\/\/www.gnu.org\/licenses\/quick-guide-gplv3.html\">GPL 3<\/a> and available on <a href=\"https:\/\/github.com\/aymanfarhat\/TheCodeShip\">Github<\/a>. The current stable version can be found on the \"master\" branch while I kept a copy of the older one under the \"old\" branch. Looking forward to your forks!<\/p>","category":[{"@attributes":{"term":"new"}},{"@attributes":{"term":"version"}}]},{"title":"Methods Within Constructor vs Prototype in Javascript","link":{"@attributes":{"href":"https:\/\/www.thecodeship.com\/web-development\/methods-within-constructor-vs-prototype-in-javascript\/","rel":"alternate"}},"published":"2013-04-02T17:26:00+03:00","updated":"2013-04-02T17:26:00+03:00","author":{"name":"Ayman Farhat"},"id":"tag:www.thecodeship.com,2013-04-02:\/web-development\/methods-within-constructor-vs-prototype-in-javascript\/","summary":"<p>At many instances when working with javascript objects, different pieces of code can give the same result on the surface yet underneath they could be different. One scenario is when adding methods to your Javascript 'classes'. First of all lets agree on the fact that there are no classes in ...<\/p>","content":"<p>At many instances when working with javascript objects, different pieces of code can give the same result on the surface yet underneath they could be different. One scenario is when adding methods to your Javascript 'classes'.<\/p>\n<p>First of all lets agree on the fact that there are no classes in Javascript, and what you may refer to as a class is known as a constructor. That is because Javascript is not your classic <a href=\"http:\/\/en.wikipedia.org\/wiki\/Class-based_programming\">class-based<\/a> language but rather a <a href=\"http:\/\/en.wikipedia.org\/wiki\/Prototype-based_programming\">prototype-based<\/a> language.<\/p>\n<h2>It's not what it looks like<\/h2>\n<p>One way that may seem very natural is to set the methods right within the constructor, just like this.<\/p>\n<div class=\"highlight\"><pre><span><\/span><span class=\"kd\">function<\/span> <span class=\"nx\">Person<\/span><span class=\"p\">(<\/span><span class=\"nx\">name<\/span><span class=\"p\">,<\/span> <span class=\"nx\">family<\/span><span class=\"p\">)<\/span> <span class=\"p\">{<\/span>\n    <span class=\"k\">this<\/span><span class=\"p\">.<\/span><span class=\"nx\">name<\/span> <span class=\"o\">=<\/span> <span class=\"nx\">name<\/span><span class=\"p\">;<\/span>\n    <span class=\"k\">this<\/span><span class=\"p\">.<\/span><span class=\"nx\">family<\/span> <span class=\"o\">=<\/span> <span class=\"nx\">family<\/span><span class=\"p\">;<\/span>\n    <span class=\"k\">this<\/span><span class=\"p\">.<\/span><span class=\"nx\">getFull<\/span> <span class=\"o\">=<\/span> <span class=\"kd\">function<\/span> <span class=\"p\">()<\/span> <span class=\"p\">{<\/span>\n        <span class=\"k\">return<\/span> <span class=\"k\">this<\/span><span class=\"p\">.<\/span><span class=\"nx\">name<\/span> <span class=\"o\">+<\/span> <span class=\"s2\">&quot; &quot;<\/span> <span class=\"o\">+<\/span> <span class=\"k\">this<\/span><span class=\"p\">.<\/span><span class=\"nx\">family<\/span><span class=\"p\">;<\/span>\n    <span class=\"p\">};<\/span>\n<span class=\"p\">}<\/span>\n<\/pre><\/div>\n\n\n<p>Looks pretty natural for someone coming from a class based language like Java, and it will work as expected.<\/p>\n<p>Yet the truth is, this approach might be wrong for many situations. In Javascript when you bind a method to the this keyword, you are providing that method to only that particular instance and it does not really have any relationship with an object instance of that constructor, pretty much like a static method. Keeping in mind that functions are first-class citizens in Javascript, we can deal with them just like objects, in this case we're only adding a property to an instance of a function object. Thats only part of the story, you must also know that any method attached via this <strong>will get re-declared<\/strong> for every new instance we create, which could affect the memory usage of the application negatively if we wish to create so many instances.<\/p>\n<h2>All that is gold does not glitter<\/h2>\n<p>That is why it is recommended in most cases to use a less obvious approach which is through our object's <a href=\"https:\/\/developer.mozilla.org\/en-US\/docs\/JavaScript\/Reference\/Global_Objects\/Object\/prototype\">prototype<\/a>. Prototype will enable us to easily define methods to all instances of the instances while saving memory. What's great is that the method will be applied to the prototype of the object, so it is only stored in the memory once, because objects coming from the same constructor point to one common prototype object. In addition to that, all instances of Person will have access to that method.<\/p>\n<p>Recreating the above example, we end up with something like this<\/p>\n<div class=\"highlight\"><pre><span><\/span><span class=\"kd\">function<\/span> <span class=\"nx\">Person<\/span><span class=\"p\">(<\/span><span class=\"nx\">name<\/span><span class=\"p\">,<\/span> <span class=\"nx\">family<\/span><span class=\"p\">)<\/span> <span class=\"p\">{<\/span>\n    <span class=\"k\">this<\/span><span class=\"p\">.<\/span><span class=\"nx\">name<\/span> <span class=\"o\">=<\/span> <span class=\"nx\">name<\/span><span class=\"p\">;<\/span>\n    <span class=\"k\">this<\/span><span class=\"p\">.<\/span><span class=\"nx\">family<\/span> <span class=\"o\">=<\/span> <span class=\"nx\">family<\/span><span class=\"p\">;<\/span>\n<span class=\"p\">}<\/span>\n\n<span class=\"nx\">Person<\/span><span class=\"p\">.<\/span><span class=\"nx\">prototype<\/span><span class=\"p\">.<\/span><span class=\"nx\">getFull<\/span> <span class=\"o\">=<\/span> <span class=\"kd\">function<\/span><span class=\"p\">()<\/span> <span class=\"p\">{<\/span>\n    <span class=\"k\">return<\/span> <span class=\"k\">this<\/span><span class=\"p\">.<\/span><span class=\"nx\">name<\/span> <span class=\"o\">+<\/span> <span class=\"s2\">&quot; &quot;<\/span> <span class=\"o\">+<\/span> <span class=\"k\">this<\/span><span class=\"p\">.<\/span><span class=\"nx\">family<\/span><span class=\"p\">;<\/span>\n<span class=\"p\">};<\/span>\n<\/pre><\/div>\n\n\n<p>Now having our object instances based on the second example, we will be able to save a significant amount of memory compared to the first approach as the number of instances of the constructor grows larger within the application. One should note though that modern Javascript engines such as V8 are smart enough for not to recreate instances of a method thanks to hidden classes.<\/p>\n<p>Apart from low memory usage, the prototype approach is obviously faster in execution when creating new object instances since no time is spent on re-declaring any methods. However, there will be a slight decrease in the speed of invoking a method in comparison to the first approach. This is because when calling <code>getFull<\/code> the Javascript runtime will check for the method on the instance of the <code>Person<\/code> it won't be found there directly so it will then check the prototype of <code>Person<\/code> to find it. Yet definitely the overhead isn't that significant unless we have a deep prototype hierarchy.<\/p>\n<h2>There is no silver bullet<\/h2>\n<p>Despite the advantages of the prototype approach, it will always depend on the situation for which one to follow. For example the first approach can be useful if we needed access to local private variables from our method. In some situations we might find that we will be creating a small number of instances and accessing local object variables is part of the code design, then the first approach is not a bad choice.<\/p>\n<p>The prototype approach would still be better when there is no need for the method to access local variables and when the creation of so many object instances is expected in which memory consumption becomes a concern.<\/p>\n<h2>Putting it all together<\/h2>\n<p>Back to our Person constructor again, consider a person has a private list of bank records which we do not wish to expose publicly yet we need to get the total balance from it. At the same time we want to have a method for getting the person's name, family and balance in a single string. To achieve that we'll apply a hybrid of both approaches, we need to have some inside access within the object yet make sure to minimize our memory consumption when we can do so(via prototype methods).<\/p>\n<div class=\"highlight\"><pre><span><\/span><span class=\"kd\">function<\/span> <span class=\"nx\">Person<\/span><span class=\"p\">(<\/span><span class=\"nx\">name<\/span><span class=\"p\">,<\/span> <span class=\"nx\">family<\/span><span class=\"p\">)<\/span> <span class=\"p\">{<\/span>\n    <span class=\"k\">this<\/span><span class=\"p\">.<\/span><span class=\"nx\">name<\/span> <span class=\"o\">=<\/span> <span class=\"nx\">name<\/span><span class=\"p\">;<\/span>\n    <span class=\"k\">this<\/span><span class=\"p\">.<\/span><span class=\"nx\">family<\/span> <span class=\"o\">=<\/span> <span class=\"nx\">family<\/span><span class=\"p\">;<\/span>\n\n    <span class=\"kd\">var<\/span> <span class=\"nx\">records<\/span> <span class=\"o\">=<\/span> <span class=\"p\">[{<\/span><span class=\"nx\">type<\/span><span class=\"o\">:<\/span> <span class=\"s2\">&quot;in&quot;<\/span><span class=\"p\">,<\/span> <span class=\"nx\">amount<\/span><span class=\"o\">:<\/span> <span class=\"mi\">0<\/span><span class=\"p\">}];<\/span>\n\n    <span class=\"k\">this<\/span><span class=\"p\">.<\/span><span class=\"nx\">addTransaction<\/span> <span class=\"o\">=<\/span> <span class=\"kd\">function<\/span><span class=\"p\">(<\/span><span class=\"nx\">trans<\/span><span class=\"p\">)<\/span> <span class=\"p\">{<\/span>\n        <span class=\"k\">if<\/span><span class=\"p\">(<\/span><span class=\"nx\">trans<\/span><span class=\"p\">.<\/span><span class=\"nx\">hasOwnProperty<\/span><span class=\"p\">(<\/span><span class=\"s2\">&quot;type&quot;<\/span><span class=\"p\">)<\/span> <span class=\"o\">&amp;&amp;<\/span> <span class=\"nx\">trans<\/span><span class=\"p\">.<\/span><span class=\"nx\">hasOwnProperty<\/span><span class=\"p\">(<\/span><span class=\"s2\">&quot;amount&quot;<\/span><span class=\"p\">))<\/span> <span class=\"p\">{<\/span>\n           <span class=\"nx\">records<\/span><span class=\"p\">.<\/span><span class=\"nx\">push<\/span><span class=\"p\">(<\/span><span class=\"nx\">trans<\/span><span class=\"p\">);<\/span>\n        <span class=\"p\">}<\/span>\n    <span class=\"p\">}<\/span>\n\n    <span class=\"k\">this<\/span><span class=\"p\">.<\/span><span class=\"nx\">balance<\/span> <span class=\"o\">=<\/span> <span class=\"kd\">function<\/span><span class=\"p\">()<\/span> <span class=\"p\">{<\/span>\n       <span class=\"kd\">var<\/span> <span class=\"nx\">total<\/span> <span class=\"o\">=<\/span> <span class=\"mi\">0<\/span><span class=\"p\">;<\/span>\n\n       <span class=\"nx\">records<\/span><span class=\"p\">.<\/span><span class=\"nx\">forEach<\/span><span class=\"p\">(<\/span><span class=\"kd\">function<\/span><span class=\"p\">(<\/span><span class=\"nx\">record<\/span><span class=\"p\">)<\/span> <span class=\"p\">{<\/span>\n           <span class=\"k\">if<\/span><span class=\"p\">(<\/span><span class=\"nx\">record<\/span><span class=\"p\">.<\/span><span class=\"nx\">type<\/span> <span class=\"o\">===<\/span> <span class=\"s2\">&quot;in&quot;<\/span><span class=\"p\">)<\/span> <span class=\"p\">{<\/span>\n             <span class=\"nx\">total<\/span> <span class=\"o\">+=<\/span> <span class=\"nx\">record<\/span><span class=\"p\">.<\/span><span class=\"nx\">amount<\/span><span class=\"p\">;<\/span>\n           <span class=\"p\">}<\/span>\n           <span class=\"k\">else<\/span> <span class=\"p\">{<\/span>\n             <span class=\"nx\">total<\/span> <span class=\"o\">-=<\/span> <span class=\"nx\">record<\/span><span class=\"p\">.<\/span><span class=\"nx\">amount<\/span><span class=\"p\">;<\/span>\n           <span class=\"p\">}<\/span>\n       <span class=\"p\">});<\/span>\n\n        <span class=\"k\">return<\/span> <span class=\"nx\">total<\/span><span class=\"p\">;<\/span>\n    <span class=\"p\">};<\/span>\n<span class=\"p\">};<\/span>\n\n<span class=\"nx\">Person<\/span><span class=\"p\">.<\/span><span class=\"nx\">prototype<\/span><span class=\"p\">.<\/span><span class=\"nx\">getFull<\/span> <span class=\"o\">=<\/span> <span class=\"kd\">function<\/span><span class=\"p\">()<\/span> <span class=\"p\">{<\/span>\n    <span class=\"k\">return<\/span> <span class=\"k\">this<\/span><span class=\"p\">.<\/span><span class=\"nx\">name<\/span> <span class=\"o\">+<\/span> <span class=\"s2\">&quot; &quot;<\/span> <span class=\"o\">+<\/span> <span class=\"k\">this<\/span><span class=\"p\">.<\/span><span class=\"nx\">family<\/span><span class=\"p\">;<\/span>\n<span class=\"p\">};<\/span>\n\n<span class=\"nx\">Person<\/span><span class=\"p\">.<\/span><span class=\"nx\">prototype<\/span><span class=\"p\">.<\/span><span class=\"nx\">getProfile<\/span> <span class=\"o\">=<\/span> <span class=\"kd\">function<\/span><span class=\"p\">()<\/span> <span class=\"p\">{<\/span>\n     <span class=\"k\">return<\/span> <span class=\"k\">this<\/span><span class=\"p\">.<\/span><span class=\"nx\">getFull<\/span><span class=\"p\">()<\/span> <span class=\"o\">+<\/span> <span class=\"s2\">&quot;, total balance: &quot;<\/span> <span class=\"o\">+<\/span> <span class=\"k\">this<\/span><span class=\"p\">.<\/span><span class=\"nx\">balance<\/span><span class=\"p\">();<\/span>\n<span class=\"p\">};<\/span>\n<\/pre><\/div>\n\n\n<p>The above example demonstrates that both approaches have their uses when it comes to working with Javascript objects. I hope you enjoyed reading this post, if you have questions, insights and further related explanations feel free to share them in the comments.<\/p>","category":[{"@attributes":{"term":"prototype"}},{"@attributes":{"term":"methods"}},{"@attributes":{"term":"constructor"}},{"@attributes":{"term":"performance"}}]},{"title":"Exercise: Alien Language Pattern Matching","link":{"@attributes":{"href":"https:\/\/www.thecodeship.com\/algorithms\/exercise-alien-language-pattern-matching\/","rel":"alternate"}},"published":"2013-04-01T10:33:00+03:00","updated":"2013-04-01T10:33:00+03:00","author":{"name":"Ayman Farhat"},"id":"tag:www.thecodeship.com,2013-04-01:\/algorithms\/exercise-alien-language-pattern-matching\/","summary":"<p>As part of my programming workout I was going through some nice Google code jam exercises and came across an interesting exercise from the 2009 qualification round related to matching words against patterns. Not hard but interesting to solve.<\/p>","content":"<p>As part of my programming workout I was going through some nice Google code jam exercises and came across an interesting exercise from the 2009 qualification round related to matching words against patterns. Not hard but interesting to solve.<\/p>\n<p>The problem is explained in detail on the site, you can chec it out here. In a nutshell, we have a set of words for a certain language; At the same time we have a set of string patterns where each pattern can have a number of different interpretations. Based on the words available in the language the program should determine the number of possible interpretations of every pattern. A pattern consists of a number of tokens, each token is either a single lowercase letter or a group of unique lowercase letters surrounded by parenthesis ( and ).<\/p>\n<p>Example:\nConsider the pattern (ab)d(dc) that would mean the first letter is either a or b, the second letter is definitely d and the last letter is either d or c. So the pattern (ab)d(dc) can define these 4 possibilities: {add, adc, bdd, bdc}. Now consider a language whose words are {abc, aaa, abd, add, bdc}, the previously mentioned pattern matches 2 words(namely add and bdc) in that language thus it has 4 possibilities and 2 interpretations.<\/p>\n<p>Solution\nOne way to solve this is to parse all string patterns into lists containing characters and nested lists of characters(to represent the options). For example (ab)d(cd) would be represented as [[\"a\",\"b\"],\"d\",[\"c\",\"d\"]] in Python. This makes the task of comparing a word against a parsed pattern pretty trivial via a loop over the characters.<\/p>\n<p>To convert a string patter to a list I have the following function which takes a list of string patterns.<\/p>\n<div class=\"highlight\"><pre><span><\/span><span class=\"k\">def<\/span> <span class=\"nf\">parse_patterns<\/span><span class=\"p\">(<\/span><span class=\"n\">patterns<\/span><span class=\"p\">):<\/span>\n    <span class=\"n\">lines<\/span> <span class=\"o\">=<\/span> <span class=\"p\">[]<\/span>\n\n    <span class=\"k\">def<\/span> <span class=\"nf\">getclosing<\/span><span class=\"p\">(<\/span><span class=\"n\">i<\/span><span class=\"p\">,<\/span><span class=\"n\">pattern<\/span><span class=\"p\">):<\/span>\n        <span class=\"k\">for<\/span> <span class=\"n\">x<\/span> <span class=\"ow\">in<\/span> <span class=\"nb\">xrange<\/span><span class=\"p\">(<\/span><span class=\"n\">i<\/span><span class=\"p\">,<\/span><span class=\"nb\">len<\/span><span class=\"p\">(<\/span><span class=\"n\">pattern<\/span><span class=\"p\">)):<\/span>\n            <span class=\"k\">if<\/span> <span class=\"n\">pattern<\/span><span class=\"p\">[<\/span><span class=\"n\">x<\/span><span class=\"p\">]<\/span> <span class=\"o\">==<\/span> <span class=\"s2\">&quot;)&quot;<\/span><span class=\"p\">:<\/span>\n                <span class=\"k\">return<\/span> <span class=\"n\">x<\/span>\n        <span class=\"k\">return<\/span> <span class=\"o\">-<\/span><span class=\"mi\">1<\/span>\n\n    <span class=\"k\">for<\/span> <span class=\"n\">pattern<\/span> <span class=\"ow\">in<\/span> <span class=\"n\">patterns<\/span><span class=\"p\">:<\/span>\n        <span class=\"n\">line<\/span> <span class=\"o\">=<\/span> <span class=\"p\">[]<\/span>\n        <span class=\"n\">i<\/span> <span class=\"o\">=<\/span> <span class=\"mi\">0<\/span>\n\n        <span class=\"k\">while<\/span> <span class=\"n\">i<\/span> <span class=\"o\">&lt;<\/span> <span class=\"nb\">len<\/span><span class=\"p\">(<\/span><span class=\"n\">pattern<\/span><span class=\"p\">):<\/span> \n            <span class=\"k\">if<\/span> <span class=\"n\">pattern<\/span><span class=\"p\">[<\/span><span class=\"n\">i<\/span><span class=\"p\">]<\/span> <span class=\"o\">==<\/span> <span class=\"s2\">&quot;(&quot;<\/span><span class=\"p\">:<\/span>\n                <span class=\"n\">j<\/span> <span class=\"o\">=<\/span> <span class=\"n\">getclosing<\/span><span class=\"p\">(<\/span><span class=\"n\">i<\/span><span class=\"p\">,<\/span><span class=\"n\">pattern<\/span><span class=\"p\">)<\/span>\n\n                <span class=\"k\">if<\/span> <span class=\"n\">j<\/span> <span class=\"o\">&gt;<\/span> <span class=\"n\">i<\/span> <span class=\"ow\">and<\/span> <span class=\"n\">j<\/span><span class=\"o\">-<\/span><span class=\"n\">i<\/span> <span class=\"o\">&gt;<\/span> <span class=\"mi\">2<\/span><span class=\"p\">:<\/span>\n                    <span class=\"n\">line<\/span><span class=\"o\">.<\/span><span class=\"n\">append<\/span><span class=\"p\">(<\/span><span class=\"nb\">list<\/span><span class=\"p\">(<\/span><span class=\"n\">pattern<\/span><span class=\"p\">[<\/span><span class=\"n\">i<\/span><span class=\"o\">+<\/span><span class=\"mi\">1<\/span><span class=\"p\">:<\/span><span class=\"n\">j<\/span><span class=\"p\">]))<\/span>\n                    <span class=\"n\">i<\/span> <span class=\"o\">=<\/span> <span class=\"n\">j<\/span>\n\n            <span class=\"k\">elif<\/span> <span class=\"n\">pattern<\/span><span class=\"p\">[<\/span><span class=\"n\">i<\/span><span class=\"p\">]<\/span> <span class=\"o\">!=<\/span> <span class=\"s2\">&quot;)&quot;<\/span><span class=\"p\">:<\/span>\n                <span class=\"n\">line<\/span><span class=\"o\">.<\/span><span class=\"n\">append<\/span><span class=\"p\">(<\/span><span class=\"n\">pattern<\/span><span class=\"p\">[<\/span><span class=\"n\">i<\/span><span class=\"p\">])<\/span>\n            <span class=\"n\">i<\/span> <span class=\"o\">+=<\/span> <span class=\"mi\">1<\/span>\n\n        <span class=\"n\">lines<\/span><span class=\"o\">.<\/span><span class=\"n\">append<\/span><span class=\"p\">(<\/span><span class=\"n\">line<\/span><span class=\"p\">)<\/span>\n\n    <span class=\"k\">return<\/span> <span class=\"n\">lines<\/span>\n<\/pre><\/div>\n\n\n<p>For every string pattern, move through it character by character. If an alphabetical character is found just add it to a list(line), on the other hand if we come across an opening bracket that means there is options in there. In that case find the index of the closing of that options set via getclosing(), then extract the string between those brackets, convert it to a list and add it to our line list.<\/p>\n<p>Now that the patterns are converted to lists, to check if a word matches a pattern all what we have to do is loop through a word and compare its values with the list. If the current value(e.g. char 'a' in word) is against a nested list at the current index then 'a' must be an element of that list to continue the match. If the character is against another character then they must me equal for the pattern to continue matching.<\/p>\n<p>def checkmatch(word, pattern):\n    for i,el in enumerate(word):\n        return !(type(pattern[i]) == str and el != pattern[i]) or (type(pattern[i])==list and el not in pattern[i]):\n    return True<\/p>\n<p>In order to keep track of how many words each pattern matches we can save total number of matches for every pattern within a list of integers where we can access each pattern's number of matches by index, 0 would be the number of matches for pattern 1 etc...<\/p>\n<div class=\"highlight\"><pre><span><\/span><span class=\"c1\"># Creates a list of 0s with the size of the number of patterns available<\/span>\n<span class=\"n\">matches<\/span> <span class=\"o\">=<\/span> <span class=\"p\">[<\/span><span class=\"mi\">0<\/span><span class=\"p\">]<\/span><span class=\"o\">*<\/span><span class=\"nb\">len<\/span><span class=\"p\">(<\/span><span class=\"n\">p_patterns<\/span><span class=\"p\">)<\/span>\n<\/pre><\/div>\n\n\n<p>All what is left now is for each word in the language compare it against all patterns, if a match occurs increment the total number of matches for the matching pattern. When done print the results.<\/p>\n<div class=\"highlight\"><pre><span><\/span><span class=\"c1\"># Check every word for matching with any pattern<\/span>\n<span class=\"k\">for<\/span> <span class=\"n\">word<\/span> <span class=\"ow\">in<\/span> <span class=\"n\">words<\/span><span class=\"p\">:<\/span>\n    <span class=\"k\">for<\/span> <span class=\"n\">i<\/span><span class=\"p\">,<\/span> <span class=\"n\">pattern<\/span> <span class=\"ow\">in<\/span> <span class=\"nb\">enumerate<\/span><span class=\"p\">(<\/span><span class=\"n\">p_patterns<\/span><span class=\"p\">):<\/span>\n        <span class=\"k\">if<\/span> <span class=\"n\">checkmatch<\/span><span class=\"p\">(<\/span><span class=\"n\">word<\/span><span class=\"p\">,<\/span><span class=\"n\">pattern<\/span><span class=\"p\">):<\/span>\n            <span class=\"n\">matches<\/span><span class=\"p\">[<\/span><span class=\"n\">i<\/span><span class=\"p\">]<\/span> <span class=\"o\">+=<\/span> <span class=\"mi\">1<\/span>\n\n<span class=\"c1\"># Printe results<\/span>\n<span class=\"k\">for<\/span> <span class=\"n\">i<\/span> <span class=\"ow\">in<\/span> <span class=\"nb\">xrange<\/span><span class=\"p\">(<\/span><span class=\"mi\">0<\/span><span class=\"p\">,<\/span><span class=\"nb\">len<\/span><span class=\"p\">(<\/span><span class=\"n\">matches<\/span><span class=\"p\">)):<\/span>\n    <span class=\"k\">print<\/span> <span class=\"s2\">&quot;Case #{0}: {1}&quot;<\/span><span class=\"o\">.<\/span><span class=\"n\">format<\/span><span class=\"p\">(<\/span><span class=\"n\">i<\/span><span class=\"o\">+<\/span><span class=\"mi\">1<\/span><span class=\"p\">,<\/span><span class=\"n\">matches<\/span><span class=\"p\">[<\/span><span class=\"n\">i<\/span><span class=\"p\">])<\/span>\n<\/pre><\/div>\n\n\n<p>Here is the full working program that takes input as specified in the original problem, also available as gist.<\/p>\n<div class=\"highlight\"><pre><span><\/span><span class=\"c1\"># Parses each string patterns to list of strings and lists(options)<\/span>\n<span class=\"k\">def<\/span> <span class=\"nf\">parse_patterns<\/span><span class=\"p\">(<\/span><span class=\"n\">patterns<\/span><span class=\"p\">):<\/span>\n    <span class=\"n\">lines<\/span> <span class=\"o\">=<\/span> <span class=\"p\">[]<\/span>\n\n    <span class=\"k\">def<\/span> <span class=\"nf\">getclosing<\/span><span class=\"p\">(<\/span><span class=\"n\">i<\/span><span class=\"p\">,<\/span><span class=\"n\">pattern<\/span><span class=\"p\">):<\/span>\n        <span class=\"k\">for<\/span> <span class=\"n\">x<\/span> <span class=\"ow\">in<\/span> <span class=\"nb\">xrange<\/span><span class=\"p\">(<\/span><span class=\"n\">i<\/span><span class=\"p\">,<\/span><span class=\"nb\">len<\/span><span class=\"p\">(<\/span><span class=\"n\">pattern<\/span><span class=\"p\">)):<\/span>\n            <span class=\"k\">if<\/span> <span class=\"n\">pattern<\/span><span class=\"p\">[<\/span><span class=\"n\">x<\/span><span class=\"p\">]<\/span> <span class=\"o\">==<\/span> <span class=\"s2\">&quot;)&quot;<\/span><span class=\"p\">:<\/span>\n                <span class=\"k\">return<\/span> <span class=\"n\">x<\/span>\n        <span class=\"k\">return<\/span> <span class=\"o\">-<\/span><span class=\"mi\">1<\/span>\n\n    <span class=\"k\">for<\/span> <span class=\"n\">pattern<\/span> <span class=\"ow\">in<\/span> <span class=\"n\">patterns<\/span><span class=\"p\">:<\/span>\n        <span class=\"n\">line<\/span> <span class=\"o\">=<\/span> <span class=\"p\">[]<\/span>\n        <span class=\"n\">i<\/span> <span class=\"o\">=<\/span> <span class=\"mi\">0<\/span>\n\n        <span class=\"k\">while<\/span> <span class=\"n\">i<\/span> <span class=\"o\">&lt;<\/span> <span class=\"nb\">len<\/span><span class=\"p\">(<\/span><span class=\"n\">pattern<\/span><span class=\"p\">):<\/span> \n            <span class=\"k\">if<\/span> <span class=\"n\">pattern<\/span><span class=\"p\">[<\/span><span class=\"n\">i<\/span><span class=\"p\">]<\/span> <span class=\"o\">==<\/span> <span class=\"s2\">&quot;(&quot;<\/span><span class=\"p\">:<\/span>\n                <span class=\"n\">j<\/span> <span class=\"o\">=<\/span> <span class=\"n\">getclosing<\/span><span class=\"p\">(<\/span><span class=\"n\">i<\/span><span class=\"p\">,<\/span><span class=\"n\">pattern<\/span><span class=\"p\">)<\/span>\n\n                <span class=\"k\">if<\/span> <span class=\"n\">j<\/span> <span class=\"o\">&gt;<\/span> <span class=\"n\">i<\/span> <span class=\"ow\">and<\/span> <span class=\"n\">j<\/span><span class=\"o\">-<\/span><span class=\"n\">i<\/span> <span class=\"o\">&gt;<\/span> <span class=\"mi\">2<\/span><span class=\"p\">:<\/span>\n                    <span class=\"n\">line<\/span><span class=\"o\">.<\/span><span class=\"n\">append<\/span><span class=\"p\">(<\/span><span class=\"nb\">list<\/span><span class=\"p\">(<\/span><span class=\"n\">pattern<\/span><span class=\"p\">[<\/span><span class=\"n\">i<\/span><span class=\"o\">+<\/span><span class=\"mi\">1<\/span><span class=\"p\">:<\/span><span class=\"n\">j<\/span><span class=\"p\">]))<\/span>\n                    <span class=\"n\">i<\/span> <span class=\"o\">=<\/span> <span class=\"n\">j<\/span>\n\n            <span class=\"k\">elif<\/span> <span class=\"n\">pattern<\/span><span class=\"p\">[<\/span><span class=\"n\">i<\/span><span class=\"p\">]<\/span> <span class=\"o\">!=<\/span> <span class=\"s2\">&quot;)&quot;<\/span><span class=\"p\">:<\/span>\n                <span class=\"n\">line<\/span><span class=\"o\">.<\/span><span class=\"n\">append<\/span><span class=\"p\">(<\/span><span class=\"n\">pattern<\/span><span class=\"p\">[<\/span><span class=\"n\">i<\/span><span class=\"p\">])<\/span>\n            <span class=\"n\">i<\/span> <span class=\"o\">+=<\/span> <span class=\"mi\">1<\/span>\n\n        <span class=\"n\">lines<\/span><span class=\"o\">.<\/span><span class=\"n\">append<\/span><span class=\"p\">(<\/span><span class=\"n\">line<\/span><span class=\"p\">)<\/span>\n\n    <span class=\"k\">return<\/span> <span class=\"n\">lines<\/span>\n\n<span class=\"c1\"># Check if a string matches a pattern represented as a list<\/span>\n<span class=\"k\">def<\/span> <span class=\"nf\">checkmatch<\/span><span class=\"p\">(<\/span><span class=\"n\">word<\/span><span class=\"p\">,<\/span> <span class=\"n\">pattern<\/span><span class=\"p\">):<\/span>\n    <span class=\"k\">for<\/span> <span class=\"n\">i<\/span><span class=\"p\">,<\/span><span class=\"n\">el<\/span> <span class=\"ow\">in<\/span> <span class=\"nb\">enumerate<\/span><span class=\"p\">(<\/span><span class=\"n\">word<\/span><span class=\"p\">):<\/span>\n        <span class=\"k\">return<\/span> <span class=\"err\">!<\/span><span class=\"p\">(<\/span><span class=\"nb\">type<\/span><span class=\"p\">(<\/span><span class=\"n\">pattern<\/span><span class=\"p\">[<\/span><span class=\"n\">i<\/span><span class=\"p\">])<\/span> <span class=\"o\">==<\/span> <span class=\"nb\">str<\/span> <span class=\"ow\">and<\/span> <span class=\"n\">el<\/span> <span class=\"o\">!=<\/span> <span class=\"n\">pattern<\/span><span class=\"p\">[<\/span><span class=\"n\">i<\/span><span class=\"p\">])<\/span> <span class=\"ow\">or<\/span> <span class=\"p\">(<\/span><span class=\"nb\">type<\/span><span class=\"p\">(<\/span><span class=\"n\">pattern<\/span><span class=\"p\">[<\/span><span class=\"n\">i<\/span><span class=\"p\">])<\/span><span class=\"o\">==<\/span><span class=\"nb\">list<\/span> <span class=\"ow\">and<\/span> <span class=\"n\">el<\/span> <span class=\"ow\">not<\/span> <span class=\"ow\">in<\/span> <span class=\"n\">pattern<\/span><span class=\"p\">[<\/span><span class=\"n\">i<\/span><span class=\"p\">]):<\/span>\n    <span class=\"k\">return<\/span> <span class=\"bp\">True<\/span>\n\n<span class=\"n\">f<\/span> <span class=\"o\">=<\/span> <span class=\"nb\">open<\/span><span class=\"p\">(<\/span><span class=\"s1\">&#39;A-large-practice.in&#39;<\/span><span class=\"p\">)<\/span>\n<span class=\"n\">lines<\/span> <span class=\"o\">=<\/span> <span class=\"n\">f<\/span><span class=\"o\">.<\/span><span class=\"n\">readlines<\/span><span class=\"p\">()<\/span>\n<span class=\"n\">f<\/span><span class=\"o\">.<\/span><span class=\"n\">close<\/span><span class=\"p\">()<\/span>\n\n<span class=\"n\">L<\/span><span class=\"p\">,<\/span> <span class=\"n\">D<\/span><span class=\"p\">,<\/span> <span class=\"n\">N<\/span> <span class=\"o\">=<\/span> <span class=\"nb\">map<\/span><span class=\"p\">(<\/span><span class=\"nb\">int<\/span><span class=\"p\">,<\/span><span class=\"n\">lines<\/span><span class=\"p\">[<\/span><span class=\"mi\">0<\/span><span class=\"p\">]<\/span><span class=\"o\">.<\/span><span class=\"n\">rstrip<\/span><span class=\"p\">(<\/span><span class=\"s1\">&#39;<\/span><span class=\"se\">\\n<\/span><span class=\"s1\">&#39;<\/span><span class=\"p\">)<\/span><span class=\"o\">.<\/span><span class=\"n\">split<\/span><span class=\"p\">(<\/span><span class=\"s1\">&#39; &#39;<\/span><span class=\"p\">))<\/span>\n\n<span class=\"c1\"># Get list of all alien words<\/span>\n<span class=\"n\">words<\/span> <span class=\"o\">=<\/span> <span class=\"p\">[<\/span><span class=\"n\">w<\/span><span class=\"o\">.<\/span><span class=\"n\">rstrip<\/span><span class=\"p\">(<\/span><span class=\"s1\">&#39;<\/span><span class=\"se\">\\n<\/span><span class=\"s1\">&#39;<\/span><span class=\"p\">)<\/span> <span class=\"k\">for<\/span> <span class=\"n\">w<\/span> <span class=\"ow\">in<\/span> <span class=\"n\">lines<\/span><span class=\"p\">[<\/span><span class=\"mi\">1<\/span><span class=\"p\">:<\/span><span class=\"n\">D<\/span><span class=\"o\">+<\/span><span class=\"mi\">1<\/span><span class=\"p\">]]<\/span>\n\n<span class=\"c1\"># Get list of patters and parse them<\/span>\n<span class=\"n\">p_patterns<\/span> <span class=\"o\">=<\/span> <span class=\"n\">parse_patterns<\/span><span class=\"p\">([<\/span><span class=\"n\">w<\/span><span class=\"o\">.<\/span><span class=\"n\">rstrip<\/span><span class=\"p\">(<\/span><span class=\"s1\">&#39;<\/span><span class=\"se\">\\n<\/span><span class=\"s1\">&#39;<\/span><span class=\"p\">)<\/span> <span class=\"k\">for<\/span> <span class=\"n\">w<\/span> <span class=\"ow\">in<\/span> <span class=\"n\">lines<\/span><span class=\"p\">[(<\/span><span class=\"n\">D<\/span><span class=\"o\">+<\/span><span class=\"mi\">1<\/span><span class=\"p\">):(<\/span><span class=\"n\">D<\/span><span class=\"o\">+<\/span><span class=\"n\">N<\/span><span class=\"o\">+<\/span><span class=\"mi\">1<\/span><span class=\"p\">)]])<\/span>\n\n<span class=\"c1\"># Keep track of number of matches for each pattern here<\/span>\n<span class=\"n\">matches<\/span> <span class=\"o\">=<\/span> <span class=\"p\">[<\/span><span class=\"mi\">0<\/span><span class=\"p\">]<\/span><span class=\"o\">*<\/span><span class=\"nb\">len<\/span><span class=\"p\">(<\/span><span class=\"n\">p_patterns<\/span><span class=\"p\">)<\/span>\n\n<span class=\"c1\"># Check every word for matching with any pattern<\/span>\n<span class=\"k\">for<\/span> <span class=\"n\">word<\/span> <span class=\"ow\">in<\/span> <span class=\"n\">words<\/span><span class=\"p\">:<\/span>\n    <span class=\"k\">for<\/span> <span class=\"n\">i<\/span><span class=\"p\">,<\/span> <span class=\"n\">pattern<\/span> <span class=\"ow\">in<\/span> <span class=\"nb\">enumerate<\/span><span class=\"p\">(<\/span><span class=\"n\">p_patterns<\/span><span class=\"p\">):<\/span>\n        <span class=\"k\">if<\/span> <span class=\"n\">checkmatch<\/span><span class=\"p\">(<\/span><span class=\"n\">word<\/span><span class=\"p\">,<\/span><span class=\"n\">pattern<\/span><span class=\"p\">):<\/span>\n            <span class=\"n\">matches<\/span><span class=\"p\">[<\/span><span class=\"n\">i<\/span><span class=\"p\">]<\/span> <span class=\"o\">+=<\/span> <span class=\"mi\">1<\/span>\n\n<span class=\"c1\"># Print results<\/span>\n<span class=\"k\">for<\/span> <span class=\"n\">i<\/span> <span class=\"ow\">in<\/span> <span class=\"nb\">xrange<\/span><span class=\"p\">(<\/span><span class=\"mi\">0<\/span><span class=\"p\">,<\/span><span class=\"nb\">len<\/span><span class=\"p\">(<\/span><span class=\"n\">matches<\/span><span class=\"p\">)):<\/span>\n    <span class=\"k\">print<\/span> <span class=\"s2\">&quot;Case #{0}: {1}&quot;<\/span><span class=\"o\">.<\/span><span class=\"n\">format<\/span><span class=\"p\">(<\/span><span class=\"n\">i<\/span><span class=\"o\">+<\/span><span class=\"mi\">1<\/span><span class=\"p\">,<\/span><span class=\"n\">matches<\/span><span class=\"p\">[<\/span><span class=\"n\">i<\/span><span class=\"p\">])<\/span>\n<\/pre><\/div>\n\n\n<p>This is a simple solution yet turned out to work very well after I tested it against the large input file. I am currently looking into optimizing the code a looking into a possible more efficient solution. Would love to hear your thoughts about this problem and the code. Have a good day and happy coding!<\/p>"},{"title":"The undefined vs null Pitfall in Javascript","link":{"@attributes":{"href":"https:\/\/www.thecodeship.com\/web-development\/the-undefined-vs-null-pitfall-in-javascript\/","rel":"alternate"}},"published":"2013-03-23T21:06:00+02:00","updated":"2013-03-23T21:06:00+02:00","author":{"name":"Ayman Farhat"},"id":"tag:www.thecodeship.com,2013-03-23:\/web-development\/the-undefined-vs-null-pitfall-in-javascript\/","summary":"<p>Sometimes I come across JavaScript code with a lot of null checking against properties that are uninitialized, which is really wrong. Many people might think that uninitialized properties get a default value of null, but that is not the case despite having something like foo == null resulting in true, assuming ...<\/p>","content":"<p>Sometimes I come across JavaScript code with a lot of null checking against properties that are uninitialized, which is really wrong. Many people might think that uninitialized properties get a default value of null, but that is not the case despite having something like foo == null resulting in true, assuming foo is an undefined variable in this example.<\/p>\n<p>The result mentioned above is the way it is, because the == operator does <a href=\"http:\/\/en.wikipedia.org\/wiki\/Type_conversion\">type coercion<\/a> so that it can compare the two for their values. Thus when you do something like <code>foo == null<\/code> (Equality), JavaScript automatically converts it to the type null so it can carry out the comparison on two values with the same types, and obviously in this comparison the result will end up being true.<\/p>\n<p>On the other hand if you did <code>foo === null<\/code> (Identity) which checks for both type and value the result will be false which is the correct result because there is a type mismatch between the two. foo's type is undefined while null is an object. You can check that for yourself via <code>typeof(null)<\/code> and <code>typeof(undefined)<\/code>. Actually null <a href=\"http:\/\/www.2ality.com\/2011\/03\/javascript-values-not-everything-is.html\">is not really<\/a> an object but a primitive value in JavaScript. Having typeof(null) output \"object\" is considered to be a bug in the language. Despite that, the point here is that null and undefined are of different types anyways.<\/p>\n<p>A property that has no definition is undefined, which also means no value, no type and never been referenced before in the scope. On the other hand, null is where the property exists but it isn't known what its value is.<\/p>\n<p>As a general rule, it is good to use === instead of == in JavaScript when comparing values especially when there is a huge possibility that your program will end up comparing values of different types at some point. What is dangerous about == is that it performs all kinds of conversions between types which could produce unexpected results in your program.<\/p>","category":[{"@attributes":{"term":"javascript"}},{"@attributes":{"term":"null"}},{"@attributes":{"term":"undefined"}},{"@attributes":{"term":"web"}}]},{"title":"TypeScript: Enhanced Javascript","link":{"@attributes":{"href":"https:\/\/www.thecodeship.com\/web-development\/typescript-enhanced-javascript\/","rel":"alternate"}},"published":"2013-03-07T15:00:00+02:00","updated":"2013-03-07T15:00:00+02:00","author":{"name":"Ayman Farhat"},"id":"tag:www.thecodeship.com,2013-03-07:\/web-development\/typescript-enhanced-javascript\/","summary":"<p>One of the most important platforms nowadays is the web, and Javascript incorporates a large share from any web application today more than ever. As we get more into the single page applications trend and Javascript on the server (such as NodeJS), this language can't be really dealt with ...<\/p>","content":"<p>One of the most important platforms nowadays is the web, and Javascript incorporates a large share from any web application today more than ever. As we get more into the single page applications trend and Javascript on the server (such as NodeJS), this language can't be really dealt with as secondary anymore. In fact it isn't just about the web, Javascript is extensively used in cross mobile app frameworks and also making its way into the desktop, it is everywhere. Yet, there is a large number of developers, enjoying languages like C#, who don't like working with Javascript for reasons such not being a strongly typed language, and not having class structures. <\/p>\n<p>This is where <a href=\"http:\/\/www.typescriptlang.org\/\">TypeScript<\/a> kicks in.<\/p>\n<h2>How can TypeScript help?<\/h2>\n<p>TypeScript isn't a language designed to replace javascript, it's a typed superset of it. This means that JavaScript code would be valid TypeScript, and what the compiler generates is JavaScript. The language was designed with the .net developers in mind and that's why it packs many cool features you may already be using in your favorite .net language such as C#. In reality all \"features\" can be achieved in Javascript by following certain patterns. Yet an advantage other than having syntactic sugar for Javascript is that you get rid from all the boilerplate Javascript code you end up writing to achieve those patterns. MSDN have published a very nice article showing off some of TypeScript's features along with comparison against classic Javascript code. The result is concise code that is enjoyable to read and maintain. I highly recommend <a href=\"http:\/\/msdn.microsoft.com\/en-us\/magazine\/jj883955.aspx\">checking the examples out<\/a>.<\/p>\n<h2>What about Coffee Script and Dart?<\/h2>\n<p>Since the release of TypeScript there have been a lot of comparison with <a href=\"http:\/\/coffeescript.org\/\">Coffee Script<\/a> and <a href=\"http:\/\/www.dartlang.org\/\">Dart<\/a>. They're both different than each other and TypeScript. In both of those languages you're not writing any Javascript, they have their own languages that get compiled to Javascript code. On the other hand, TypeScript is a superset of Javascript where you write it just like you write Javascript, and any Javascript code is already valid TypeScript. A superset language is one that contains all the features of a given language and has been expanded or enhanced to include other features as well.<\/p>\n<h2>What's ahead?<\/h2>\n<p>In my opinion Typescript represents what Javascript is supposed to be, or will become like one day. Afterall, ECMAScript is progressing and some of the features supported in TypeScript will probably be available in ECMAScript 6 such as <a href=\"http:\/\/wiki.ecmascript.org\/doku.php?id=strawman:maximally_minimal_classes\">classes<\/a>, <a href=\"http:\/\/wiki.ecmascript.org\/doku.php?id=harmony:modules&amp;s=modules+syntax\">modules<\/a>, and <a href=\"http:\/\/wiki.ecmascript.org\/doku.php?id=harmony:arrow_function_syntax\">arrow functions<\/a>. As we head to a future dominated by Javascript, the language will keep moving on with new features. Alternatives to writing Javascript like Coffee Script are already getting popular, while some good features available in TypeScript are being weaved into ECMAScript 6. Where do you and your work stand from all what's going on? Would love to hear your opinion.<\/p>","category":[{"@attributes":{"term":"superset"}},{"@attributes":{"term":"TypeScript"}},{"@attributes":{"term":"enhanced"}}]},{"title":"Deploy Django on Apache with Virtualenv and mod_wsgi","link":{"@attributes":{"href":"https:\/\/www.thecodeship.com\/deployment\/deploy-django-apache-virtualenv-and-mod_wsgi\/","rel":"alternate"}},"published":"2013-03-03T04:03:00+02:00","updated":"2013-03-03T04:03:00+02:00","author":{"name":"Ayman Farhat"},"id":"tag:www.thecodeship.com,2013-03-03:\/deployment\/deploy-django-apache-virtualenv-and-mod_wsgi\/","summary":"<p>I have recently got into Django and built a small project with it which is this blog. Then came the time for me to deploy the project on an unmanaged VPS, the process went fine yet with a few problems here and there due to the lack of a definitive ...<\/p>","content":"<p>I have recently got into Django and built a small project with it which is this blog. Then came the time for me to deploy the project on an unmanaged VPS, the process went fine yet with a few problems here and there due to the lack of a definitive guide to deployment. So I ended up checking several sources to have everything work the way I wanted. In this post, I'll be sharing the whole process with you in detail and what will hopefully be helpful to the fellow newcomers.<\/p>\n<h2>Prerequisites<\/h2>\n<p>In this tutorial we will assume that we have a fresh Ubuntu server installation with Python already installed and nothing else. Furthermore, you should be able to have shell access to the server. If you're looking for a good and affordable VPS hosting I highly recommend <a href=\"https:\/\/www.digitalocean.com\/?refcode=b45671a67881\">Digital Ocean<\/a>.<\/p>\n<h2>Installing Pip<\/h2>\n<p>Pip is a tool for installing and managing Python packages,basically a replacement for easy_install, many Python developers should be already familiar with it and have it already. If not you can install it easily via<\/p>\n<div class=\"highlight\"><pre><span><\/span>$ sudo apt-get install python-pip python-dev build-essential\n<\/pre><\/div>\n\n\n<p>Then upgrade to latest version<\/p>\n<div class=\"highlight\"><pre><span><\/span>$ sudo pip install --upgrade pip\n<\/pre><\/div>\n\n\n<p>The python-dev and build-essential packages are recommended to install along, because it isn't possible to install any Python module that ships with a C extension without them later on.<\/p>\n<h2>Setting up virtualenv<\/h2>\n<p><a href=\"http:\/\/www.virtualenv.org\/\">Virtualenv<\/a> is, a tool for creating isolated Python environments. Virtualenv helps you create several environments that have their own installation directories for setting up private python environments and their libraries that are specific to your project and its needs. Its really a neat tool that is extremely recommended especially on your development machine for dealing with multiple Django projects and their dependencies. On top of virtualenv we have <a href=\"http:\/\/doughellmann.com\/2008\/05\/virtualenvwrapper.html\">virtualenvwrapper<\/a> which is basically a set of extensions that include wrappers that make it much easier to manage your virtualenvs such as creating, deleting and switching between them. So lets install virtualenvwrapper via pip, and virtualenv(its prerequisite) will get fetched and installed automatically<\/p>\n<div class=\"highlight\"><pre><span><\/span>$ pip install virtualenvwrapper\n<\/pre><\/div>\n\n\n<p>Next we need to set the location where the virtual environments will live and the location of the script we just installed(virtualenvwrapper.sh) so that we can access it later on. To do that, you'll have to add a couple of lines to your login shell startup file which is .bash_profile in this case. This file is located within your user's home directory. Open the file<\/p>\n<div class=\"highlight\"><pre><span><\/span>$ sudo nano ~\/.bash_profile\n<\/pre><\/div>\n\n\n<p>Set the following lines<\/p>\n<div class=\"highlight\"><pre><span><\/span><span class=\"n\">export<\/span> <span class=\"n\">WORKON_HOME<\/span><span class=\"o\">=<\/span><span class=\"err\">$<\/span><span class=\"n\">HOME<\/span><span class=\"o\">\/<\/span><span class=\"p\">.<\/span><span class=\"n\">virtualenvs<\/span>\n<span class=\"k\">source<\/span> <span class=\"o\">\/<\/span><span class=\"n\">usr<\/span><span class=\"o\">\/<\/span><span class=\"k\">local<\/span><span class=\"o\">\/<\/span><span class=\"n\">bin<\/span><span class=\"o\">\/<\/span><span class=\"n\">virtualenvwrapper<\/span><span class=\"p\">.<\/span><span class=\"n\">sh<\/span>\n<\/pre><\/div>\n\n\n<p>Reload your startup file<\/p>\n<div class=\"highlight\"><pre><span><\/span>$ <span class=\"nb\">source<\/span> ~\/.bash_profile\n<\/pre><\/div>\n\n\n<p>Now that the file is loaded we can access the script which will ease managing our virtual environments. For more information about shell startup files(.bashrc and .bash_profile) check this <a href=\"http:\/\/hacktux.com\/bash\/bashrc\/bash_profile\">article<\/a><\/p>\n<h2>Creating a virtual environment for the project<\/h2>\n<p>To create a new virtual environment for our project execute<\/p>\n<div class=\"highlight\"><pre><span><\/span>$ mkvirtualenv myprojectenv --no-site-packages\n<\/pre><\/div>\n\n\n<p>This creates a new virtual environment on the server just for our project without any inherent packages(thanks to the not-site-packages) option. myprojectenv now has just the essentials Python, and Pip. When you create a new virtualenv it will get switched to it automatically so you'll be seeing the name of your environment in the terminal as long as its loaded. To get out of your currently loaded virtualenv simply enter \"deactivate\". To get into a virtualenv and manage it enter \"workon\" followed by the name of your available virtualenv such as myprojectenv. Introducing virtualenv and virtualenvwrapper in detail is beyond the scope of this tutorial so I suggest to read more about them independantly.<\/p>\n<h2>Uploading your project and installing dependencies<\/h2>\n<p>Now that the virtual environment is ready, we're going to upload our project files to some directory on the server I will choose for this example: home\/django_projects\/MyProject.<\/p>\n<h3>Generate requirements<\/h3>\n<p>But before upload lets generate a requirements file for our project on the development machine, from the working environment of the project, via pip:<\/p>\n<div class=\"highlight\"><pre><span><\/span>$ pip freeze &gt; requirements.txt\n<\/pre><\/div>\n\n\n<p>This will generate a list of all installed libraries and their version numbers within the current working environment. You may want to edit that file in case there are unnecessary libraries listed.<\/p>\n<h3>Upload files<\/h3>\n<p>Now upload your project files to your server via ftp or scp to <code>\/home\/django_projects\/MyProject\/<\/code><\/p>\n<h3>Install dependencies<\/h3>\n<p>After the upload navigate to your MyProject directory<\/p>\n<div class=\"highlight\"><pre><span><\/span>$ <span class=\"nb\">cd<\/span> \/home\/django_projects\/MyProject\/\n<\/pre><\/div>\n\n\n<p>Make sure you're working on the virtualenv of that particular project, if not<\/p>\n<div class=\"highlight\"><pre><span><\/span>$ workon myprojectenv\n<\/pre><\/div>\n\n\n<p>Install the dependencies in requirements.txt via pip<\/p>\n<div class=\"highlight\"><pre><span><\/span>$ pip install -r requirements.txt \n<\/pre><\/div>\n\n\n<h2>Installing and configuring Apache<\/h2>\n<p>Now that the project files are in place and the python requirements installed we need to set up apache to execute a .wsgi file that will launch our django application.<\/p>\n<h3>Install apache2 and mod_wsgi<\/h3>\n<p>Install apache2 and some required components<\/p>\n<div class=\"highlight\"><pre><span><\/span>$ sudo apt-get install apache2 apache2.2-common apache2-mpm-prefork apache2-utils libexpat1\n<\/pre><\/div>\n\n\n<p>Now lets install mod_wsgi, which enables you to serve python web apps from Apache server. It is also one of the recommended ways of getting Django into production.<\/p>\n<div class=\"highlight\"><pre><span><\/span>$ sudo apt-get install libapache2-mod-wsgi\n<\/pre><\/div>\n\n\n<p>After install make sure you restart apache for the changes to take effect<\/p>\n<div class=\"highlight\"><pre><span><\/span>$ sudo service apache2 restart\n<\/pre><\/div>\n\n\n<h3>Create a virtual host<\/h3>\n<p>Open up or create the virtualhost file for your domain<\/p>\n<div class=\"highlight\"><pre><span><\/span>$ sudo nano \/etc\/apache2\/sites-available\/mydomain.com\n<\/pre><\/div>\n\n\n<p>Create your new virtual host node which should look something like this<\/p>\n<div class=\"highlight\"><pre><span><\/span><span class=\"nt\">&lt;VirtualHost<\/span> <span class=\"err\">*:80<\/span><span class=\"nt\">&gt;<\/span>\n    ServerAdmin webmaster@mydomain.com\n    ServerName mydomain.com\n    ServerAlias www.mydomain.com\n    WSGIScriptAlias \/ var\/www\/mydomain.com\/index.wsgi\n\n    Alias \/static\/ \/var\/www\/mydomain.com\/static\/\n    <span class=\"nt\">&lt;Location<\/span> <span class=\"err\">&quot;\/static\/&quot;<\/span><span class=\"nt\">&gt;<\/span>\n        Options -Indexes\n    <span class=\"nt\">&lt;\/Location &gt;<\/span>\n<span class=\"nt\">&lt;\/VirtualHost &gt;<\/span>\n<\/pre><\/div>\n\n\n<p>A couple of things to notice here. WSGIScriptAlias denotes the location of wsgi file that is going to get executed when our domain gets accessed.This file will have some Python code which will run our Django app as you'll see later. The other alias denotes the location of the static files for my django project. This contents of the static folder will contain all static files belonging to all the apps within my django project. The content will later be generated via the collectstatic command. You will also notice that I have put those files away from my home folder. \/var\/www\/mydomain it just a convention which I prefer especially for static content being under the \"document root\".<\/p>\n<h3>Create the wsgi file<\/h3>\n<p>Now that apache points to \/var\/www\/mydomain.com\/index.wsgi we need to create that file.<\/p>\n<div class=\"highlight\"><pre><span><\/span><span class=\"n\">sudo<\/span> <span class=\"n\">nano<\/span> <span class=\"o\">\/<\/span><span class=\"n\">var<\/span><span class=\"o\">\/<\/span><span class=\"n\">www<\/span><span class=\"o\">\/<\/span><span class=\"n\">mydomain<\/span><span class=\"p\">.<\/span><span class=\"n\">com<\/span><span class=\"o\">\/<\/span><span class=\"k\">index<\/span><span class=\"p\">.<\/span><span class=\"n\">wsgi<\/span>\n<\/pre><\/div>\n\n\n<p>Your Python code in that file should be similar to this<\/p>\n<div class=\"highlight\"><pre><span><\/span><span class=\"kn\">import<\/span> <span class=\"nn\">os<\/span>\n<span class=\"kn\">import<\/span> <span class=\"nn\">sys<\/span>\n<span class=\"kn\">import<\/span> <span class=\"nn\">site<\/span>\n\n<span class=\"c1\"># Add the site-packages of the chosen virtualenv to work with<\/span>\n<span class=\"n\">site<\/span><span class=\"o\">.<\/span><span class=\"n\">addsitedir<\/span><span class=\"p\">(<\/span><span class=\"s1\">&#39;~\/.virtualenvs\/myprojectenv\/local\/lib\/python2.7\/site-packages&#39;<\/span><span class=\"p\">)<\/span>\n\n<span class=\"c1\"># Add the app&#39;s directory to the PYTHONPATH<\/span>\n<span class=\"n\">sys<\/span><span class=\"o\">.<\/span><span class=\"n\">path<\/span><span class=\"o\">.<\/span><span class=\"n\">append<\/span><span class=\"p\">(<\/span><span class=\"s1\">&#39;\/home\/django_projects\/MyProject&#39;<\/span><span class=\"p\">)<\/span>\n<span class=\"n\">sys<\/span><span class=\"o\">.<\/span><span class=\"n\">path<\/span><span class=\"o\">.<\/span><span class=\"n\">append<\/span><span class=\"p\">(<\/span><span class=\"s1\">&#39;\/home\/django_projects\/MyProject\/myproject&#39;<\/span><span class=\"p\">)<\/span>\n\n<span class=\"n\">os<\/span><span class=\"o\">.<\/span><span class=\"n\">environ<\/span><span class=\"p\">[<\/span><span class=\"s1\">&#39;DJANGO_SETTINGS_MODULE&#39;<\/span><span class=\"p\">]<\/span> <span class=\"o\">=<\/span> <span class=\"s1\">&#39;myproject.settings&#39;<\/span>\n\n<span class=\"c1\"># Activate your virtual env<\/span>\n<span class=\"n\">activate_env<\/span><span class=\"o\">=<\/span><span class=\"n\">os<\/span><span class=\"o\">.<\/span><span class=\"n\">path<\/span><span class=\"o\">.<\/span><span class=\"n\">expanduser<\/span><span class=\"p\">(<\/span><span class=\"s2\">&quot;~\/.virtualenvs\/myprojectenv\/bin\/activate_this.py&quot;<\/span><span class=\"p\">)<\/span>\n<span class=\"nb\">execfile<\/span><span class=\"p\">(<\/span><span class=\"n\">activate_env<\/span><span class=\"p\">,<\/span> <span class=\"nb\">dict<\/span><span class=\"p\">(<\/span><span class=\"vm\">__file__<\/span><span class=\"o\">=<\/span><span class=\"n\">activate_env<\/span><span class=\"p\">))<\/span>\n\n<span class=\"kn\">import<\/span> <span class=\"nn\">django.core.handlers.wsgi<\/span>\n<span class=\"n\">application<\/span> <span class=\"o\">=<\/span> <span class=\"n\">django<\/span><span class=\"o\">.<\/span><span class=\"n\">core<\/span><span class=\"o\">.<\/span><span class=\"n\">handlers<\/span><span class=\"o\">.<\/span><span class=\"n\">wsgi<\/span><span class=\"o\">.<\/span><span class=\"n\">WSGIHandler<\/span><span class=\"p\">()<\/span>\n<\/pre><\/div>\n\n\n<p>site.addsitedir() will let you use the site packages and dependencies within the virtualenv you created initially for your specific project with your specific packages. sys.path.append() adds your project's directory to the Python path. It is advised you add both the main directory holding your apps and its child that contains the settings.py file which in this case is \"myproject\".<\/p>\n<h2>Static files<\/h2>\n<p>Now we need to set our static files directory and generate the static files. Navigate to your settings.py file in your project and update the following settings:<\/p>\n<div class=\"highlight\"><pre><span><\/span><span class=\"n\">STATIC_ROOT<\/span> <span class=\"o\">=<\/span> <span class=\"s1\">&#39;\/var\/www\/mydomain.com\/static\/&#39;<\/span>\n<span class=\"n\">STATIC_URL<\/span> <span class=\"o\">=<\/span> <span class=\"s1\">&#39;\/static\/&#39;<\/span>\n<\/pre><\/div>\n\n\n<p>Make sure that you have created the static folder. Now within MyProject execute collectstatic<\/p>\n<div class=\"highlight\"><pre><span><\/span>$ python manage.py collectstatic\n<\/pre><\/div>\n\n\n<p>Everything should be set now. Restart apache for the changes on your virtual host to take effect and everything should work well.<\/p>\n<div class=\"highlight\"><pre><span><\/span>$ sudo service apache2 restart\n<\/pre><\/div>\n\n\n<p>Basically thats all what it takes to deploy a django project on a vanilla Ubuntu server. I tried to make the steps as detailed and friendly as possible, in case you spot any error please feel free to share in the comments. Also I am open to any ideas regarding better execution of anything related to this task. Definitely there are still a couple of things to discuss about this topic such as database deployment, migrations, caching, and running mod_wsgi in daemon mode(which is extremely recommended especially if you're running multiple Django projects). I'll be discussing all that in later posts so stay tuned!<\/p>","category":[{"@attributes":{"term":"virtualenv"}},{"@attributes":{"term":"wsgi"}},{"@attributes":{"term":"deploy"}},{"@attributes":{"term":"apache2"}},{"@attributes":{"term":"django"}},{"@attributes":{"term":"tutorial"}}]},{"title":"Birth","link":{"@attributes":{"href":"https:\/\/www.thecodeship.com\/general\/birth\/","rel":"alternate"}},"published":"2013-02-26T17:35:00+02:00","updated":"2013-02-26T17:35:00+02:00","author":{"name":"Ayman Farhat"},"id":"tag:www.thecodeship.com,2013-02-26:\/general\/birth\/","summary":"<p>Hello everyone, I have recently decided to start a blog for sharing my thoughts about programming, software, web development and everything that goes around those topics. Programming is not the simple skill you acquire within a defined period of time and keep using it over and over again within repetitive ...<\/p>","content":"<p>Hello everyone, I have recently decided to start a blog for sharing my thoughts about programming, software, web development and everything that goes around those topics.<\/p>\n<p>Programming is not the simple skill you acquire within a defined period of time and keep using it over and over again within repetitive tasks without learning anything new. Despite the fact that many programmers, developers and students think of it that way, the reality is completely different. It is way much more interesting than that!<\/p>\n<p>Computer programming and software development is a journey of a lifetime which involves devotion, learning new ideas every day, exploring rapidly changing technologies and becoming a better programmer year after year. All in all it is an enjoyable adventure to those who love continuous learning and problem solving.<\/p>\n<p>So as you may have inferred from the name of this blog \"The Code Ship\", we will be sailing together on a journey into a sea of computer programming adventures. On the way we will be discussing everyday problems and their possible solutions, classic and non classic algorithm problems, introducing recent technologies\/frameworks and much more. Every blog post here will be reflecting part of my progress and activities within the programming and software development world. As I learn I will be documenting what I have here.<\/p>\n<p>I hope that you will enjoy reading the coming posts, looking forward for a great journey together!<\/p>\n<p>Ayman<\/p>","category":[{"@attributes":{"term":"introduction"}},{"@attributes":{"term":"birth"}}]}]}