{"@attributes":{"version":"2.0"},"channel":{"title":"DEV Community: edA\u2011qa mort\u2011ora\u2011y","description":"The latest articles on DEV Community by edA\u2011qa mort\u2011ora\u2011y (@mortoray).","link":"https:\/\/dev.to\/mortoray","image":{"url":"https:\/\/media2.dev.to\/dynamic\/image\/width=90,height=90,fit=cover,gravity=auto,format=auto\/https:%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F16729%2F332ce8ed-1887-41dc-b8df-b14e172486d8.jpg","title":"DEV Community: edA\u2011qa mort\u2011ora\u2011y","link":"https:\/\/dev.to\/mortoray"},"language":"en","item":[{"title":"System Architecture for Edaqa's Room","pubDate":"Thu, 15 Apr 2021 18:34:58 +0000","link":"https:\/\/dev.to\/mortoray\/system-architecture-for-edaqa-s-room-5655","guid":"https:\/\/dev.to\/mortoray\/system-architecture-for-edaqa-s-room-5655","description":"<p>I tried explaining to a friend how <a href=\"https:\/\/edaqa.link\/EdaqasRoom\" rel=\"noopener noreferrer\">my games<\/a> were setup, but it became confusing quickly. Drawing all the component boxes, I\u2019m surprised to see how complex it has become. I think it\u2019s a decent example of modern system architecture, and will go through the setup here. This is for a multiplayer game, so I\u2019ll point out how this might differ from a more typical web application.<\/p>\n\n<p>I could reasonably call this architecture the platform on which my game runs. A higher-level of code runs on top of, but is intimately tied, to this platform.<\/p>\n\n<p><a href=\"https:\/\/media.dev.to\/dynamic\/image\/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto\/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2F2qfv0dqo689en1j93a4e.png\" class=\"article-body-image-wrapper\"><img src=\"https:\/\/media.dev.to\/dynamic\/image\/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto\/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2F2qfv0dqo689en1j93a4e.png\" alt=\"Architecture diagram\"><\/a><\/p>\n\n<h1>\n  \n  \n  Client\n<\/h1>\n\n<p>I like to start at <a href=\"https:\/\/mortoray.com\/2019\/02\/05\/the-user\/\" rel=\"noopener noreferrer\">the user\u2019s view<\/a> on the system, as it keeps me grounded in the system's purpose. Mostly the user interacts via the website, but I also send email confirmation on purchase. The starting point to the game could be via the immediate web link, or the link in the email.<\/p>\n\n<p>I was tempted to split the client into a game and website proper, as they are fairly distinct aspects of the system. But the discussion of the website\u2019s logical structure is better left for another article.<\/p>\n\n<p>Note the two lines from the browser to the HTTP server. One is normal HTTP traffic, and the other is for WebSocket. Though they go through the same machines, they are handled differently. I\u2019ll provide more detail later, but the way I handle WebSocket is specific to a multiplayer game \u2014 a need for a fast response motivates the design.<\/p>\n\n<p>In terms of fault tolerance, it\u2019s the client which is most likely to fail. From browser incompatibility to crashes, and slow or lost connections, the client is an endless pool of problems. The servers are virtually faultless by comparison. As this is an interactive multiplayer game, it\u2019s vital to handle common client problems correctly.  The higher level code handles most of the faults, which this architecture supporting it.<\/p>\n\n<h1>\n  \n  \n  Cloud Processing Services\n<\/h1>\n\n<p>The three red boxes contain the abstract aspects of the cloud service. These services are mainly configurations and I have no insight into their internal structure. They contain only transient data.<\/p>\n\n<ul>\n<li>\n<strong>Content Delivery Network (CDN):<\/strong> The CDN serves all the static assets of the website and the game. Most of these resources use the web server as the origin, as it gives me the cleanest control over versions. The CDN provides faster loading to the client and reduces load on the host machines. I could do an entire article on the challenges of getting this working. (Service: AWS CloudFront)<\/li>\n<li>\n<strong>HTTP Frontend:<\/strong> This takes care of the incoming connections, as well as SSL handling. It provides, when needed, a slow rollout to upgrading the hosts. It\u2019s a security barrier between the public world and my private hosts. Thankfully, it routes both normal HTTP and Websocket traffic. (Service: AWS Elastic Load Balancer) <\/li>\n<li>\n<strong>Email Sender:<\/strong> Sends purchase confirmation emails to the user. I mentioned the client layer is fault prone, and email is no exception. You absolutely want a third-party service handling the challenging requirements of modern email. (Service: AWS Simple Email Service)<\/li>\n<\/ul>\n\n<h1>\n  \n  \n  Host\n<\/h1>\n\n<p>My host contains several microservices, which I\u2019m grouping into a large block. With Python as the main server language, I was forced into the microservice architecture. Separate processes is the only way I can get stability and parallel processing of these services.<\/p>\n\n<p>These are all launched as systemd services on an AWS Linux image.<\/p>\n\n<ul>\n<li>\n<strong>Web Server:<\/strong> Handles all web requests, including static files, templates, game launchers, and APIs. These requests are stateless. (Service: Python Code with Eventlet and Flask)<\/li>\n<li>\n<strong>Game Server:<\/strong> Implements the game message queues, which are shared message rooms per game \u2014 think of it like a chat server with channels. This is stateful per game. It handles client connections and transmits messages but does not understand the logical game state. For fault tolerance, it was vital that misbehaving clients don\u2019t interfere with other games. (Python Code with Asyncio and Websockets)<\/li>\n<li>\n<strong>Message Service:<\/strong> Migrates game messages from the live database to the long-term database store. This happens regularly to minimize the memory use of the live database, allowing more games to live on one host. (Service: Python Code)<\/li>\n<li>\n<strong>Confirm Service:<\/strong> Sends emails when somebody purchases a game. I avoid doing any external processing in the web server itself, instead having it post a job that is handled by this service. This keeps the web server responsive and stable. (Service: Python Code)<\/li>\n<li>\n<strong>Stats Service:<\/strong> This is a relatively fresh addition, needed for my <a href=\"https:\/\/edaqa.link\/EdaqasAffiliates\" rel=\"noopener noreferrer\">affiliate program<\/a>. I previously calculated game stats offline for analysis, but am working on features to present those at the end of the game. There is a bit of ping-pong with the web server to get this working. This is external, as it has slow DB queries and slow processing. It operates sequentially, as I do not want multiple stats running in parallel. (Service: Python Code)<\/li>\n<li>\n<strong>Live Database:<\/strong> Contains game state for all games on this host. The game uses a sequenced message queue. For a synchronized visual response between players, it is vital this service is fast. Therefore I use a local Redis store to keep live messages, with the message service moving them offline. (Service: Redis)<\/li>\n<li>\n<strong>Message Queue:<\/strong> Provides the message queue for these services to talk to each other. This is per-host because a few of the services need access to the Live Data for a game. The Confirm service does not need live data, and I could orchestrate the stats service to not need it either. However, having an additional shared message queue is unnecessary overhead.  (Service: Redis)<\/li>\n<\/ul>\n\n<p>The diagram creates siblings of the Live Database and Message Queue boxes, since the same process implements both. This is another point where the needs of the game dictate this local Redis server. Most web apps can probably use an off host queue and an external DB service. When you look at my alternate design later, you\u2019ll see I\u2019d be happy to have this part even faster.<\/p>\n\n<p>I estimate a host can handle at least 100 concurrent games, around 400 users, and I dream about the day when I need many hosts. I can also add region specific hosts, providing faster turnaround for groups playing in other countries.<\/p>\n\n<h2>\n  \n  \n  WebSocket\n<\/h2>\n\n<p>The diagram shows two different connections between the client and the HTTP Frontend, which continue to the backend.<\/p>\n\n<p>The black HTTP connection is stateless, and it doesn\u2019t matter which host it ends up at. Ultimately, when my dreams of high load come to fruition, I\u2019d separate this, putting it on a different host pool, or potentially recreate it as lambda functions.<\/p>\n\n<p>The orange WebSocket connection is stateful and must always arrive at the same machine. This is sticky per game; all players of the same game must reach the same machine.  This must be done as a single host to minimize turnaround time. Shared, non-local queues, lambda functions, and DBs, all introduce too much of a response lag. This is particular to a multiplayer game.<\/p>\n\n<h2>\n  \n  \n  Alternate Game Server Design\n<\/h2>\n\n<p>Again, I\u2019m kind of forced into the above architecture because of Python. Should I ever need more performance, or wish to reduce hardware needs, I\u2019d reimplement this, likely choosing C++, though any compiled static language with good threading and async IO would work.<\/p>\n\n<p>A new single server would be a single application replacing these services:<\/p>\n\n<ul>\n<li>\n<strong>game server:<\/strong> Depending on the language and framework, this socket handling code could look very different. Much of the speed improvement though would come simply from better data parsing and encoding.<\/li>\n<li>\n<strong>message service:<\/strong> I\u2019d gain more control over when this runs and have an easier time reloading messages for clients<\/li>\n<li>\n<strong>stats service:<\/strong> I would make this a lot simpler since it wouldn\u2019t need as much cross-process coordination to work.<\/li>\n<li>\n<strong>live database:<\/strong> Simple in memory collections replace the Redis DB, providing faster turnaround, but complicating persistence and fault management.<\/li>\n<li>\n<strong>message queue:<\/strong> The remaining job messages would migrate to a shared queue, like SQS.<\/li>\n<\/ul>\n\n<p>This alternate architecture is simpler, at least to me, and I estimate it could easily handle 100x as many games on a single host. Or rather, it\u2019d let me handle as many games as now, but with several much smaller hosts. That would improve fault tolerance.<\/p>\n\n<p>Added coding time keeps this on the long-term backlog. Unless some here-to-unknown feature appears where I need this, it\u2019ll be cheaper to keep the microservices model and spin up more hosts as required.<\/p>\n\n<p>An intermediate solution is to code strictly the websocket channels in another language, since it\u2019s the most inefficient part. Though I recently reprogrammed this part, still in Python, to be <a href=\"https:\/\/mortoray.com\/2020\/12\/06\/high-throughput-game-message-server-with-python-websockets\/\" rel=\"noopener noreferrer\">massively more efficient<\/a>. New rewrites are on the long-term backlog.<\/p>\n\n<h1>\n  \n  \n  Storage\n<\/h1>\n\n<p>The storage boxes contain all the long-term data for my game. There are no game assets here; I store them on the host where I upload each game. This provides the easiest way to manage game versions.<\/p>\n\n<ul>\n<li>\n<strong>Media Store:<\/strong> Holds large static assets which aren\u2019t part of the game proper, such as trailers and marketing materials. I synchronize this on-demand with a local work computer. (Service: AWS S3)<\/li>\n<li>\n<strong>Log Store:<\/strong> Collects and stores the logs from the HTTP Frontend. I analyze these offline regularly. (Service: AWS S3)<\/li>\n<li>\n<strong>Database:<\/strong> This is the heart of my business data, storing purchase information and persisting long-term game state. (Service: Mongo)<\/li>\n<\/ul>\n\n<h1>\n  \n  \n  What\u2019s Missing\n<\/h1>\n\n<p>I\u2019ve left several components out of the diagram to focus on the core experience. I\u2019ll describe them briefly here.<\/p>\n\n<p>I don\u2019t show monitoring, partially because it\u2019s incomplete, but also because it\u2019s merely a line from every box to a monitoring agent. The structure doesn\u2019t change for monitoring, but it\u2019s part of the live environment.<\/p>\n\n<p>I\u2019ve left DNS out of the diagram for simplicity. I use multiple endpoints for the client, the web server and the CDN, as well as for email, which adds up to many DNS entries. In AWS one has Route 53, but the individual services can thankfully configure, and maintain most of their entries automatically.<\/p>\n\n<p>I have many offline scripts that access the database and the log store. This includes accounting scripts which calculate cross-currency payments and affiliate payouts \u2014 world sales with tax are a nightmare! I also do analysis of game records to help me design future games.<\/p>\n\n<p>There\u2019s an additional system used to manage the mailing list. As the sign-up form is part of the website, and people can follow links from the emails to the website, it is a legitimate part of the architecture.<\/p>\n\n<h1>\n  \n  \n  Layers upon layers\n<\/h1>\n\n<p>I\u2019m tempted to call this the hardware architecture, but with cloud services, everything is logical. It\u2019s a definite layer in my system. Can I call it the \u201c<a href=\"https:\/\/en.wikipedia.org\/wiki\/DevOps\" rel=\"noopener noreferrer\">DevOps<\/a> Layer\u201d?<\/p>\n\n<p>The website on top of this is fairly standard, but the game is not. I will come back and do some articles about how <a href=\"https:\/\/edaqa.link\/EdaqasRoom3\" rel=\"noopener noreferrer\">the game<\/a> functions. I can also show how the system architecture and game architecture work together.<\/p>\n\n<p>Other than a few game specific parts, the architecture is fairly standard for an internet application. I believe this is a good approach to what I needed.<\/p>\n\n","category":["webdev","architecture","gamedev","cloud"]},{"title":"Highly inefficient invisible animations (CSS\/Firefox\/Chrome\/React)","pubDate":"Sat, 16 Jan 2021 16:06:46 +0000","link":"https:\/\/dev.to\/mortoray\/highly-inefficient-invisible-animations-css-firefox-chrome-react-1g86","guid":"https:\/\/dev.to\/mortoray\/highly-inefficient-invisible-animations-css-firefox-chrome-react-1g86","description":"<p>The cursor in my text editor was lagging. It\u2019s quite unusual given my 8 cores machine with 32GB of RAM. While tracking down that issue, I discovered that <a href=\"https:\/\/edaqa.link\/Carnival-Dev\">my escape game<\/a> was consuming 20-30% of the CPU while idling. That\u2019s bad! It turns out it was invisible elements being rotated via CSS.<\/p>\n\n<p>It\u2019s a bit of a pain. This means we need to remove all those elements which fade-away, otherwise they pile up and create load. Here I\u2019ll show you my solution using React \u2014 the top-layers of my game are in React, that\u2019s why I used it. I\u2019m not suggesting you use React to solve this problem.  But if you have animated HTML elements, get rid of them if they aren\u2019t visible.<\/p>\n\n<h1>\n  \n  \n  The Problem\n<\/h1>\n\n<p>While loading scenes, I display an indicator in the top-right corner of the screen.<\/p>\n\n<p><iframe width=\"710\" height=\"399\" src=\"https:\/\/www.youtube.com\/embed\/9oW3J-NYY5Y\">\n<\/iframe>\n<\/p>\n\n<p>This fades in when loading starts and fades out when loading is done. I wanted to avoid an abrupt transition. I handled this with CSS classes to hide and show the element. My React code looks like this:<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight jsx\"><code>    <span class=\"p\">&lt;<\/span><span class=\"nc\">SVGElement<\/span> \n        <span class=\"na\">url<\/span><span class=\"p\">=<\/span><span class=\"si\">{<\/span><span class=\"nx\">url<\/span><span class=\"si\">}<\/span>\n        <span class=\"na\">className<\/span><span class=\"p\">=<\/span><span class=\"si\">{<\/span><span class=\"nx\">RB<\/span><span class=\"p\">.<\/span><span class=\"nx\">class_name<\/span><span class=\"p\">(<\/span><span class=\"dl\">\"<\/span><span class=\"s2\">load-marker<\/span><span class=\"dl\">\"<\/span><span class=\"p\">,<\/span> <span class=\"nx\">className<\/span><span class=\"p\">,<\/span> <span class=\"nx\">is_loading<\/span> <span class=\"o\">&amp;&amp;<\/span> <span class=\"dl\">'<\/span><span class=\"s1\">loading<\/span><span class=\"dl\">'<\/span><span class=\"p\">)<\/span><span class=\"si\">}<\/span>\n    <span class=\"p\">\/&gt;<\/span>\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<p><code>SVGElement<\/code> is my component to load SVG files and display them inline. An <code>img<\/code> tag will perform the same way for this setup. The key is the <code>is_loading &amp;&amp; \u2018loading\u2019<\/code> part of the <code>className<\/code> attribute. This adds the <code>loading<\/code> class name to the element while it\u2019s loading. When finished loading, I remove the class name.<\/p>\n\n<p>This is the CSS (SCSS):<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight scss\"><code><span class=\"nc\">.load-marker<\/span> <span class=\"p\">{<\/span>\n    <span class=\"k\">&amp;<\/span><span class=\"nd\">:not<\/span><span class=\"o\">(<\/span><span class=\"nc\">.loading<\/span><span class=\"o\">)<\/span> <span class=\"p\">{<\/span>\n        <span class=\"nl\">animation-name<\/span><span class=\"p\">:<\/span> <span class=\"n\">fade-out<\/span><span class=\"p\">;<\/span>\n        <span class=\"nl\">animation-fill-mode<\/span><span class=\"p\">:<\/span> <span class=\"n\">forwards<\/span><span class=\"p\">;<\/span>\n        <span class=\"nl\">animation-duration<\/span><span class=\"p\">:<\/span> <span class=\"m\">0<\/span><span class=\"mi\">.5s<\/span><span class=\"p\">;<\/span>\n        <span class=\"nl\">animation-timing-function<\/span><span class=\"p\">:<\/span> <span class=\"n\">ease-in-out<\/span><span class=\"p\">;<\/span>\n    <span class=\"p\">}<\/span>\n    <span class=\"k\">&amp;<\/span><span class=\"nc\">.loading<\/span> <span class=\"p\">{<\/span>\n        <span class=\"nl\">animation-fill-mode<\/span><span class=\"p\">:<\/span> <span class=\"n\">forwards<\/span><span class=\"p\">;<\/span>\n        <span class=\"nl\">animation-duration<\/span><span class=\"p\">:<\/span> <span class=\"m\">0<\/span><span class=\"mi\">.5s<\/span><span class=\"p\">;<\/span>\n        <span class=\"nl\">animation-timing-function<\/span><span class=\"p\">:<\/span> <span class=\"n\">ease-in-out<\/span><span class=\"p\">;<\/span>\n        <span class=\"nl\">animation-name<\/span><span class=\"p\">:<\/span> <span class=\"n\">fade-in<\/span><span class=\"p\">;<\/span>\n    <span class=\"p\">}<\/span>\n    <span class=\"k\">@keyframes<\/span> <span class=\"nt\">fade-out<\/span> <span class=\"p\">{<\/span>\n        <span class=\"nt\">from<\/span> <span class=\"p\">{<\/span>\n            <span class=\"nl\">opacity<\/span><span class=\"p\">:<\/span> <span class=\"m\">1<\/span><span class=\"p\">;<\/span>\n            <span class=\"nl\">visibility<\/span><span class=\"p\">:<\/span> <span class=\"nb\">visible<\/span><span class=\"p\">;<\/span>\n        <span class=\"p\">}<\/span>\n        <span class=\"nt\">to<\/span> <span class=\"p\">{<\/span>\n            <span class=\"nl\">opacity<\/span><span class=\"p\">:<\/span> <span class=\"m\">0<\/span><span class=\"p\">;<\/span>\n            <span class=\"nl\">visibility<\/span><span class=\"p\">:<\/span> <span class=\"nb\">collapse<\/span><span class=\"p\">;<\/span>\n        <span class=\"p\">}<\/span>\n    <span class=\"p\">}<\/span>\n    <span class=\"k\">@keyframes<\/span> <span class=\"nt\">fade-in<\/span> <span class=\"p\">{<\/span>\n        <span class=\"nt\">from<\/span> <span class=\"p\">{<\/span>\n            <span class=\"nl\">opacity<\/span><span class=\"p\">:<\/span> <span class=\"m\">0<\/span><span class=\"p\">;<\/span>\n            <span class=\"nl\">visibility<\/span><span class=\"p\">:<\/span> <span class=\"nb\">collapse<\/span><span class=\"p\">;<\/span>\n        <span class=\"p\">}<\/span>\n        <span class=\"nt\">to<\/span> <span class=\"p\">{<\/span>\n            <span class=\"nl\">opacity<\/span><span class=\"p\">:<\/span> <span class=\"m\">1<\/span><span class=\"p\">;<\/span>\n            <span class=\"nl\">visibility<\/span><span class=\"p\">:<\/span> <span class=\"nb\">visible<\/span><span class=\"p\">;<\/span>\n        <span class=\"p\">}<\/span>\n    <span class=\"p\">}<\/span>\n<span class=\"p\">}<\/span>\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<blockquote>\n<p>I have an urge to digress into a rant about CSS's animation system!  I've written animation and layout systems before, and argh, this is acid thrown in my eyes. Indeed, <a href=\"https:\/\/fuseopen.com\/\">that system<\/a> has a clear adding and removing animation support, making this whole setup trivial. But this is CSS, and, alas\u2026<\/p>\n<\/blockquote>\n\n<p>When an item loses the <code>.loading<\/code> class it will transition to a transparent state. The problem however came from some other CSS:<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight scss\"><code><span class=\"nc\">.loader<\/span> <span class=\"p\">{<\/span>\n    <span class=\"nt\">svg<\/span> <span class=\"p\">{<\/span>\n        <span class=\"nl\">animation<\/span><span class=\"p\">:<\/span> <span class=\"n\">rotation<\/span> <span class=\"m\">6s<\/span> <span class=\"n\">infinite<\/span> <span class=\"n\">linear<\/span><span class=\"p\">;<\/span>\n        <span class=\"nl\">overflow<\/span><span class=\"p\">:<\/span> <span class=\"nb\">visible<\/span><span class=\"p\">;<\/span>\n        <span class=\"nl\">position<\/span><span class=\"p\">:<\/span> <span class=\"nb\">absolute<\/span><span class=\"p\">;<\/span>\n        <span class=\"nl\">top<\/span><span class=\"p\">:<\/span> <span class=\"m\">20px<\/span><span class=\"p\">;<\/span>\n        <span class=\"nl\">right<\/span><span class=\"p\">:<\/span> <span class=\"m\">20px<\/span><span class=\"p\">;<\/span>\n        <span class=\"nl\">width<\/span><span class=\"p\">:<\/span> <span class=\"m\">70px<\/span><span class=\"p\">;<\/span>\n        <span class=\"nl\">height<\/span><span class=\"p\">:<\/span> <span class=\"m\">70px<\/span><span class=\"p\">;<\/span>\n    <span class=\"p\">}<\/span>\n    <span class=\"k\">@keyframes<\/span> <span class=\"nt\">rotation<\/span> <span class=\"p\">{<\/span>\n        <span class=\"nt\">from<\/span> <span class=\"p\">{<\/span>\n            <span class=\"nl\">transform<\/span><span class=\"p\">:<\/span> <span class=\"nf\">rotate<\/span><span class=\"p\">(<\/span><span class=\"m\">0deg<\/span><span class=\"p\">);<\/span>\n        <span class=\"p\">}<\/span>\n        <span class=\"nt\">to<\/span> <span class=\"p\">{<\/span>\n            <span class=\"nl\">transform<\/span><span class=\"p\">:<\/span> <span class=\"nf\">rotate<\/span><span class=\"p\">(<\/span><span class=\"m\">360deg<\/span><span class=\"p\">);<\/span>\n        <span class=\"p\">}<\/span>\n    <span class=\"p\">}<\/span>\n<span class=\"p\">}<\/span>\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<p>That <code>infinite<\/code> bit is the problem. It\u2019s irrelevant that we\u2019ve faded the opacity to 0, the animation is still running! Firefox still does a style and layout update, each frame. Why it ends up consuming so much CPU, I have no idea.  Chrome also consumed CPU, but only around 10%.  Note, 10% is still ridiculous for a static screen.<\/p>\n\n<p>I could also \u201csolve\u201d the problem by not spinning the item unless something is loading. This creates a rough transition where the icon abruptly stops rotating while fading away. Not good.<\/p>\n\n<h1>\n  \n  \n  The Solution\n<\/h1>\n\n<p>I have two animated indicators, the loader and a disconnected icon, for when you lose the WebSocket connection to the server. I abstracted a common base component to handle them the same. This is how I use it, for the loader:<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight jsx\"><code><span class=\"k\">export<\/span> <span class=\"kd\">function<\/span> <span class=\"nx\">Loader<\/span><span class=\"p\">({<\/span> <span class=\"nx\">is_loading<\/span> <span class=\"p\">})<\/span> <span class=\"p\">{<\/span>\n    <span class=\"k\">return<\/span> <span class=\"p\">&lt;<\/span><span class=\"nc\">HideLoader<\/span>\n        <span class=\"na\">url<\/span><span class=\"p\">=<\/span><span class=\"si\">{<\/span><span class=\"nx\">theme<\/span><span class=\"p\">.<\/span><span class=\"nx\">marker_loading<\/span><span class=\"si\">}<\/span>\n        <span class=\"na\">is_loading<\/span><span class=\"p\">=<\/span><span class=\"si\">{<\/span><span class=\"nx\">is_loading<\/span><span class=\"si\">}<\/span>\n        <span class=\"na\">className<\/span><span class=\"p\">=<\/span><span class=\"s\">\"loader\"<\/span>\n    <span class=\"p\">\/&gt;<\/span>\n<span class=\"p\">}<\/span>\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<p>This is the implementation:<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight jsx\"><code><span class=\"kd\">function<\/span> <span class=\"nx\">HideLoaderImpl<\/span><span class=\"p\">({<\/span> <span class=\"nx\">is_loading<\/span><span class=\"p\">,<\/span> <span class=\"nx\">url<\/span><span class=\"p\">,<\/span> <span class=\"nx\">className<\/span> <span class=\"p\">})<\/span> <span class=\"p\">{<\/span>\n    <span class=\"kd\">const<\/span> <span class=\"p\">[<\/span> <span class=\"nx\">timer_id<\/span><span class=\"p\">,<\/span> <span class=\"nx\">set_timer_id<\/span> <span class=\"p\">]<\/span> <span class=\"o\">=<\/span> <span class=\"nx\">React<\/span><span class=\"p\">.<\/span><span class=\"nx\">useState<\/span><span class=\"p\">(<\/span><span class=\"mi\">0<\/span><span class=\"p\">)<\/span>\n\n    <span class=\"nx\">React<\/span><span class=\"p\">.<\/span><span class=\"nx\">useEffect<\/span><span class=\"p\">(()<\/span> <span class=\"o\">=&gt;<\/span> <span class=\"p\">{<\/span>\n        <span class=\"k\">if<\/span><span class=\"p\">(<\/span> <span class=\"o\">!<\/span><span class=\"nx\">is_loading<\/span> <span class=\"o\">&amp;&amp;<\/span> <span class=\"o\">!<\/span><span class=\"nx\">timer_id<\/span> <span class=\"p\">)<\/span> <span class=\"p\">{<\/span>\n            <span class=\"kd\">const<\/span> <span class=\"nx\">css_duration<\/span> <span class=\"o\">=<\/span> <span class=\"mi\">1000<\/span>\n            <span class=\"kd\">const<\/span> <span class=\"nx\">new_timer_id<\/span> <span class=\"o\">=<\/span> <span class=\"nx\">setTimeout<\/span><span class=\"p\">(<\/span> <span class=\"p\">()<\/span> <span class=\"o\">=&gt;<\/span> <span class=\"nx\">set_timer_id<\/span><span class=\"p\">(<\/span><span class=\"mi\">0<\/span><span class=\"p\">),<\/span> <span class=\"nx\">css_duration<\/span> <span class=\"p\">)<\/span>\n            <span class=\"nx\">set_timer_id<\/span><span class=\"p\">(<\/span><span class=\"nx\">new_timer_id<\/span><span class=\"p\">)<\/span>\n        <span class=\"p\">}<\/span>\n    <span class=\"p\">},<\/span> <span class=\"p\">[<\/span><span class=\"nx\">is_loading<\/span><span class=\"p\">])<\/span> <span class=\"c1\">\/\/ only trigger on an is_loading change<\/span>\n\n    <span class=\"kd\">const<\/span> <span class=\"nx\">visible<\/span> <span class=\"o\">=<\/span> <span class=\"nx\">is_loading<\/span> <span class=\"o\">||<\/span> <span class=\"nx\">timer_id<\/span>\n    <span class=\"k\">if<\/span><span class=\"p\">(<\/span><span class=\"o\">!<\/span><span class=\"nx\">visible<\/span><span class=\"p\">)<\/span> <span class=\"p\">{<\/span>\n        <span class=\"k\">return<\/span> <span class=\"kc\">null<\/span>\n    <span class=\"p\">}<\/span>\n\n    <span class=\"k\">return<\/span> <span class=\"p\">(<\/span>\n        <span class=\"p\">&lt;<\/span><span class=\"nc\">SVGElement<\/span> \n            <span class=\"na\">url<\/span><span class=\"p\">=<\/span><span class=\"si\">{<\/span><span class=\"nx\">url<\/span><span class=\"si\">}<\/span>\n            <span class=\"na\">className<\/span><span class=\"p\">=<\/span><span class=\"si\">{<\/span><span class=\"nx\">RB<\/span><span class=\"p\">.<\/span><span class=\"nx\">class_name<\/span><span class=\"p\">(<\/span><span class=\"dl\">\"<\/span><span class=\"s2\">load-marker<\/span><span class=\"dl\">\"<\/span><span class=\"p\">,<\/span> <span class=\"nx\">className<\/span><span class=\"p\">,<\/span> <span class=\"nx\">is_loading<\/span> <span class=\"o\">&amp;&amp;<\/span> <span class=\"dl\">'<\/span><span class=\"s1\">loading<\/span><span class=\"dl\">'<\/span><span class=\"p\">)<\/span><span class=\"si\">}<\/span>\n        <span class=\"p\">\/&gt;<\/span>\n    <span class=\"p\">)<\/span>\n<span class=\"p\">}<\/span>\n\n<span class=\"kd\">const<\/span> <span class=\"nx\">HideLoader<\/span> <span class=\"o\">=<\/span> <span class=\"nx\">React<\/span><span class=\"p\">.<\/span><span class=\"nx\">memo<\/span><span class=\"p\">(<\/span><span class=\"nx\">HideLoaderImpl<\/span><span class=\"p\">)<\/span>\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<p>At first glance, it\u2019s not obvious how this achieves a delayed removal of the element. The HTML generation is clear, when <code>visible<\/code> is false, then display nothing. When true, display the element as before, with the same logic for setting the <code>loading<\/code> class name.<\/p>\n\n<p>If <code>is_loading<\/code> is true, then <code>visible<\/code> will be true. This is the simple case. But there is the other true condition when we have a <code>timer_id<\/code>. <\/p>\n\n<p>The <code>setTimeout<\/code> callback does nothing but clear the <code>timer_id<\/code> when it\u2019s done. At first I suspected I\u2019d have to track another variable, setting at the start and end of the timeout. It turns out that all I need to know is whether there is a timeout at all. So long as I have a timer, I know that I shouldn\u2019t remove the element.<\/p>\n\n<p>The condition list to <a href=\"https:\/\/reactjs.org\/docs\/hooks-reference.html#conditionally-firing-an-effect\"><code>React.useEffect<\/code><\/a> is important here. I provide only <code>is_loading<\/code> \u2014 I only wish for the effect to run if the value of <code>is_loading<\/code> has changed. Some style guides will insist that I include <code>timer_id<\/code> (and <code>set_timer_id<\/code>) as well in the list. That approach defines the second argument to <code>useEffect<\/code> as a dependency list, but this is incorrect. It\u2019s actually a list of values, which if changed, will trigger the effect to run again.  The React documents are clear about this. Yet also say it\u2019s a dependency list, and recommend a lint plugin that would complain about my code. That recommendation makes sense for <code>useCallback<\/code> and <code>useMemo<\/code>, but not for <code>useEffect<\/code>.<\/p>\n\n<p>Adding <code>timer_id<\/code> to the list would be wrong. When the timer finishes, it sets the <code>timer_id<\/code> to 0. That change would cause the effect to trigger again. This is a case where we do \u201cdepend\u201d on the <code>timer_id<\/code> value, but we shouldn\u2019t re-execute when it changes, as that would end up creating create a new timer.<\/p>\n\n<p>In any case, this simple code now does what I want. It defers the DOM removal of the element until after the end of the animation. Well, it defers it one second, which is long enough to cover the 0.5s CSS animation. It\u2019s complicated to keep these times in sync \u2014 <em>more fist shaking at the CSS animation system!<\/em><\/p>\n\n<blockquote>\n<p>If you\u2019ve got an eye for defects, there is one there. The loader icon can be removed too early: when <code>is_loading<\/code> becomes true, then false, then within one second becomes true and false again. I don\u2019t create a new timer if one already exists, so the deferral time will still be from the first timer. In practice, this will not likely happen, and the impact is minimal. The fix is to cancel an existing timeout and always create a new one.<\/p>\n<\/blockquote>\n\n<h1>\n  \n  \n  My lagging cursor\n<\/h1>\n\n<p>I never got an obvious answer why my cursor was lagging. There were all sorts of applications, idle applications, consuming 5-10% CPU.  It\u2019s perhaps a real cost of high-level languages. More on that another day. I still hope that future apps will strive for less energy use.<\/p>\n\n<p>For now, remove all those invisible animated HTML elements.<\/p>\n\n","category":["webdev","javascript","react","css"]},{"title":"High-Throughput Game Message Server with Python websockets","pubDate":"Sat, 05 Dec 2020 22:14:02 +0000","link":"https:\/\/dev.to\/mortoray\/high-throughput-game-message-server-with-python-websockets-3mnf","guid":"https:\/\/dev.to\/mortoray\/high-throughput-game-message-server-with-python-websockets-3mnf","description":"<p>An error came up during a competition with <a href=\"https:\/\/edaqasroom.com\/game\/carnival\">my game<\/a>. One of the 80 players got stuck. Like really stuck: a breaking defect! The error should not have happened, could not have happened, yet it did. The only possibility was the Web Socket stack I use. Turns out that layer didn\u2019t work as intended, and there was no way to fix it. Alas, I sought an alternate solution. <\/p>\n\n<p>Here I describe what I came up with: a new, more focused stack, using Python websockets. Lots of coroutines, asyncio, and queues. I have the complete working example at the end of this article. This is likely the last state where it can standalone; I\u2019ll be changing it to fit even tighter with my game code.<\/p>\n\n<h1>\n  \n  \n  Logical Structure and the Problem\n<\/h1>\n\n<p><a href=\"https:\/\/edaqasroom.com\/game\/carnival\">My game<\/a> is a multiplayer puzzle game coordinated by messages. I segregate each instance of the game from the others. For messages, I do this with the classic \u201crooms\u201d concept. It\u2019s the name that Flask-SocketIO uses as well, and that\u2019s what my first implementation used.<\/p>\n\n<p>Mostly, messages in the game don\u2019t need a defined total order. They can arrive in a different order to the different clients. There are a few situations where this isn\u2019t true however, places where I need some messages to have a defined total order. There aren\u2019t many of them, and that\u2019s likely why I didn\u2019t notice the defect earlier.<\/p>\n\n<p>When I first started the project I asked whether the library, if used on a single client system, echoing in order, could maintain an order to the clients. The answer was yes, but the truth is no. Given the load is mostly network bound, it usually maintains an order. Only in a few stress times, or because of dumb luck, it sends messages out-of-order.<\/p>\n\n<p>Fine, I can probably put a queue around it. Ugh, the throughput drops to an abysmal 70msgs\/s. Without the queue it was already a slow 1200msg\/s, but that was enough for my game. After a bit of back-and-forth, me and the library author disagree on what is acceptable throughput.<\/p>\n\n<p>So I grabbed the <a href=\"https:\/\/github.com\/aaugustin\/websockets\">websockets<\/a> library instead, whipped together a proof of concept, and got 12,000msgs \/s. Yeah, that\u2019s more like I\u2019d expect.<\/p>\n\n<blockquote>\n<p>Actually, I\u2019d expect even more. And long term, if I get enough traffic, I\u2019ll rewrite this in C++. The throughput should be entirely network bound, but it\u2019s still CPU bound on the server. I\u2019ve done a lot of low-level networking before to know I can push it higher, but for my needs now, 12K\/s is way more than enough. I\u2019d likely scale the number of servers before worrying about optimizing one of them.<\/p>\n<\/blockquote>\n\n<p>On to the code!<\/p>\n\n<h1>\n  \n  \n  A Python websockets Messaging server\n<\/h1>\n\n<p>The \u201cwebsockets\u201d module is a minimal implementation of <a href=\"https:\/\/en.wikipedia.org\/wiki\/WebSocket\">WebSockets<\/a>. That sounded like what I wanted. I didn\u2019t want to go to the low-level of handling the protocol. This left me writing all my high-level logic, in particular the client rooms.<\/p>\n\n<p>The library is easy to use. I got a basic example working with little effort. Of course, then there are lots of details to cover. Here\u2019s a quick list of things I needed to support, features first:<\/p>\n\n<p>-Handle an incoming client where \u201chandle\u201d is mainly done by the library<br>\n-Allow a client to join a game room (each client can only join one room in my system, simplifying the code)<br>\n-Allow another client to join the same game room<br>\n-Allow other clients to join other rooms<br>\n-Allow a client to send a message<br>\n-Provide a total order to the message, with a message id<br>\n-Dispatch that message to all clients in the room<\/p>\n\n<blockquote>\n<p>In my final code I\u2019ll persist the messages to a Redis store, then eventually to a MongoDB. That is not part of my example code.<\/p>\n<\/blockquote>\n\n<p>And there are several situations, or errors, that I\u2019d have to deal with.<\/p>\n\n<p>-A client disconnects cleanly or abruptly<br>\n-The client sends crap<br>\n-The client is slow<br>\n-Cleanup a room if there are no more clients in it<\/p>\n<h1>\n  \n  \n  Structure\n<\/h1>\n\n<p>My server maintains a list of clients in a list of rooms:<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight python\"><code><span class=\"o\">@<\/span><span class=\"n\">dataclass<\/span>\n<span class=\"k\">class<\/span> <span class=\"nc\">Client<\/span><span class=\"p\">:<\/span>\n    <span class=\"n\">socket<\/span><span class=\"p\">:<\/span> <span class=\"n\">Any<\/span> <span class=\"c1\"># What type?\n<\/span>    <span class=\"nb\">id<\/span><span class=\"p\">:<\/span> <span class=\"nb\">int<\/span>\n    <span class=\"n\">disconnected<\/span><span class=\"p\">:<\/span> <span class=\"nb\">bool<\/span> <span class=\"o\">=<\/span> <span class=\"bp\">False<\/span>\n\n<span class=\"o\">@<\/span><span class=\"n\">dataclass<\/span>\n<span class=\"k\">class<\/span> <span class=\"nc\">Room<\/span><span class=\"p\">:<\/span>\n    <span class=\"n\">key<\/span><span class=\"p\">:<\/span> <span class=\"nb\">str<\/span>\n    <span class=\"n\">clients<\/span><span class=\"p\">:<\/span> <span class=\"n\">Dict<\/span><span class=\"p\">[<\/span><span class=\"nb\">int<\/span><span class=\"p\">,<\/span><span class=\"n\">Client<\/span><span class=\"p\">]<\/span> <span class=\"o\">=<\/span> <span class=\"n\">field<\/span><span class=\"p\">(<\/span><span class=\"n\">default_factory<\/span><span class=\"o\">=<\/span><span class=\"nb\">dict<\/span><span class=\"p\">)<\/span>\n    <span class=\"n\">new_clients<\/span><span class=\"p\">:<\/span> <span class=\"n\">List<\/span><span class=\"p\">[<\/span><span class=\"n\">Client<\/span><span class=\"p\">]<\/span> <span class=\"o\">=<\/span> <span class=\"n\">field<\/span><span class=\"p\">(<\/span><span class=\"n\">default_factory<\/span><span class=\"o\">=<\/span><span class=\"nb\">list<\/span><span class=\"p\">)<\/span>\n    <span class=\"n\">msg_id<\/span><span class=\"p\">:<\/span> <span class=\"nb\">int<\/span> <span class=\"o\">=<\/span> <span class=\"mi\">0<\/span>\n    <span class=\"n\">event_queue<\/span><span class=\"p\">:<\/span> <span class=\"n\">asyncio<\/span><span class=\"p\">.<\/span><span class=\"n\">Queue<\/span> <span class=\"o\">=<\/span> <span class=\"n\">field<\/span><span class=\"p\">(<\/span><span class=\"n\">default_factory<\/span><span class=\"o\">=<\/span><span class=\"n\">asyncio<\/span><span class=\"p\">.<\/span><span class=\"n\">Queue<\/span><span class=\"p\">)<\/span>\n    <span class=\"n\">listening<\/span><span class=\"p\">:<\/span> <span class=\"nb\">bool<\/span> <span class=\"o\">=<\/span> <span class=\"bp\">False<\/span>\n    <span class=\"n\">future<\/span><span class=\"p\">:<\/span> <span class=\"n\">Any<\/span> <span class=\"o\">=<\/span> <span class=\"bp\">None<\/span> <span class=\"c1\"># What Type?\n<\/span><\/code><\/pre>\n\n<\/div>\n\n\n\n<p>I use type annotations for my Python, along with MyPy to check the types. Alas, for several library classes I\u2019m unsure of types.  Since many of them are created automatically, or are returned from other functions, it\u2019s difficult to determine the type. I will eventually find out all the types.<\/p>\n\n<p>In these data types, the <code>socket<\/code> is the only part directly connected to the \u201cwebsockets\u201d module. It tracks the incoming connection and used to send and receive data.<\/p>\n\n<p>In brief, the <code>listen_room<\/code> function handles the incoming client connections. I push all messages onto the <code>event_queue<\/code> of the <code>Room<\/code>. The <code>listen_room<\/code> function listens to this queue and sends messages to all clients in the room.<\/p>\n\n<h2>\n  \n  \n  One listener per room\n<\/h2>\n\n<p>I initially had a single listening queue that handled all the rooms. When I eventually write the lower-level server, like in C++, I\u2019d keep this structure. When you get low-enough level, you can control a lot more details, removing the need for coroutines entirely.<\/p>\n\n<p>But in Python there are a few reasons I\u2019m using one listener per room:<\/p>\n\n<p>-<strong>not<\/strong> overhead<br>\n-Redis<br>\n-bad clients<\/p>\n\n<p>Overhead is all my Python code, and the library code, surrounding the writing to the clients. It\u2019s not a lot, but it can add up with a lot of activity. I suspect the JSON parsing and formatting is the biggest part of it. But this is not a reason I have one listener per room. Since the Python code is running as a single real thread, it is irrelevant whether this code happens in one listener, or many listeners. It\u2019s all unavoidable computational load.<\/p>\n\n<p>The first real reason, Redis, is the well-behaved motivator. For each outgoing message I have to create a unique message id. In the sample code I track this in Python, in the <code>Room<\/code> class. On my final server, I\u2019ll track this in a Redis integer key. Additionally, I\u2019ll store all messages in a Redis list.  A separate process will clear this regularly and persist the messages to a MongoDB.   The calls to Redis take time, time that the server could instead process messages for other rooms. Thus I want to segregate the rooms. While one room waits on Redis, the others can continue processing.<\/p>\n\n<p>The second reason, bad clients, is an unfortunate need. It\u2019s possible that a client gets disconnected, or fails to process messages quickly enough. For the most part, this is handled by buffers. The calls to <code>socket.send<\/code> are effectively asynchronous, at least until the queue fills up. When that happens, <code>send<\/code> will wait until there is space in the queue. While waiting all the other rooms will stall, being unable to send any messages. By having one queue per room, I limit the damage of a client to that room only.<\/p>\n\n<p>This won\u2019t likely happen. First off, the websockets library has a timeout feature. Unresponsive clients will be disconnected long-before the outgoing socket buffers get filled up. My game simply doesn\u2019t generate enough messages to ever fill the buffers. Extrapolating from my stress test, with an estimated average message size, there is room for 25K game messages in the standard buffers. And a typical run-through of my game, with a team, generates only 3 to 4 thousand messages. <\/p>\n\n<p>In any case, it\u2019s good protection to have.<\/p>\n<h2>\n  \n  \n  <code>clients<\/code>, <code>new_clients<\/code> and memory\n<\/h2>\n\n<p>One advantage of having a single real thread is not needing to worry about actual <a href=\"https:\/\/en.wikipedia.org\/wiki\/Race_condition#Data_race\">data races<\/a>. They simply don\u2019t happen as they would in a multi-threaded application. Yay! No memory corruption is possible. <\/p>\n\n<p>It doesn\u2019t mean that race conditions, something different,  don\u2019t happen. The logical concerns of concurrency still exist, though to a lesser degree. Thanks cooperative threading! The most significant concern in my code is with the <code>clients<\/code> object. The queue listener iterates over the clients. If the list is modified during the iteration, Python will throw a concurrent modification exception. That is a strict no-no, as the iterator has no idea what it should do.<\/p>\n\n<p>There are three cases where the list needs to be modified:<\/p>\n\n<p>-when a client disconnects in <code>listen_socket<\/code><br>\n-when a client disconnects in <code>listen_room<\/code><br>\n-when a new client joins the room<\/p>\n\n<p>At first I handled disconnect in the <code>listen_socket<\/code> function, but through testing noticed it can be a <code>socket.send()<\/code> call that detects the disconnect first. Thus the disconnect happens in multiple places. In both cases, I merely mark the client as <code>disconnected<\/code> in the <code>Client<\/code> structure. The <code>listen_room<\/code> skips disconnected clients while sending messages. It\u2019ll track them and safely remove them from the room after the iteration loop. <\/p>\n\n<p>When a new client joins the room, <code>listen_socket<\/code> adds it to the <code>new_clients<\/code> list. <code>listen_room<\/code> will then add new clients prior to each message loop. It does this just after retrieving a message to ensure that all new clients get the message. This means that the room messages can arrive at a client prior to the \u201cjoined\u201d response from joining the room. In my game, getting this ordering, along with sending old messages, is important for clients getting a consistent game state. I\u2019ll likely have to adjust this code a bit.<\/p>\n\n<p>At no point does <code>listen_socket<\/code> know if it\u2019s safe to work with <code>clients<\/code>, since it can\u2019t tell if <code>listen_room<\/code> is inside or outside of the loop. A lock isn\u2019t a bad idea, but it introduces an avoidable delay on the incoming listening side, and delays in the room listener. Why lock when I don\u2019t have to?<\/p>\n\n<p>In retrospect, it might be a disadvantage that much of the coroutine parallelism is implicit, especially if using something like eventlets. As a programmer, it\u2019s less apparent where the logical thread switching happens. You just have to know that every await operation, every asyncio call, and every websocket call is a potential location for a thread switch. It\u2019d be nice to say you should assume a switch is possible at any time, but then I couldn\u2019t rely on it not switching in some places and would require a bunch of locks.<\/p>\n\n<p>Use <a href=\"https:\/\/mortoray.com\/2019\/02\/20\/how-does-a-mutex-work-what-does-it-cost\/\">locks<\/a> if you aren\u2019t sure. Performance is irrelevant if your game breaks. <strong>grumble<\/strong><\/p>\n<h2>\n  \n  \n  Stats and throughput\n<\/h2>\n\n<p>I added a simple <code>Stats<\/code> class to track throughput on the server. It emits timings for all the incoming and outgoing messages per room. The 12K\/s is what happens if I have multiple clients connected to unique rooms. My machine hits that 12K limit with the server process pegged at 100% CPU use.<\/p>\n\n<blockquote>\n<p>Unfortunately, I must adjust my number down to 10K. Once I moved the rooms to individual listeners, I hit far more overhead. I\u2019m not entirely sure why \u2014 I can\u2019t imagine it\u2019s the extra number of coroutines. Likely there are some tweaks in the async stuff to improve it, but it\u2019s still fast enough that I\u2019m not concerned.<\/p>\n<\/blockquote>\n\n<p>As a curiosity, I measured a single client connected to the server. It\u2019s getting slightly over 5Kmsgs\/s. Since this is a client and server, I have two processes. They are both at 58% CPU use. Ideally they should be at 50% CPU use, since they send a message back and forth. That extra 8% is processing spent doing stuff other than handling the message.  Perhaps if I wrote the system in C++ it\u2019d get closer to 50%, but never reach there completely. The throughput however should go up.<\/p>\n\n<blockquote>\n<p>When I say C++ would be faster, it\u2019s because of my experience. I have better control of what happens and know how to use that control. It\u2019s easy to get it wrong and up with a steaming pile that is worse than the clean Python version. Server code is hard!<\/p>\n<\/blockquote>\n\n<p>The stats don\u2019t directly measure response time. But knowing the ping pong nature of the tests, I can calculate roughly what that\u2019d be. At fractional milliseconds per message, it\u2019ll be noise compared to the true network overhead when deployed.<\/p>\n\n<p>This statistics I calculate here aren\u2019t great. If I were trying to write a truly high-performance server, I\u2019d track averages, standard deviations, extremes, and record it all better. The numbers it\u2019s showing are so over-powered for my game though that there\u2019s no need for more.<\/p>\n<h1>\n  \n  \n  Next steps\n<\/h1>\n\n<p>Now I need to get this integrated with my existing server. I think I lose the ability to use a single port and single Python instance. That\u2019s not a big loss, it\u2019s something I was intending on doing at some point, anyway. The game server shouldn\u2019t be the same as the web server. This is both for performance and stability.  Eventually, should I have enough load, I must run multiple game servers (multiple servers handling web socket connections).  I have a plan to scale that direction, but it\u2019ll be a long time before I get there.<\/p>\n<h1>\n  \n  \n  Code\n<\/h1>\n\n<p>Below is the code for the server, the Python client, and a sample client for the browser. While the details may change slightly, the structure will stay close to this. Given the size of the code, it\u2019s better not to spin this into a library. I\u2019ll directly it adapt it for <a href=\"https:\/\/edaqasroom.com\/game\/carnival\">my game<\/a> server.<\/p>\n\n<p>This code is fairly stable and should work as a starting point for your own needs. Though of course, I can\u2019t make any promises of that. I\u2019ll likely discover things in my application that require fixes.<\/p>\n<h2>\n  \n  \n  ws_server.py\n<\/h2>\n\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight python\"><code><span class=\"kn\">from<\/span> <span class=\"nn\">typing<\/span> <span class=\"kn\">import<\/span> <span class=\"o\">*<\/span>\n<span class=\"kn\">from<\/span> <span class=\"nn\">dataclasses<\/span> <span class=\"kn\">import<\/span> <span class=\"n\">dataclass<\/span><span class=\"p\">,<\/span> <span class=\"n\">field<\/span>\n<span class=\"kn\">import<\/span> <span class=\"nn\">asyncio<\/span><span class=\"p\">,<\/span> <span class=\"n\">websockets<\/span><span class=\"p\">,<\/span> <span class=\"n\">json<\/span><span class=\"p\">,<\/span> <span class=\"n\">time<\/span>\n<span class=\"kn\">from<\/span> <span class=\"nn\">collections<\/span> <span class=\"kn\">import<\/span> <span class=\"n\">defaultdict<\/span>\n\n<span class=\"c1\">#sys.path.append('..\/server')\n#from escape.live_game_state import LiveGameState\n<\/span>\n<span class=\"k\">def<\/span> <span class=\"nf\">encode_msg<\/span><span class=\"p\">(<\/span><span class=\"n\">msg<\/span><span class=\"p\">:<\/span> <span class=\"n\">Dict<\/span><span class=\"p\">)<\/span> <span class=\"o\">-&gt;<\/span> <span class=\"nb\">str<\/span><span class=\"p\">:<\/span>\n    <span class=\"k\">return<\/span> <span class=\"n\">json<\/span><span class=\"p\">.<\/span><span class=\"n\">dumps<\/span><span class=\"p\">(<\/span><span class=\"n\">msg<\/span><span class=\"p\">,<\/span> <span class=\"n\">ensure_ascii<\/span><span class=\"o\">=<\/span><span class=\"bp\">False<\/span><span class=\"p\">)<\/span>\n\n<span class=\"k\">def<\/span> <span class=\"nf\">decode_msg<\/span><span class=\"p\">(<\/span><span class=\"n\">text<\/span><span class=\"p\">:<\/span> <span class=\"nb\">str<\/span><span class=\"p\">)<\/span> <span class=\"o\">-&gt;<\/span> <span class=\"n\">Dict<\/span><span class=\"p\">:<\/span>\n    <span class=\"k\">return<\/span> <span class=\"n\">json<\/span><span class=\"p\">.<\/span><span class=\"n\">loads<\/span><span class=\"p\">(<\/span><span class=\"n\">text<\/span><span class=\"p\">)<\/span>\n\n\n<span class=\"o\">@<\/span><span class=\"n\">dataclass<\/span>\n<span class=\"k\">class<\/span> <span class=\"nc\">Client<\/span><span class=\"p\">:<\/span>\n    <span class=\"n\">socket<\/span><span class=\"p\">:<\/span> <span class=\"n\">Any<\/span> <span class=\"c1\"># What type?\n<\/span>    <span class=\"nb\">id<\/span><span class=\"p\">:<\/span> <span class=\"nb\">int<\/span>\n    <span class=\"n\">disconnected<\/span><span class=\"p\">:<\/span> <span class=\"nb\">bool<\/span> <span class=\"o\">=<\/span> <span class=\"bp\">False<\/span>\n\n<span class=\"o\">@<\/span><span class=\"n\">dataclass<\/span>\n<span class=\"k\">class<\/span> <span class=\"nc\">Room<\/span><span class=\"p\">:<\/span>\n    <span class=\"n\">key<\/span><span class=\"p\">:<\/span> <span class=\"nb\">str<\/span>\n    <span class=\"n\">clients<\/span><span class=\"p\">:<\/span> <span class=\"n\">Dict<\/span><span class=\"p\">[<\/span><span class=\"nb\">int<\/span><span class=\"p\">,<\/span><span class=\"n\">Client<\/span><span class=\"p\">]<\/span> <span class=\"o\">=<\/span> <span class=\"n\">field<\/span><span class=\"p\">(<\/span><span class=\"n\">default_factory<\/span><span class=\"o\">=<\/span><span class=\"nb\">dict<\/span><span class=\"p\">)<\/span>\n    <span class=\"n\">new_clients<\/span><span class=\"p\">:<\/span> <span class=\"n\">List<\/span><span class=\"p\">[<\/span><span class=\"n\">Client<\/span><span class=\"p\">]<\/span> <span class=\"o\">=<\/span> <span class=\"n\">field<\/span><span class=\"p\">(<\/span><span class=\"n\">default_factory<\/span><span class=\"o\">=<\/span><span class=\"nb\">list<\/span><span class=\"p\">)<\/span>\n    <span class=\"n\">msg_id<\/span><span class=\"p\">:<\/span> <span class=\"nb\">int<\/span> <span class=\"o\">=<\/span> <span class=\"mi\">0<\/span>\n    <span class=\"n\">event_queue<\/span><span class=\"p\">:<\/span> <span class=\"n\">asyncio<\/span><span class=\"p\">.<\/span><span class=\"n\">Queue<\/span> <span class=\"o\">=<\/span> <span class=\"n\">field<\/span><span class=\"p\">(<\/span><span class=\"n\">default_factory<\/span><span class=\"o\">=<\/span><span class=\"n\">asyncio<\/span><span class=\"p\">.<\/span><span class=\"n\">Queue<\/span><span class=\"p\">)<\/span>\n    <span class=\"n\">listening<\/span><span class=\"p\">:<\/span> <span class=\"nb\">bool<\/span> <span class=\"o\">=<\/span> <span class=\"bp\">False<\/span>\n    <span class=\"n\">future<\/span><span class=\"p\">:<\/span> <span class=\"n\">Any<\/span> <span class=\"o\">=<\/span> <span class=\"bp\">None<\/span> <span class=\"c1\"># What Type?\n<\/span>\n    <span class=\"k\">def<\/span> <span class=\"nf\">client_count<\/span><span class=\"p\">(<\/span><span class=\"bp\">self<\/span><span class=\"p\">)<\/span> <span class=\"o\">-&gt;<\/span> <span class=\"nb\">int<\/span><span class=\"p\">:<\/span>\n        <span class=\"k\">return<\/span> <span class=\"nb\">len<\/span><span class=\"p\">([<\/span><span class=\"n\">c<\/span><span class=\"p\">.<\/span><span class=\"nb\">id<\/span> <span class=\"k\">for<\/span> <span class=\"n\">c<\/span> <span class=\"ow\">in<\/span> <span class=\"bp\">self<\/span><span class=\"p\">.<\/span><span class=\"n\">clients<\/span><span class=\"p\">.<\/span><span class=\"n\">values<\/span><span class=\"p\">()<\/span> <span class=\"k\">if<\/span> <span class=\"ow\">not<\/span> <span class=\"n\">c<\/span><span class=\"p\">.<\/span><span class=\"n\">disconnected<\/span><span class=\"p\">])<\/span>\n\n<span class=\"n\">client_id_count<\/span> <span class=\"o\">=<\/span> <span class=\"mi\">0<\/span>\n\n<span class=\"n\">rooms<\/span><span class=\"p\">:<\/span> <span class=\"n\">Dict<\/span><span class=\"p\">[<\/span><span class=\"nb\">str<\/span><span class=\"p\">,<\/span> <span class=\"n\">Room<\/span><span class=\"p\">]<\/span> <span class=\"o\">=<\/span> <span class=\"p\">{}<\/span>\n\n\n<span class=\"c1\"># Used to get a basic idea of throughput\n<\/span><span class=\"k\">class<\/span> <span class=\"nc\">Stats<\/span><span class=\"p\">:<\/span>\n    <span class=\"k\">def<\/span> <span class=\"nf\">__init__<\/span><span class=\"p\">(<\/span><span class=\"bp\">self<\/span><span class=\"p\">,<\/span> <span class=\"n\">name<\/span><span class=\"p\">):<\/span>\n        <span class=\"bp\">self<\/span><span class=\"p\">.<\/span><span class=\"n\">_name<\/span> <span class=\"o\">=<\/span> <span class=\"n\">name<\/span>\n        <span class=\"bp\">self<\/span><span class=\"p\">.<\/span><span class=\"n\">_count<\/span> <span class=\"o\">=<\/span> <span class=\"mi\">0<\/span>\n        <span class=\"bp\">self<\/span><span class=\"p\">.<\/span><span class=\"n\">_time<\/span> <span class=\"o\">=<\/span> <span class=\"n\">time<\/span><span class=\"p\">.<\/span><span class=\"n\">monotonic<\/span><span class=\"p\">()<\/span>\n\n    <span class=\"k\">def<\/span> <span class=\"nf\">incr<\/span><span class=\"p\">(<\/span><span class=\"bp\">self<\/span><span class=\"p\">,<\/span> <span class=\"n\">amount<\/span> <span class=\"o\">=<\/span> <span class=\"mi\">1<\/span><span class=\"p\">):<\/span>\n        <span class=\"bp\">self<\/span><span class=\"p\">.<\/span><span class=\"n\">_count<\/span> <span class=\"o\">+=<\/span> <span class=\"n\">amount<\/span>\n        <span class=\"k\">if<\/span> <span class=\"bp\">self<\/span><span class=\"p\">.<\/span><span class=\"n\">_count<\/span> <span class=\"o\">&gt;<\/span> <span class=\"mi\">5000<\/span><span class=\"p\">:<\/span>\n            <span class=\"n\">end_time<\/span> <span class=\"o\">=<\/span> <span class=\"n\">time<\/span><span class=\"p\">.<\/span><span class=\"n\">monotonic<\/span><span class=\"p\">()<\/span>\n            <span class=\"k\">print<\/span><span class=\"p\">(<\/span> <span class=\"sa\">f<\/span><span class=\"s\">'<\/span><span class=\"si\">{<\/span><span class=\"bp\">self<\/span><span class=\"p\">.<\/span><span class=\"n\">_name<\/span><span class=\"si\">}<\/span><span class=\"s\"> <\/span><span class=\"si\">{<\/span><span class=\"bp\">self<\/span><span class=\"p\">.<\/span><span class=\"n\">_count<\/span> <span class=\"o\">\/<\/span> <span class=\"p\">(<\/span><span class=\"n\">end_time<\/span><span class=\"o\">-<\/span><span class=\"bp\">self<\/span><span class=\"p\">.<\/span><span class=\"n\">_time<\/span><span class=\"p\">)<\/span><span class=\"si\">}<\/span><span class=\"s\">\/s'<\/span> <span class=\"p\">)<\/span>\n            <span class=\"bp\">self<\/span><span class=\"p\">.<\/span><span class=\"n\">_count<\/span> <span class=\"o\">=<\/span> <span class=\"mi\">0<\/span>\n            <span class=\"bp\">self<\/span><span class=\"p\">.<\/span><span class=\"n\">_time<\/span> <span class=\"o\">=<\/span> <span class=\"n\">end_time<\/span>\n\n\n<span class=\"k\">async<\/span> <span class=\"k\">def<\/span> <span class=\"nf\">listen_room<\/span><span class=\"p\">(<\/span><span class=\"n\">room<\/span><span class=\"p\">):<\/span>\n    <span class=\"k\">if<\/span> <span class=\"n\">room<\/span><span class=\"p\">.<\/span><span class=\"n\">listening<\/span><span class=\"p\">:<\/span>\n        <span class=\"k\">raise<\/span> <span class=\"nb\">Exception<\/span><span class=\"p\">(<\/span><span class=\"sa\">f<\/span><span class=\"s\">'Already listening to <\/span><span class=\"si\">{<\/span><span class=\"n\">room<\/span><span class=\"p\">.<\/span><span class=\"n\">key<\/span><span class=\"si\">}<\/span><span class=\"s\">'<\/span><span class=\"p\">)<\/span>\n\n    <span class=\"n\">room<\/span><span class=\"p\">.<\/span><span class=\"n\">listening<\/span> <span class=\"o\">=<\/span> <span class=\"bp\">True<\/span>\n    <span class=\"k\">print<\/span><span class=\"p\">(<\/span><span class=\"sa\">f<\/span><span class=\"s\">'Listen Room <\/span><span class=\"si\">{<\/span><span class=\"n\">room<\/span><span class=\"p\">.<\/span><span class=\"n\">key<\/span><span class=\"si\">}<\/span><span class=\"s\">'<\/span><span class=\"p\">)<\/span>\n    <span class=\"n\">stats<\/span> <span class=\"o\">=<\/span> <span class=\"n\">Stats<\/span><span class=\"p\">(<\/span><span class=\"sa\">f<\/span><span class=\"s\">'Outgoing <\/span><span class=\"si\">{<\/span><span class=\"n\">room<\/span><span class=\"p\">.<\/span><span class=\"n\">key<\/span><span class=\"si\">}<\/span><span class=\"s\">'<\/span><span class=\"p\">)<\/span>\n    <span class=\"k\">while<\/span> <span class=\"bp\">True<\/span><span class=\"p\">:<\/span>\n        <span class=\"n\">qevent<\/span> <span class=\"o\">=<\/span> <span class=\"k\">await<\/span> <span class=\"n\">room<\/span><span class=\"p\">.<\/span><span class=\"n\">event_queue<\/span><span class=\"p\">.<\/span><span class=\"n\">get<\/span><span class=\"p\">()<\/span>\n        <span class=\"k\">if<\/span> <span class=\"n\">qevent<\/span> <span class=\"o\">==<\/span> <span class=\"bp\">None<\/span><span class=\"p\">:<\/span>\n            <span class=\"k\">break<\/span>\n\n        <span class=\"c1\"># Add any new clients that have shown up, this handler must control this to avoid it\n<\/span>        <span class=\"c1\"># happening inside the loop below\n<\/span>        <span class=\"k\">if<\/span> <span class=\"nb\">len<\/span><span class=\"p\">(<\/span><span class=\"n\">room<\/span><span class=\"p\">.<\/span><span class=\"n\">new_clients<\/span><span class=\"p\">)<\/span> <span class=\"o\">&gt;<\/span> <span class=\"mi\">0<\/span><span class=\"p\">:<\/span>\n            <span class=\"k\">for<\/span> <span class=\"n\">client<\/span> <span class=\"ow\">in<\/span> <span class=\"n\">room<\/span><span class=\"p\">.<\/span><span class=\"n\">new_clients<\/span><span class=\"p\">:<\/span>\n                <span class=\"n\">room<\/span><span class=\"p\">.<\/span><span class=\"n\">clients<\/span><span class=\"p\">[<\/span><span class=\"n\">client<\/span><span class=\"p\">.<\/span><span class=\"nb\">id<\/span><span class=\"p\">]<\/span> <span class=\"o\">=<\/span> <span class=\"n\">client<\/span>\n            <span class=\"n\">room<\/span><span class=\"p\">.<\/span><span class=\"n\">new_clients<\/span> <span class=\"o\">=<\/span> <span class=\"p\">[]<\/span>\n\n        <span class=\"c1\"># In my game I'll track IDs in Redis, to survie unexpected failures.\n<\/span>        <span class=\"c1\"># The messages will also be pushed there, to be picked up by another process for DB storage\n<\/span>        <span class=\"n\">room<\/span><span class=\"p\">.<\/span><span class=\"n\">msg_id<\/span> <span class=\"o\">+=<\/span> <span class=\"mi\">1<\/span>\n        <span class=\"n\">qevent<\/span><span class=\"p\">[<\/span><span class=\"s\">'msg_id'<\/span><span class=\"p\">]<\/span> <span class=\"o\">=<\/span> <span class=\"n\">room<\/span><span class=\"p\">.<\/span><span class=\"n\">msg_id<\/span>\n\n        <span class=\"n\">count<\/span> <span class=\"o\">=<\/span> <span class=\"mi\">0<\/span>\n        <span class=\"n\">disconnected<\/span><span class=\"p\">:<\/span> <span class=\"n\">List<\/span><span class=\"p\">[<\/span><span class=\"nb\">int<\/span><span class=\"p\">]<\/span> <span class=\"o\">=<\/span> <span class=\"p\">[]<\/span>\n        <span class=\"k\">for<\/span> <span class=\"n\">client<\/span> <span class=\"ow\">in<\/span> <span class=\"n\">room<\/span><span class=\"p\">.<\/span><span class=\"n\">clients<\/span><span class=\"p\">.<\/span><span class=\"n\">values<\/span><span class=\"p\">():<\/span>\n            <span class=\"k\">if<\/span> <span class=\"n\">client<\/span><span class=\"p\">.<\/span><span class=\"n\">disconnected<\/span><span class=\"p\">:<\/span>\n                <span class=\"n\">disconnected<\/span><span class=\"p\">.<\/span><span class=\"n\">append<\/span><span class=\"p\">(<\/span><span class=\"n\">client<\/span><span class=\"p\">.<\/span><span class=\"nb\">id<\/span><span class=\"p\">)<\/span>\n                <span class=\"k\">continue<\/span>\n            <span class=\"n\">count<\/span> <span class=\"o\">+=<\/span> <span class=\"mi\">1<\/span>\n\n            <span class=\"c1\"># There's likely some asyncio technique to do this in parallel\n<\/span>            <span class=\"k\">try<\/span><span class=\"p\">:<\/span>\n                <span class=\"k\">await<\/span> <span class=\"n\">client<\/span><span class=\"p\">.<\/span><span class=\"n\">socket<\/span><span class=\"p\">.<\/span><span class=\"n\">send<\/span><span class=\"p\">(<\/span><span class=\"n\">encode_msg<\/span><span class=\"p\">(<\/span><span class=\"n\">qevent<\/span><span class=\"p\">))<\/span>\n            <span class=\"k\">except<\/span> <span class=\"n\">websockets<\/span><span class=\"p\">.<\/span><span class=\"n\">ConnectionClosed<\/span><span class=\"p\">:<\/span>\n                <span class=\"k\">print<\/span><span class=\"p\">(<\/span><span class=\"s\">\"Lost client in send\"<\/span><span class=\"p\">)<\/span>\n                <span class=\"n\">client<\/span><span class=\"p\">.<\/span><span class=\"n\">disconnected<\/span> <span class=\"o\">=<\/span> <span class=\"bp\">True<\/span>\n                <span class=\"c1\"># Hoping incoming will detect disconnected as well\n<\/span>\n        <span class=\"n\">stats<\/span><span class=\"p\">.<\/span><span class=\"n\">incr<\/span><span class=\"p\">(<\/span><span class=\"n\">count<\/span><span class=\"p\">)<\/span>\n\n        <span class=\"c1\"># Remove clients that aren't there anymore. I don't really need this in my game, but it's\n<\/span>        <span class=\"c1\"># good to not let long-lived rooms build-up cruft.\n<\/span>        <span class=\"k\">for<\/span> <span class=\"n\">d<\/span> <span class=\"ow\">in<\/span> <span class=\"n\">disconnected<\/span><span class=\"p\">:<\/span>\n            <span class=\"c1\"># Check again since they may have reconnected in other loop\n<\/span>            <span class=\"k\">if<\/span> <span class=\"n\">room<\/span><span class=\"p\">.<\/span><span class=\"n\">clients<\/span><span class=\"p\">[<\/span><span class=\"n\">d<\/span><span class=\"p\">]:<\/span>\n                <span class=\"k\">del<\/span> <span class=\"n\">room<\/span><span class=\"p\">.<\/span><span class=\"n\">clients<\/span><span class=\"p\">[<\/span><span class=\"n\">d<\/span><span class=\"p\">]<\/span>\n\n    <span class=\"k\">print<\/span><span class=\"p\">(<\/span><span class=\"sa\">f<\/span><span class=\"s\">'Unlisten Room <\/span><span class=\"si\">{<\/span><span class=\"n\">room<\/span><span class=\"p\">.<\/span><span class=\"n\">key<\/span><span class=\"si\">}<\/span><span class=\"s\">'<\/span><span class=\"p\">)<\/span>\n    <span class=\"n\">room<\/span><span class=\"p\">.<\/span><span class=\"n\">listening<\/span> <span class=\"o\">=<\/span> <span class=\"bp\">False<\/span>\n\n\n<span class=\"k\">async<\/span> <span class=\"k\">def<\/span> <span class=\"nf\">listen_socket<\/span><span class=\"p\">(<\/span><span class=\"n\">websocket<\/span><span class=\"p\">,<\/span> <span class=\"n\">path<\/span><span class=\"p\">):<\/span>\n    <span class=\"k\">global<\/span> <span class=\"n\">rooms<\/span><span class=\"p\">,<\/span> <span class=\"n\">client_id_count<\/span>\n    <span class=\"k\">print<\/span><span class=\"p\">(<\/span><span class=\"s\">\"connect\"<\/span><span class=\"p\">,<\/span> <span class=\"n\">path<\/span><span class=\"p\">)<\/span>\n    <span class=\"n\">client_id_count<\/span> <span class=\"o\">+=<\/span> <span class=\"mi\">1<\/span>\n    <span class=\"n\">room<\/span><span class=\"p\">:<\/span> <span class=\"n\">Optional<\/span><span class=\"p\">[<\/span><span class=\"n\">Room<\/span><span class=\"p\">]<\/span> <span class=\"o\">=<\/span> <span class=\"bp\">None<\/span>\n    <span class=\"n\">client<\/span> <span class=\"o\">=<\/span> <span class=\"n\">Client<\/span><span class=\"p\">(<\/span><span class=\"nb\">id<\/span><span class=\"o\">=<\/span><span class=\"n\">client_id_count<\/span><span class=\"p\">,<\/span> <span class=\"n\">socket<\/span><span class=\"o\">=<\/span><span class=\"n\">websocket<\/span><span class=\"p\">)<\/span>\n\n    <span class=\"n\">stats<\/span> <span class=\"o\">=<\/span> <span class=\"n\">Stats<\/span><span class=\"p\">(<\/span><span class=\"s\">'Incoming'<\/span><span class=\"p\">)<\/span>\n    <span class=\"k\">try<\/span><span class=\"p\">:<\/span>\n        <span class=\"k\">async<\/span> <span class=\"k\">for<\/span> <span class=\"n\">message_raw<\/span> <span class=\"ow\">in<\/span> <span class=\"n\">websocket<\/span><span class=\"p\">:<\/span>\n            <span class=\"n\">message<\/span> <span class=\"o\">=<\/span> <span class=\"n\">decode_msg<\/span><span class=\"p\">(<\/span><span class=\"n\">message_raw<\/span><span class=\"p\">)<\/span>\n            <span class=\"k\">if<\/span> <span class=\"n\">message<\/span><span class=\"p\">[<\/span><span class=\"s\">'type'<\/span><span class=\"p\">]<\/span> <span class=\"o\">==<\/span> <span class=\"s\">'join'<\/span><span class=\"p\">:<\/span>\n                <span class=\"c1\"># Get\/create room\n<\/span>                <span class=\"n\">room_key<\/span> <span class=\"o\">=<\/span> <span class=\"n\">message<\/span><span class=\"p\">[<\/span><span class=\"s\">'room'<\/span><span class=\"p\">]<\/span>\n                <span class=\"k\">if<\/span> <span class=\"ow\">not<\/span> <span class=\"n\">room_key<\/span> <span class=\"ow\">in<\/span> <span class=\"n\">rooms<\/span><span class=\"p\">:<\/span>\n                    <span class=\"n\">room<\/span> <span class=\"o\">=<\/span> <span class=\"n\">Room<\/span><span class=\"p\">(<\/span><span class=\"n\">key<\/span><span class=\"o\">=<\/span><span class=\"n\">room_key<\/span><span class=\"p\">)<\/span>\n                    <span class=\"n\">rooms<\/span><span class=\"p\">[<\/span><span class=\"n\">room_key<\/span><span class=\"p\">]<\/span> <span class=\"o\">=<\/span> <span class=\"n\">room<\/span>\n\n                    <span class=\"n\">room<\/span><span class=\"p\">.<\/span><span class=\"n\">future<\/span> <span class=\"o\">=<\/span> <span class=\"n\">asyncio<\/span><span class=\"p\">.<\/span><span class=\"n\">ensure_future<\/span><span class=\"p\">(<\/span><span class=\"n\">listen_room<\/span><span class=\"p\">(<\/span><span class=\"n\">room<\/span><span class=\"p\">))<\/span>\n                <span class=\"k\">else<\/span><span class=\"p\">:<\/span>\n                    <span class=\"n\">room<\/span> <span class=\"o\">=<\/span> <span class=\"n\">rooms<\/span><span class=\"p\">[<\/span><span class=\"n\">room_key<\/span><span class=\"p\">]<\/span>\n\n                <span class=\"c1\"># Add client to the room\n<\/span>                <span class=\"n\">room<\/span><span class=\"p\">.<\/span><span class=\"n\">new_clients<\/span><span class=\"p\">.<\/span><span class=\"n\">append<\/span><span class=\"p\">(<\/span><span class=\"n\">client<\/span><span class=\"p\">)<\/span>\n\n                <span class=\"c1\"># Tell the client which id they are.\n<\/span>                <span class=\"k\">await<\/span> <span class=\"n\">websocket<\/span><span class=\"p\">.<\/span><span class=\"n\">send<\/span><span class=\"p\">(<\/span><span class=\"n\">encode_msg<\/span><span class=\"p\">({<\/span>\n                    <span class=\"s\">'type'<\/span><span class=\"p\">:<\/span> <span class=\"s\">'joined'<\/span><span class=\"p\">,<\/span>\n                    <span class=\"s\">'client_id'<\/span><span class=\"p\">:<\/span> <span class=\"n\">client<\/span><span class=\"p\">.<\/span><span class=\"nb\">id<\/span>\n                <span class=\"p\">}))<\/span>\n\n            <span class=\"k\">elif<\/span> <span class=\"n\">room<\/span><span class=\"p\">:<\/span>\n                <span class=\"c1\"># Identify message and pass it off to the room queue\n<\/span>                <span class=\"n\">message<\/span><span class=\"p\">[<\/span><span class=\"s\">'client_id'<\/span><span class=\"p\">]<\/span> <span class=\"o\">=<\/span> <span class=\"n\">client<\/span><span class=\"p\">.<\/span><span class=\"nb\">id<\/span>\n                <span class=\"k\">await<\/span> <span class=\"n\">room<\/span><span class=\"p\">.<\/span><span class=\"n\">event_queue<\/span><span class=\"p\">.<\/span><span class=\"n\">put<\/span><span class=\"p\">(<\/span><span class=\"n\">message<\/span><span class=\"p\">)<\/span>\n            <span class=\"k\">else<\/span><span class=\"p\">:<\/span>\n                <span class=\"c1\"># Behave as trival echo server if not in room (will be removed in my final version)\n<\/span>                <span class=\"k\">await<\/span> <span class=\"n\">websocket<\/span><span class=\"p\">.<\/span><span class=\"n\">send<\/span><span class=\"p\">(<\/span><span class=\"n\">encode_msg<\/span><span class=\"p\">(<\/span><span class=\"n\">message<\/span><span class=\"p\">))<\/span>\n            <span class=\"n\">stats<\/span><span class=\"p\">.<\/span><span class=\"n\">incr<\/span><span class=\"p\">()<\/span>\n    <span class=\"k\">except<\/span> <span class=\"n\">websockets<\/span><span class=\"p\">.<\/span><span class=\"n\">ConnectionClosed<\/span><span class=\"p\">:<\/span>\n        <span class=\"k\">pass<\/span>\n    <span class=\"k\">except<\/span> <span class=\"nb\">Exception<\/span> <span class=\"k\">as<\/span> <span class=\"n\">e<\/span><span class=\"p\">:<\/span>\n        <span class=\"c1\"># In case something else happens we want to ditch this client.  This won't come from\n<\/span>        <span class=\"c1\"># websockets, but likely the code above, like having a broken JSON message\n<\/span>        <span class=\"k\">print<\/span><span class=\"p\">(<\/span><span class=\"n\">e<\/span><span class=\"p\">)<\/span>\n        <span class=\"k\">pass<\/span>\n\n    <span class=\"c1\"># Only mark disconnected for queue loop on clients isn't broken\n<\/span>    <span class=\"n\">client<\/span><span class=\"p\">.<\/span><span class=\"n\">disconnected<\/span> <span class=\"o\">=<\/span> <span class=\"bp\">True<\/span>\n    <span class=\"k\">if<\/span> <span class=\"n\">room<\/span> <span class=\"ow\">is<\/span> <span class=\"ow\">not<\/span> <span class=\"bp\">None<\/span><span class=\"p\">:<\/span>\n        <span class=\"c1\"># Though if zero we can kill the listener and clean up fully\n<\/span>        <span class=\"k\">if<\/span> <span class=\"n\">room<\/span><span class=\"p\">.<\/span><span class=\"n\">client_count<\/span><span class=\"p\">()<\/span> <span class=\"o\">==<\/span> <span class=\"mi\">0<\/span><span class=\"p\">:<\/span>\n            <span class=\"k\">await<\/span> <span class=\"n\">room<\/span><span class=\"p\">.<\/span><span class=\"n\">event_queue<\/span><span class=\"p\">.<\/span><span class=\"n\">put<\/span><span class=\"p\">(<\/span><span class=\"bp\">None<\/span><span class=\"p\">)<\/span>\n            <span class=\"k\">del<\/span> <span class=\"n\">rooms<\/span><span class=\"p\">[<\/span><span class=\"n\">room<\/span><span class=\"p\">.<\/span><span class=\"n\">key<\/span><span class=\"p\">]<\/span>\n            <span class=\"k\">await<\/span> <span class=\"n\">room<\/span><span class=\"p\">.<\/span><span class=\"n\">future<\/span>\n            <span class=\"k\">print<\/span><span class=\"p\">(<\/span><span class=\"sa\">f<\/span><span class=\"s\">\"Cleaned Room <\/span><span class=\"si\">{<\/span><span class=\"n\">room<\/span><span class=\"p\">.<\/span><span class=\"n\">key<\/span><span class=\"si\">}<\/span><span class=\"s\">\"<\/span><span class=\"p\">)<\/span>\n\n    <span class=\"k\">print<\/span><span class=\"p\">(<\/span><span class=\"s\">\"disconnect\"<\/span><span class=\"p\">,<\/span> <span class=\"n\">rooms<\/span><span class=\"p\">)<\/span>\n\n\n<span class=\"k\">def<\/span> <span class=\"nf\">main<\/span><span class=\"p\">()<\/span> <span class=\"o\">-&gt;<\/span> <span class=\"bp\">None<\/span><span class=\"p\">:<\/span>\n    <span class=\"n\">start_server<\/span> <span class=\"o\">=<\/span> <span class=\"n\">websockets<\/span><span class=\"p\">.<\/span><span class=\"n\">serve<\/span><span class=\"p\">(<\/span><span class=\"n\">listen_socket<\/span><span class=\"p\">,<\/span> <span class=\"s\">\"localhost\"<\/span><span class=\"p\">,<\/span> <span class=\"mi\">8765<\/span><span class=\"p\">,<\/span> <span class=\"n\">ping_interval<\/span><span class=\"o\">=<\/span><span class=\"mi\">5<\/span><span class=\"p\">,<\/span> <span class=\"n\">ping_timeout<\/span><span class=\"o\">=<\/span><span class=\"mi\">5<\/span><span class=\"p\">)<\/span>\n\n    <span class=\"n\">asyncio<\/span><span class=\"p\">.<\/span><span class=\"n\">get_event_loop<\/span><span class=\"p\">().<\/span><span class=\"n\">run_until_complete<\/span><span class=\"p\">(<\/span><span class=\"n\">start_server<\/span><span class=\"p\">)<\/span>\n\n    <span class=\"n\">asyncio<\/span><span class=\"p\">.<\/span><span class=\"n\">get_event_loop<\/span><span class=\"p\">().<\/span><span class=\"n\">run_forever<\/span><span class=\"p\">()<\/span>\n\n\n<span class=\"n\">main<\/span><span class=\"p\">()<\/span>\n<\/code><\/pre>\n\n<\/div>\n\n<h2>\n  \n  \n  ws_client.py\n<\/h2>\n\n<p>A simple client that validates the correct ordering of messages. Provide a room on the command line.<\/p>\n\n<p>There's an option to slow this client which forces the server to disconnect it when the buffers fill up.<\/p>\n\n<p>You'll note my client test code is rougher than my server code, and lacking many type definitions. This code will not be used long-term, but the server code will be.<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight python\"><code><span class=\"kn\">from<\/span> <span class=\"nn\">typing<\/span> <span class=\"kn\">import<\/span> <span class=\"o\">*<\/span>\n<span class=\"kn\">import<\/span> <span class=\"nn\">asyncio<\/span><span class=\"p\">,<\/span> <span class=\"n\">json<\/span><span class=\"p\">,<\/span> <span class=\"n\">websockets<\/span><span class=\"p\">,<\/span> <span class=\"n\">time<\/span><span class=\"p\">,<\/span> <span class=\"n\">sys<\/span>\n\n<span class=\"k\">if<\/span> <span class=\"nb\">len<\/span><span class=\"p\">(<\/span><span class=\"n\">sys<\/span><span class=\"p\">.<\/span><span class=\"n\">argv<\/span><span class=\"p\">)<\/span> <span class=\"o\">&lt;<\/span> <span class=\"mi\">2<\/span><span class=\"p\">:<\/span>\n    <span class=\"k\">print<\/span><span class=\"p\">(<\/span><span class=\"sa\">f<\/span><span class=\"s\">\"Sytnax <\/span><span class=\"si\">{<\/span><span class=\"n\">sys<\/span><span class=\"p\">.<\/span><span class=\"n\">argv<\/span><span class=\"p\">[<\/span><span class=\"mi\">0<\/span><span class=\"p\">]<\/span><span class=\"si\">}<\/span><span class=\"s\"> room (delay)\"<\/span> <span class=\"p\">)<\/span>\n    <span class=\"n\">sys<\/span><span class=\"p\">.<\/span><span class=\"nb\">exit<\/span><span class=\"p\">(<\/span><span class=\"o\">-<\/span><span class=\"mi\">1<\/span><span class=\"p\">)<\/span>\n\n<span class=\"n\">room<\/span> <span class=\"o\">=<\/span> <span class=\"n\">sys<\/span><span class=\"p\">.<\/span><span class=\"n\">argv<\/span><span class=\"p\">[<\/span><span class=\"mi\">1<\/span><span class=\"p\">]<\/span>\n<span class=\"c1\"># A non-zero slow creates a client that can't keep up. If there are other clients in the room\n# it will end up breaking, causing the server to disconnect it.\n<\/span><span class=\"n\">slow<\/span> <span class=\"o\">=<\/span> <span class=\"mf\">0.0<\/span>\n<span class=\"k\">if<\/span> <span class=\"nb\">len<\/span><span class=\"p\">(<\/span><span class=\"n\">sys<\/span><span class=\"p\">.<\/span><span class=\"n\">argv<\/span><span class=\"p\">)<\/span> <span class=\"o\">&gt;<\/span> <span class=\"mi\">2<\/span><span class=\"p\">:<\/span>\n    <span class=\"n\">slow<\/span> <span class=\"o\">=<\/span> <span class=\"nb\">float<\/span><span class=\"p\">(<\/span><span class=\"n\">sys<\/span><span class=\"p\">.<\/span><span class=\"n\">argv<\/span><span class=\"p\">[<\/span><span class=\"mi\">2<\/span><span class=\"p\">])<\/span>\n\n<span class=\"k\">def<\/span> <span class=\"nf\">encode_msg<\/span><span class=\"p\">(<\/span><span class=\"n\">msg<\/span><span class=\"p\">:<\/span> <span class=\"n\">Dict<\/span><span class=\"p\">)<\/span> <span class=\"o\">-&gt;<\/span> <span class=\"nb\">str<\/span><span class=\"p\">:<\/span>\n    <span class=\"k\">return<\/span> <span class=\"n\">json<\/span><span class=\"p\">.<\/span><span class=\"n\">dumps<\/span><span class=\"p\">(<\/span><span class=\"n\">msg<\/span><span class=\"p\">,<\/span> <span class=\"n\">ensure_ascii<\/span><span class=\"o\">=<\/span><span class=\"bp\">False<\/span><span class=\"p\">)<\/span>\n\n<span class=\"k\">def<\/span> <span class=\"nf\">decode_msg<\/span><span class=\"p\">(<\/span><span class=\"n\">text<\/span><span class=\"p\">:<\/span> <span class=\"nb\">str<\/span><span class=\"p\">)<\/span> <span class=\"o\">-&gt;<\/span> <span class=\"n\">Dict<\/span><span class=\"p\">:<\/span>\n    <span class=\"k\">return<\/span> <span class=\"n\">json<\/span><span class=\"p\">.<\/span><span class=\"n\">loads<\/span><span class=\"p\">(<\/span><span class=\"n\">text<\/span><span class=\"p\">)<\/span>\n\n<span class=\"c1\"># An even simpler stats tracker than the server \n<\/span><span class=\"n\">trigger_count<\/span> <span class=\"o\">=<\/span> <span class=\"mf\">5000.0<\/span>\n<span class=\"k\">if<\/span> <span class=\"n\">slow<\/span> <span class=\"o\">&gt;<\/span> <span class=\"mi\">0<\/span><span class=\"p\">:<\/span>\n    <span class=\"n\">trigger_count<\/span> <span class=\"o\">\/=<\/span> <span class=\"p\">(<\/span><span class=\"mi\">1<\/span><span class=\"o\">+<\/span><span class=\"n\">slow<\/span><span class=\"p\">)<\/span> <span class=\"o\">*<\/span> <span class=\"mi\">100<\/span>\n\n\n<span class=\"k\">async<\/span> <span class=\"k\">def<\/span> <span class=\"nf\">reader<\/span><span class=\"p\">(<\/span><span class=\"n\">websocket<\/span><span class=\"p\">):<\/span>\n    <span class=\"n\">count<\/span> <span class=\"o\">=<\/span> <span class=\"mi\">0<\/span>\n    <span class=\"n\">seq<\/span> <span class=\"o\">=<\/span> <span class=\"mi\">0<\/span>\n    <span class=\"n\">last_time<\/span> <span class=\"o\">=<\/span> <span class=\"n\">time<\/span><span class=\"p\">.<\/span><span class=\"n\">monotonic<\/span><span class=\"p\">()<\/span>\n    <span class=\"n\">client_id<\/span> <span class=\"o\">=<\/span> <span class=\"bp\">None<\/span>\n    <span class=\"n\">last_msg_id<\/span> <span class=\"o\">=<\/span> <span class=\"bp\">None<\/span>\n\n    <span class=\"k\">async<\/span> <span class=\"k\">for<\/span> <span class=\"n\">message_raw<\/span> <span class=\"ow\">in<\/span> <span class=\"n\">websocket<\/span><span class=\"p\">:<\/span>\n        <span class=\"n\">count<\/span> <span class=\"o\">+=<\/span> <span class=\"mi\">1<\/span>\n        <span class=\"n\">msg<\/span> <span class=\"o\">=<\/span> <span class=\"n\">decode_msg<\/span><span class=\"p\">(<\/span><span class=\"n\">message_raw<\/span><span class=\"p\">)<\/span>\n\n        <span class=\"k\">if<\/span> <span class=\"n\">msg<\/span><span class=\"p\">[<\/span><span class=\"s\">'type'<\/span><span class=\"p\">]<\/span> <span class=\"o\">==<\/span> <span class=\"s\">'joined'<\/span><span class=\"p\">:<\/span>\n            <span class=\"n\">client_id<\/span> <span class=\"o\">=<\/span> <span class=\"n\">msg<\/span><span class=\"p\">[<\/span><span class=\"s\">'client_id'<\/span><span class=\"p\">]<\/span>\n        <span class=\"k\">else<\/span><span class=\"p\">:<\/span>\n            <span class=\"c1\"># Ensure the messages have a single total order\n<\/span>            <span class=\"n\">msg_id<\/span> <span class=\"o\">=<\/span> <span class=\"n\">msg<\/span><span class=\"p\">[<\/span><span class=\"s\">'msg_id'<\/span><span class=\"p\">]<\/span>\n            <span class=\"k\">if<\/span> <span class=\"n\">last_msg_id<\/span> <span class=\"ow\">is<\/span> <span class=\"bp\">None<\/span><span class=\"p\">:<\/span>\n                <span class=\"n\">last_msg_id<\/span> <span class=\"o\">==<\/span> <span class=\"n\">msg_id<\/span>\n            <span class=\"k\">else<\/span><span class=\"p\">:<\/span>\n                <span class=\"k\">if<\/span> <span class=\"n\">msg_id<\/span> <span class=\"o\">!=<\/span> <span class=\"p\">(<\/span><span class=\"n\">last_msg_id<\/span><span class=\"o\">+<\/span><span class=\"mi\">1<\/span><span class=\"p\">):<\/span>\n                    <span class=\"k\">print<\/span><span class=\"p\">(<\/span><span class=\"n\">last_msg_id<\/span><span class=\"p\">,<\/span> <span class=\"n\">msg_id<\/span><span class=\"p\">)<\/span>\n                    <span class=\"k\">raise<\/span> <span class=\"nb\">Exception<\/span><span class=\"p\">(<\/span><span class=\"s\">\"bad msg sequence\"<\/span><span class=\"p\">)<\/span>\n\n        <span class=\"k\">if<\/span> <span class=\"n\">msg<\/span><span class=\"p\">[<\/span><span class=\"s\">'type'<\/span><span class=\"p\">]<\/span> <span class=\"o\">==<\/span> <span class=\"s\">'ping'<\/span> <span class=\"ow\">and<\/span> <span class=\"n\">client_id<\/span> <span class=\"o\">==<\/span> <span class=\"n\">msg<\/span><span class=\"p\">[<\/span><span class=\"s\">'client_id'<\/span><span class=\"p\">]:<\/span>\n            <span class=\"c1\"># Ensure our own measures retain the order we sent them\n<\/span>            <span class=\"k\">if<\/span> <span class=\"n\">msg<\/span><span class=\"p\">[<\/span><span class=\"s\">'seq'<\/span><span class=\"p\">]<\/span> <span class=\"o\">!=<\/span> <span class=\"n\">seq<\/span><span class=\"p\">:<\/span>\n                <span class=\"k\">print<\/span><span class=\"p\">(<\/span><span class=\"n\">seq<\/span><span class=\"p\">,<\/span> <span class=\"n\">message_raw<\/span><span class=\"p\">)<\/span>\n                <span class=\"k\">raise<\/span> <span class=\"nb\">Exception<\/span><span class=\"p\">(<\/span><span class=\"s\">\"bad message seq\"<\/span><span class=\"p\">)<\/span>\n\n        <span class=\"c1\"># Track rough throughput\n<\/span>        <span class=\"k\">if<\/span> <span class=\"n\">count<\/span> <span class=\"o\">&gt;=<\/span> <span class=\"n\">trigger_count<\/span><span class=\"p\">:<\/span>\n            <span class=\"n\">next_time<\/span> <span class=\"o\">=<\/span> <span class=\"n\">time<\/span><span class=\"p\">.<\/span><span class=\"n\">monotonic<\/span><span class=\"p\">()<\/span>\n            <span class=\"k\">print<\/span><span class=\"p\">(<\/span> <span class=\"sa\">f<\/span><span class=\"s\">'<\/span><span class=\"si\">{<\/span><span class=\"n\">count<\/span> <span class=\"o\">\/<\/span><span class=\"p\">(<\/span><span class=\"n\">next_time<\/span> <span class=\"o\">-<\/span> <span class=\"n\">last_time<\/span><span class=\"p\">)<\/span><span class=\"si\">}<\/span><span class=\"s\">\/s <\/span><span class=\"si\">{<\/span><span class=\"n\">room<\/span><span class=\"si\">}<\/span><span class=\"s\">'<\/span> <span class=\"p\">)<\/span>\n            <span class=\"n\">last_time<\/span> <span class=\"o\">=<\/span> <span class=\"n\">time<\/span><span class=\"p\">.<\/span><span class=\"n\">monotonic<\/span><span class=\"p\">()<\/span>\n            <span class=\"n\">count<\/span> <span class=\"o\">=<\/span> <span class=\"mi\">0<\/span>\n\n        <span class=\"k\">if<\/span> <span class=\"n\">client_id<\/span> <span class=\"o\">==<\/span> <span class=\"n\">msg<\/span><span class=\"p\">[<\/span><span class=\"s\">'client_id'<\/span><span class=\"p\">]:<\/span>\n            <span class=\"n\">seq<\/span> <span class=\"o\">+=<\/span> <span class=\"mi\">1<\/span>\n            <span class=\"k\">await<\/span> <span class=\"n\">websocket<\/span><span class=\"p\">.<\/span><span class=\"n\">send<\/span><span class=\"p\">(<\/span><span class=\"n\">encode_msg<\/span><span class=\"p\">({<\/span><span class=\"s\">'type'<\/span><span class=\"p\">:<\/span> <span class=\"s\">'ping'<\/span><span class=\"p\">,<\/span> <span class=\"s\">'seq'<\/span><span class=\"p\">:<\/span> <span class=\"n\">seq<\/span> <span class=\"p\">}))<\/span>\n\n        <span class=\"k\">if<\/span> <span class=\"n\">slow<\/span> <span class=\"o\">&gt;<\/span> <span class=\"mi\">0<\/span><span class=\"p\">:<\/span>\n            <span class=\"k\">await<\/span> <span class=\"n\">asyncio<\/span><span class=\"p\">.<\/span><span class=\"n\">sleep<\/span><span class=\"p\">(<\/span><span class=\"n\">slow<\/span><span class=\"p\">)<\/span>\n\n\n<span class=\"k\">async<\/span> <span class=\"k\">def<\/span> <span class=\"nf\">hello<\/span><span class=\"p\">():<\/span>\n    <span class=\"n\">uri<\/span> <span class=\"o\">=<\/span> <span class=\"s\">\"ws:\/\/localhost:8765\"<\/span>\n    <span class=\"k\">async<\/span> <span class=\"k\">with<\/span> <span class=\"n\">websockets<\/span><span class=\"p\">.<\/span><span class=\"n\">connect<\/span><span class=\"p\">(<\/span><span class=\"n\">uri<\/span><span class=\"p\">)<\/span> <span class=\"k\">as<\/span> <span class=\"n\">websocket<\/span><span class=\"p\">:<\/span>\n        <span class=\"k\">print<\/span><span class=\"p\">(<\/span><span class=\"s\">\"Connect\"<\/span><span class=\"p\">)<\/span>\n        <span class=\"k\">await<\/span> <span class=\"n\">websocket<\/span><span class=\"p\">.<\/span><span class=\"n\">send<\/span><span class=\"p\">(<\/span> <span class=\"n\">encode_msg<\/span><span class=\"p\">({<\/span> <span class=\"s\">'type'<\/span><span class=\"p\">:<\/span> <span class=\"s\">'join'<\/span><span class=\"p\">,<\/span> <span class=\"s\">'room'<\/span><span class=\"p\">:<\/span> <span class=\"n\">room<\/span> <span class=\"p\">})<\/span> <span class=\"p\">)<\/span>\n        <span class=\"n\">consumer_task<\/span> <span class=\"o\">=<\/span> <span class=\"n\">asyncio<\/span><span class=\"p\">.<\/span><span class=\"n\">ensure_future<\/span><span class=\"p\">(<\/span>\n            <span class=\"n\">reader<\/span><span class=\"p\">(<\/span><span class=\"n\">websocket<\/span><span class=\"p\">))<\/span>\n        <span class=\"n\">done<\/span> <span class=\"o\">=<\/span> <span class=\"k\">await<\/span> <span class=\"n\">asyncio<\/span><span class=\"p\">.<\/span><span class=\"n\">wait<\/span><span class=\"p\">(<\/span>\n            <span class=\"p\">[<\/span><span class=\"n\">consumer_task<\/span><span class=\"p\">],<\/span>\n            <span class=\"n\">return_when<\/span><span class=\"o\">=<\/span><span class=\"n\">asyncio<\/span><span class=\"p\">.<\/span><span class=\"n\">FIRST_COMPLETED<\/span><span class=\"p\">,<\/span>\n        <span class=\"p\">)<\/span>\n\n\n<span class=\"n\">asyncio<\/span><span class=\"p\">.<\/span><span class=\"n\">get_event_loop<\/span><span class=\"p\">().<\/span><span class=\"n\">run_until_complete<\/span><span class=\"p\">(<\/span><span class=\"n\">hello<\/span><span class=\"p\">())<\/span>\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<h2>\n  \n  \n  ws_client.html\n<\/h2>\n\n<p>A simple web browser client, to prove that it's working where I need it.<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight python\"><code><span class=\"o\">&lt;<\/span><span class=\"n\">html<\/span><span class=\"o\">&gt;<\/span>\n<span class=\"o\">&lt;<\/span><span class=\"n\">script<\/span><span class=\"o\">&gt;<\/span>\n<span class=\"n\">function<\/span> <span class=\"n\">loaded<\/span><span class=\"p\">()<\/span> <span class=\"p\">{<\/span>\n\n    <span class=\"n\">console<\/span><span class=\"p\">.<\/span><span class=\"n\">log<\/span><span class=\"p\">(<\/span><span class=\"s\">\"Loaded\"<\/span><span class=\"p\">)<\/span>\n    <span class=\"n\">const<\/span> <span class=\"n\">socket<\/span> <span class=\"o\">=<\/span> <span class=\"n\">new<\/span> <span class=\"n\">WebSocket<\/span><span class=\"p\">(<\/span><span class=\"s\">'ws:\/\/localhost:8765'<\/span><span class=\"p\">);<\/span>\n\n    <span class=\"n\">socket<\/span><span class=\"p\">.<\/span><span class=\"n\">addEventListener<\/span><span class=\"p\">(<\/span><span class=\"s\">'open'<\/span><span class=\"p\">,<\/span> <span class=\"n\">function<\/span> <span class=\"p\">(<\/span><span class=\"n\">event<\/span><span class=\"p\">)<\/span> <span class=\"p\">{<\/span>\n        <span class=\"n\">socket<\/span><span class=\"p\">.<\/span><span class=\"n\">send<\/span><span class=\"p\">(<\/span><span class=\"n\">encode_msg<\/span><span class=\"p\">({<\/span>\n            <span class=\"nb\">type<\/span><span class=\"p\">:<\/span> <span class=\"s\">'join'<\/span><span class=\"p\">,<\/span>\n            <span class=\"n\">room<\/span><span class=\"p\">:<\/span> <span class=\"s\">'js-room'<\/span><span class=\"p\">,<\/span>\n        <span class=\"p\">}));<\/span>\n        <span class=\"n\">console<\/span><span class=\"p\">.<\/span><span class=\"n\">log<\/span><span class=\"p\">(<\/span><span class=\"s\">\"Opened\"<\/span><span class=\"p\">)<\/span>\n    <span class=\"p\">});<\/span>\n\n    <span class=\"n\">socket<\/span><span class=\"p\">.<\/span><span class=\"n\">addEventListener<\/span><span class=\"p\">(<\/span><span class=\"s\">'message'<\/span><span class=\"p\">,<\/span> <span class=\"n\">function<\/span> <span class=\"p\">(<\/span><span class=\"n\">event<\/span><span class=\"p\">)<\/span> <span class=\"p\">{<\/span>\n        <span class=\"n\">console<\/span><span class=\"p\">.<\/span><span class=\"n\">log<\/span><span class=\"p\">(<\/span><span class=\"s\">'Message from server '<\/span><span class=\"p\">,<\/span> <span class=\"n\">event<\/span><span class=\"p\">.<\/span><span class=\"n\">data<\/span><span class=\"p\">);<\/span>\n    <span class=\"p\">});<\/span>\n\n    <span class=\"n\">window<\/span><span class=\"p\">.<\/span><span class=\"n\">addEventListener<\/span><span class=\"p\">(<\/span><span class=\"s\">'beforeunload'<\/span><span class=\"p\">,<\/span> <span class=\"p\">()<\/span> <span class=\"o\">=&gt;<\/span> <span class=\"p\">{<\/span>\n        <span class=\"n\">console<\/span><span class=\"p\">.<\/span><span class=\"n\">log<\/span><span class=\"p\">(<\/span><span class=\"s\">\"UNLOAD\"<\/span><span class=\"p\">)<\/span>\n        <span class=\"n\">socket<\/span><span class=\"p\">.<\/span><span class=\"n\">close<\/span><span class=\"p\">()<\/span>\n    <span class=\"p\">})<\/span>\n    <span class=\"n\">console<\/span><span class=\"p\">.<\/span><span class=\"n\">log<\/span><span class=\"p\">(<\/span><span class=\"s\">\"LOADED\"<\/span><span class=\"p\">)<\/span>\n\n    <span class=\"n\">let<\/span> <span class=\"n\">count<\/span> <span class=\"o\">=<\/span> <span class=\"mi\">0<\/span>\n    <span class=\"n\">setInterval<\/span><span class=\"p\">(()<\/span> <span class=\"o\">=&gt;<\/span> <span class=\"p\">{<\/span>\n        <span class=\"n\">count<\/span> <span class=\"o\">=<\/span> <span class=\"n\">count<\/span><span class=\"o\">+<\/span><span class=\"mi\">1<\/span>\n        <span class=\"n\">socket<\/span><span class=\"p\">.<\/span><span class=\"n\">send<\/span><span class=\"p\">(<\/span><span class=\"n\">encode_msg<\/span><span class=\"p\">({<\/span>\n            <span class=\"nb\">type<\/span><span class=\"p\">:<\/span> <span class=\"s\">'ping'<\/span><span class=\"p\">,<\/span>\n            <span class=\"n\">seq<\/span><span class=\"p\">:<\/span> <span class=\"n\">count<\/span><span class=\"p\">,<\/span>\n        <span class=\"p\">}))<\/span>\n    <span class=\"p\">},<\/span> <span class=\"mi\">500<\/span><span class=\"p\">)<\/span>\n<span class=\"p\">}<\/span>\n\n<span class=\"n\">function<\/span> <span class=\"n\">encode_msg<\/span><span class=\"p\">(<\/span><span class=\"n\">msg<\/span><span class=\"p\">)<\/span> <span class=\"p\">{<\/span> \n    <span class=\"k\">return<\/span> <span class=\"n\">JSON<\/span><span class=\"p\">.<\/span><span class=\"n\">stringify<\/span><span class=\"p\">(<\/span><span class=\"n\">msg<\/span><span class=\"p\">)<\/span>\n<span class=\"p\">}<\/span>\n\n<span class=\"n\">function<\/span> <span class=\"n\">decode_msg<\/span><span class=\"p\">(<\/span><span class=\"n\">text<\/span><span class=\"p\">)<\/span> <span class=\"p\">{<\/span>\n    <span class=\"k\">return<\/span> <span class=\"n\">JSON<\/span><span class=\"p\">.<\/span><span class=\"n\">parse<\/span><span class=\"p\">(<\/span><span class=\"n\">text<\/span><span class=\"p\">)<\/span>\n<span class=\"p\">}<\/span>\n\n<span class=\"o\">&lt;\/<\/span><span class=\"n\">script<\/span><span class=\"o\">&gt;<\/span>\n<span class=\"o\">&lt;<\/span><span class=\"n\">body<\/span> <span class=\"n\">onload<\/span><span class=\"o\">=<\/span><span class=\"s\">\"loaded()\"<\/span><span class=\"o\">&gt;<\/span>\n    <span class=\"o\">&lt;<\/span><span class=\"n\">p<\/span><span class=\"o\">&gt;<\/span><span class=\"n\">Text<\/span><span class=\"o\">&lt;\/<\/span><span class=\"n\">p<\/span><span class=\"o\">&gt;<\/span>\n<span class=\"o\">&lt;\/<\/span><span class=\"n\">body<\/span><span class=\"o\">&gt;<\/span>\n<span class=\"o\">&lt;\/<\/span><span class=\"n\">html<\/span><span class=\"o\">&gt;<\/span>\n<\/code><\/pre>\n\n<\/div>\n\n\n\n","category":["webdev","python","programming","gamedev"]},{"title":"Designing a Snacking Bear Puzzle","pubDate":"Mon, 23 Nov 2020 16:20:35 +0000","link":"https:\/\/dev.to\/mortoray\/designing-a-snacking-bear-puzzle-7gc","guid":"https:\/\/dev.to\/mortoray\/designing-a-snacking-bear-puzzle-7gc","description":"<p>In this stream I design a snacking bear puzzle. Watch my, sometimes painful, process of assembling graphics, thinking about a puzzle, and putting it together.<\/p>\n\n<p>First, the puzzle:<\/p>\n\n<p><a href=\"https:\/\/res.cloudinary.com\/practicaldev\/image\/fetch\/s--ChxaK61p--\/c_limit%2Cf_auto%2Cfl_progressive%2Cq_auto%2Cw_880\/https:\/\/dev-to-uploads.s3.amazonaws.com\/i\/529mye04b3wdfwzieq60.png\" class=\"article-body-image-wrapper\"><img src=\"https:\/\/res.cloudinary.com\/practicaldev\/image\/fetch\/s--ChxaK61p--\/c_limit%2Cf_auto%2Cfl_progressive%2Cq_auto%2Cw_880\/https:\/\/dev-to-uploads.s3.amazonaws.com\/i\/529mye04b3wdfwzieq60.png\" alt=\"Alt Text\"><\/a><\/p>\n\n<p><a href=\"https:\/\/edaqasroom.com\/singles\/snacking-bears\">Check you answer<\/a><\/p>\n\n<p>Now, the stream:<\/p>\n\n<p><iframe width=\"710\" height=\"399\" src=\"https:\/\/www.youtube.com\/embed\/_GFN5wobNUQ\">\n<\/iframe>\n<\/p>\n\n","category":["puzzle","stream","design"]},{"title":"What I look for while play-testing","pubDate":"Mon, 23 Nov 2020 15:15:48 +0000","link":"https:\/\/dev.to\/mortoray\/what-i-look-for-while-play-testing-1h89","guid":"https:\/\/dev.to\/mortoray\/what-i-look-for-while-play-testing-1h89","description":"<p>I\u2019ve completed a lot of play-testing lately for my game <a href=\"https:\/\/edaqa.link\/CarnivalDev\">Carnival<\/a>. It\u2019s a fascinating experience to watch people work through the puzzles. And it\u2019s a humbling experience as people stumble and flounder on failings in my designs. Without a doubt, focused user testing has made my game significantly. Here I\u2019d like to write about the principal things I\u2019m watching for.<\/p>\n\n<p>A friend of mine spurred this article, as she discovered I keep extensive notes during the play-test. She also tested the game and was curious to know if I wrote anything bad about her. I assured her it was all good stuff, while silently discarding the evidence behind my back.<\/p>\n\n<h1>\n  \n  \n  Play-testing Process\n<\/h1>\n\n<p><a href=\"https:\/\/edaqa.link\/CarnivalDev\">Carnival<\/a> is played online in the browser with teams that meet in an audio\/video chat. While the faces can be helpful for testing, it\u2019s mainly the audio I\u2019m listening to. For the video, I have one participant share their screen. It\u2019d be great if I could have all participants share their screens, but I\u2019d probably need more monitors to make sense of it.<\/p>\n\n<p>Hearing the people play is super useful, in addition to watching what they do on screen. Where they move their mouse can be telling, but hearing their \u201chmms\u201d and \u201chuhs\u201d, exasperated laments, and cries of fowl, tell even more. All of this helps me understand their thought process in solving puzzles and easing the play for future players.<\/p>\n\n<p>In the early rounds of play-testing, I instruct people about what to expect and how the game works. This allows me to test prior to the game being finished, avoiding some redundant work. I try to reduce the verbal preamble quickly as possible for subsequent teams, replacing it with the in-games systems. By the late stages of testing, I start with only a few hellos, then send the teams off on their own.<\/p>\n\n<blockquote>\n<p>I don\u2019t record these sessions. First off, I don\u2019t care to deal with the privacy aspect of archiving such data. But second, I\u2019d never watch them again. It\u2019s almost always better to test with a new group of people than labour endlessly on a single session. More people equals better testing, especially with a puzzle oriented game.<\/p>\n<\/blockquote>\n\n<h1>\n  \n  \n  Cryptic Notes\n<\/h1>\n\n<p>As mentioned, I take a lot of notes for these sessions, where lots means 2-3 pages of cryptic scrawling for the roughly 90 minute test sessions. A symbol showing the type of comment prefixes each line . At least in theory, in practice, I have some lines that have nothing before them. If something occurs to me during the test, I note it down, even if I\u2019m unsure it\u2019ll be helpful.<\/p>\n\n<p>For clarity here, I\u2019ll replace my symbols with emojis. That way we can debate their semantic importance, rather than dissect my post-modern scribbling approach to art.<\/p>\n\n<h2>\n  \n  \n  \ud83d\udc1f Red Herring\n<\/h2>\n\n<p>This became the most important symbol in testing. Red herring\u2019s a problem in puzzle games. These are things, an item, a graphic, some dialog, a pattern, virtually anything that misdirects the player. The player already has to resolve many important pieces of information in their head.<\/p>\n\n<p>I\u2019m talking about unintentional red herrings. These are colour patterns, or objects, that reasonable look like clues to a puzzle. They arise naturally out of the graphic design by accident. I have nothing but ire for designers who intentionally add red herrings in their game. It\u2019s an undebatable poor design that frustrates players. There are enough unintended red herrings to deal with already.<\/p>\n\n<p>For example, I have a puzzle in <a href=\"https:\/\/edaqa.link\/CarnivalDev\">Carnival<\/a> where you need to set a row of lights to the correct colours. I intend them to match another pattern on the screen, as hinted in some dialog. Lo-and-behold, some testers found another sequence of colours, in some flags, that seemed to match the lights as well. They then proceeded to match that pattern, which of course failed. This gets a big \u201c\ud83d\udc1f\u201d mark in the notes.<\/p>\n\n<blockquote>\n<p>\ud83d\udc1f sequence of flags matches light pattern<\/p>\n<\/blockquote>\n\n<h2>\n  \n  \n  \u2b58 Obvious thing to change\n<\/h2>\n\n<p>Many small, sometimes large, obvious ideas crop up during the tests. These may be ways to improve the puzzles, graphics improvements, overall UX ideas, or basically anything. The \u2b58 means I had a concrete thought and I should go back and improve it later \u2014 at which point I add a \u2714 to it.<\/p>\n\n<p>Each circle is a clear opportunity to improve the game. Rather than theorizing about things to improve, these all come from actual players. Fixing them would have a direct impact on some future player.  This doesn\u2019t mean they all get resolved, as priorities still play a role, and some of them are hard to fix.<\/p>\n\n<p>One example I have is with a ticket in the game. The player acquires the ticket and must present it to get into the carnival. I thought it\u2019d be obvious that you show the ticket to the man in the booth, but one player tried to use the ticket on the entry sign. It\u2019s not ridiculous, since the sign, with a \u201cNo Entry\u201d label hanging on top, is where you enter the park. What happens then is frustration. In their head the player has resolved the situation: they found a ticket and are using it to get inside. It not working is like throwing a wrench into their thought process.<\/p>\n\n<p>The obvious resolution was to make the sign accept the ticket, and that\u2019s the note I made:<\/p>\n\n<blockquote>\n<p>\u2b58 tried show ticket to sign, accept ticket<\/p>\n<\/blockquote>\n\n<p>Though, I ended up using a more humourous fix. The booth agent cracks a joke about an inanimate sign. This has the effect of affirming the player\u2019s action made sense, but was slightly off. Additionally, it made it clear where the ticket should be used instead. Oh, I could talk at length about this leading\u2026<\/p>\n\n<h2>\n  \n  \n  \ud83c\udfc1 Puzzle Solved\n<\/h2>\n\n<p>Whenever a player solves a puzzle, I note it with a little flag, and the time.<\/p>\n\n<blockquote>\n<p>\ud83c\udfc1 rabbit hand puppet, 23:03<\/p>\n<\/blockquote>\n\n<p>This is the wall time, since it\u2019s what I have most available. I record the start of the session as well, letting me calculate later the time offsets between puzzles.<\/p>\n\n<p>My goal here is to establish a good pacing for the game. Droughts, long periods between solutions, demotivates the player, and increases the likelihood they dislike the game.  Given my experience with the previous game, and numerous escape rooms I\u2019ve played, the pacing mostly worked as is.<\/p>\n\n<p>However, there were some clear problems on some puzzles \u2014 they took too long. In most cases, I didn\u2019t need to know the time to see that people were frustrated. But sometimes the time helped decide the frustration was okay. Perhaps the player was being impatient, rather than having an actual problem.<\/p>\n\n<p>Timing can also deceive. In one game the players took nearly 15 minutes for one puzzle, well beyond the typical 5 I consider a maximum \u2014 these aren\u2019t like fundamentally fixed numbers, so don\u2019t quote me on them.  But after the puzzle they both said \u201cwow, that was great\u201d \u2014 or something to that effect, I can\u2019t always read my writing.  This makes evaluation tricky, but at least the timed notes give some help.<\/p>\n\n<h2>\n  \n  \n  \u2753 Hint requested\n<\/h2>\n\n<p>Everybody gets stuck, but an experienced team should not need hints. It\u2019s why I treat hint requests during play-testing seriously.  The game has a built-in progressive hint system, so all I do is note where they requested the hint, and how many they needed.<\/p>\n\n<blockquote>\n<p>\u2753\u2753\u2753 dart game<\/p>\n<\/blockquote>\n\n<p>You might think requesting a lot of hints is bad, but for experienced teams, if they need one, they typically need many. This results from the hints being progressive; the players often already know the information in the first couple of hints, and only the later one reveals something new.  Short of deciphering the players thoughts with electrodes plugged into their brain, there\u2019s no real way to avoid this.<\/p>\n\n<blockquote>\n<p>Well, there is a way to avoid some hints, and I do that. Some hints can be tied to game actions, in particular with the inventory. But the general purpose hints can\u2019t be tied to in-game events. I can never really be certain the player already knows the hint.<\/p>\n<\/blockquote>\n\n<p>Requesting hints is part of puzzle games. It\u2019s totally fine if players ask for them, but it shouldn\u2019t be the standard approach to solving a puzzle. And since people who play-test are mainly interested in this genre of game, it biases the results \u2014 I assume the average player will require more. It\u2019s a good sign when some play tests have no \u2753's on them. Past that point, I can consider each tension point more carefully.<\/p>\n\n<h2>\n  \n  \n  \u201f What they said or did\n<\/h2>\n\n<p>To give context to my notes, and the players thought process, I take several notes about what they say, or what they do. The latter also uses quotes, because I couldn\u2019t think quickly what other symbol made sense, and in practice the notes are mixed.<\/p>\n\n<blockquote>\n<p>\u201fIs this random? click on light<\/p>\n<\/blockquote>\n\n<p>These notes primarily serve as anchors to the other ones<\/p>\n\n<p>I\u2019ll also make notes of solutions they\u2019ve tried that have failed. These can often give ideas of how they are thinking, or, sometimes, I end up accepting alternate solutions if they seem equally valid. For some of these I\u2019ll end up using a CIRCLE-\u201f combination in the notes.<\/p>\n\n<blockquote>\n<p>\u2b58 \u201f12345, hmm, doesn\u2019t work<\/p>\n<\/blockquote>\n\n<h2>\n  \n  \n  \u2757 Something is broken\n<\/h2>\n\n<p>I put this last since it\u2019s not the point of this level of play-testing. I have already resolved most of the functional defects, and the game is fully playable before I begin play-testing. Several minor defects still appear, and if I\u2019ve recently changed something, engine defects are possible (the frightening and thankfully rare \u2757\u2757).<\/p>\n\n<blockquote>\n<p>\u2757 B-girl font not converted path<\/p>\n<\/blockquote>\n\n<p>The \u2757 is for things that are 100% definitely technical defects. This could be a graphic that is missing, the wrong font used somewhere, or a typo in the text.  I suppose I could use them for defects in the puzzles, though oddly, I\u2019ve not have that situation come up yet. As this is a multiplayer game played over flaky networks, I\u2019ve tried hard, from the start, to make the logical game state consistent. That appears to work. While defects are possible in the puzzles still, I\u2019ve likely worked them out prior to starting play-testing.<\/p>\n\n<p>I think that\u2019s an important point: I\u2019ve played the game entirely many times before I do any play-testing. I want play-testing to focus on the things I can\u2019t find myself. Even the initial play-testers get a game that is working, albeit potentially without the hint system, and not all graphics in their final form, but functionally playable from beginning to end.<\/p>\n\n<h1>\n  \n  \n  Scribbles\n<\/h1>\n\n<p>These symbols are a general guide to the notes I take.  I still have other things written, sometimes in combination, or sometimes with no markings.<\/p>\n\n<p>I have found though that trying to itemize my thoughts produces firm notes. Rather than write everything, and anything, I focus on specific action items:<\/p>\n\n<p>-\ud83d\udc1f Something is misleading<br>\n-\u2757 something is broken<br>\n-\u2b58 Point for improvement<br>\n-\u2753 a puzzle may have a problem<\/p>\n\n<p>Where \ud83c\udfc1 and \u201f are then used to anchor those points, giving context when I go back later.<\/p>\n\n\n\n\n<p>While not a perfectly defined process, that\u2019s about how I did play-testing of <a href=\"https:\/\/edaqa.link\/CarnivalDev\">Carnival<\/a>. Watching people play, think, and laugh is fascinating. I also love watching the streams of people playing my game, as it gives another insight. I take notes from those as well.<\/p>\n\n","category":["testing","programming","playtest","gamedev"]},{"title":"Edaqa's Room: Removing Red Herrings from Carnival","pubDate":"Sat, 24 Oct 2020 12:54:58 +0000","link":"https:\/\/dev.to\/mortoray\/edaqa-s-room-removing-red-herrings-from-carnival-4ae1","guid":"https:\/\/dev.to\/mortoray\/edaqa-s-room-removing-red-herrings-from-carnival-4ae1","description":"<p><iframe width=\"710\" height=\"399\" src=\"https:\/\/www.youtube.com\/embed\/25HNEfz-828\">\n<\/iframe>\n<\/p>\n\n<p>I go through the results of a recent play-test, removing some distractions and unintended misdirections.<\/p>\n\n<p>This type of play-testing and corrections is a part of providing a smooth gameplay experience. I don't want people getting stuck on the wrong things. And without the play-testers, there's no way I can identify all those wrong things.<\/p>\n\n<p><a href=\"https:\/\/edaqa.link\/EdaqasRoom-DEVSpecial\">Edaqa's Room: Dev.to Special<\/a><\/p>\n\n<p><a href=\"https:\/\/twitch.tv\/mortoray\/\">Twitch Channel<\/a><br>\n<a href=\"https:\/\/youtube.com\/c\/EdaqaMortoray\/\">YouTube Channel<\/a><\/p>\n\n<p>-- <\/p>\n\n<p>All about the development of technology, graphics, and puzzles for my online escape and puzzle games. I'll show you how all the pieces work, how I design the games, and likely lots of poking fun at the Web.<\/p>\n\n","category":["stream","gamedev","webdev"]},{"title":"Edaqa's Room: Sound and Music using WebAudio","pubDate":"Sun, 27 Sep 2020 13:50:45 +0000","link":"https:\/\/dev.to\/mortoray\/edaqa-s-room-sound-and-music-using-webaudio-4ocg","guid":"https:\/\/dev.to\/mortoray\/edaqa-s-room-sound-and-music-using-webaudio-4ocg","description":"<p><iframe width=\"710\" height=\"399\" src=\"https:\/\/www.youtube.com\/embed\/_c7ome6YjFU\">\n<\/iframe>\n<\/p>\n\n<p>Today I show you how I use Web Audio to add sound effects and music to my game.<\/p>\n\n<ul>\n<li>Creating AudioContext<\/li>\n<li>Loading sounds with XMLHttpRequest and decodeAudioData<\/li>\n<li>Creating and playing buffers with createBufferSource<\/li>\n<li>Using the Audio element for music<\/li>\n<li>Fade in\/out<\/li>\n<\/ul>\n\n<p><a href=\"https:\/\/edaqa.link\/EdaqasRoom-DEVSpecial\">Edaqa's Room: Dev.to Special<\/a><\/p>\n\n<p><a href=\"https:\/\/twitch.tv\/mortoray\/\">Twitch Channel<\/a><br>\n<a href=\"https:\/\/youtube.com\/c\/EdaqaMortoray\/\">YouTube Channel<\/a><\/p>\n\n<p>-- <\/p>\n\n<p>All about the development of technology, graphics, and puzzles for my online escape and puzzle games. I'll show you how all the pieces work, how I design the games, and likely lots of poking fun at the Web.<\/p>\n\n","category":["stream","gamedev","webdev"]},{"title":"How to write a custom selector in React","pubDate":"Sun, 16 Aug 2020 18:32:46 +0000","link":"https:\/\/dev.to\/mortoray\/how-to-write-a-custom-selector-in-react-12cl","guid":"https:\/\/dev.to\/mortoray\/how-to-write-a-custom-selector-in-react-12cl","description":"<p>What's an efficient way to react to global state updates in a React app? If using Redux, you'd use a selector. But I don't use Redux for <a href=\"https:\/\/edaqa.link\/EdaqasRoom-Prototype-1\">my puzzle game<\/a>, as I have my own state object. It works similar to redux \u2014 I have an immutable state which is completely replaced on modification. All changes in the logical game state are done there.<\/p>\n\n<p>I used React's contexts to subscribe to state changes for the UI. This works, except parts of my UI are needlessly rerendered. The context update is sent upon any change, even if that part of the UI doesn't care about it. In practice this isn't too bad for my game, since I have few components listening, and pass properties down to memoized components. Still, I don't like inefficiency, and I know <code>useSelector<\/code> from other projects.<\/p>\n\n<p>How could I get the selector logic in my own React code? I have a game state, and I know which parts I'm interested in, so it should be easy. I thought a long time about how it should be done, a lot more time that it took to finally implement.  I'll cover what I did here, hopefully reducing the time you need to search for solutions.<\/p>\n\n<h1>\n  \n  \n  What does React offer?\n<\/h1>\n\n<p>Somewhere in <a href=\"https:\/\/reactjs.org\/\">React<\/a> is a subscription mechanism. That's how the components know to update when something changes. There are two options: context and state. They are both needed to build a selector.<\/p>\n\n<p>Using a context is well documented. Nonetheless, here's a brief outline of how I used this prior to creating a selector. <em>My actual code is TypeScript and has a layer of wrapping around this.<\/em><br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight jsx\"><code><span class=\"kd\">let<\/span> <span class=\"nx\">GameContext<\/span> <span class=\"o\">=<\/span> <span class=\"nx\">React<\/span><span class=\"p\">.<\/span><span class=\"nx\">createContext<\/span><span class=\"p\">([<\/span><span class=\"nx\">game_state<\/span><span class=\"p\">,<\/span> <span class=\"nx\">game_manager<\/span><span class=\"p\">])<\/span>\n<span class=\"kd\">let<\/span> <span class=\"nx\">game_manager<\/span> <span class=\"o\">=<\/span> <span class=\"nx\">get_game_magically_from_global<\/span><span class=\"p\">()<\/span>\n\n<span class=\"kd\">function<\/span> <span class=\"nx\">MainComponent<\/span><span class=\"p\">()<\/span> <span class=\"p\">{<\/span>\n    <span class=\"c1\">\/\/ I use React's state system to track the game state within this component.<\/span>\n    <span class=\"kd\">const<\/span> <span class=\"p\">[<\/span><span class=\"nx\">game_state<\/span><span class=\"p\">,<\/span> <span class=\"nx\">set_game_state<\/span><span class=\"p\">]<\/span> <span class=\"o\">=<\/span> <span class=\"nx\">React<\/span><span class=\"p\">.<\/span><span class=\"nx\">useState<\/span><span class=\"p\">(<\/span><span class=\"nx\">game_manager<\/span><span class=\"p\">.<\/span><span class=\"nx\">get_current_state<\/span><span class=\"p\">())<\/span>\n\n    <span class=\"c1\">\/\/ My game manager needs to tell me when the state changes.<\/span>\n    <span class=\"nx\">React<\/span><span class=\"p\">.<\/span><span class=\"nx\">useEffect<\/span><span class=\"p\">(()<\/span> <span class=\"o\">=&gt;<\/span> <span class=\"p\">{<\/span>\n        <span class=\"nx\">game_manager<\/span><span class=\"p\">.<\/span><span class=\"nx\">watch_state<\/span><span class=\"p\">(<\/span><span class=\"nx\">set_game_state<\/span><span class=\"p\">)<\/span>\n    <span class=\"p\">},<\/span> <span class=\"p\">[<\/span><span class=\"nx\">set_game_state<\/span><span class=\"p\">])<\/span>\n\n    <span class=\"c1\">\/\/ Provide the current state value to the context to pass down through the tree<\/span>\n    <span class=\"k\">return<\/span> <span class=\"p\">(<\/span>\n        <span class=\"p\">&lt;<\/span><span class=\"nc\">GameContext<\/span><span class=\"p\">.<\/span><span class=\"nc\">Provider<\/span> <span class=\"na\">value<\/span><span class=\"p\">=<\/span><span class=\"si\">{<\/span><span class=\"p\">[<\/span><span class=\"nx\">game_state<\/span><span class=\"p\">,<\/span> <span class=\"nx\">game_manager<\/span><span class=\"p\">]<\/span><span class=\"si\">}<\/span><span class=\"p\">&gt;<\/span>\n            <span class=\"p\">&lt;<\/span><span class=\"nc\">EdaqasRoomUI<\/span> <span class=\"p\">\/&gt;<\/span>\n        <span class=\"p\">&lt;\/<\/span><span class=\"nc\">GameContext<\/span><span class=\"p\">&gt;<\/span>\n    <span class=\"p\">)<\/span>\n<span class=\"p\">}<\/span>\n\n\n<span class=\"kd\">function<\/span> <span class=\"nx\">NiftyGameItem<\/span><span class=\"p\">()<\/span> <span class=\"p\">{<\/span>\n    <span class=\"kd\">const<\/span> <span class=\"p\">[<\/span><span class=\"nx\">game_state<\/span><span class=\"p\">,<\/span> <span class=\"nx\">game_manager<\/span><span class=\"p\">]<\/span> <span class=\"o\">=<\/span> <span class=\"nx\">React<\/span><span class=\"p\">.<\/span><span class=\"nx\">useContext<\/span><span class=\"p\">(<\/span><span class=\"nx\">GameContext<\/span><span class=\"p\">)<\/span>\n\n    <span class=\"kd\">const<\/span> <span class=\"nx\">drop<\/span> <span class=\"o\">=<\/span> <span class=\"nx\">React<\/span><span class=\"p\">.<\/span><span class=\"nx\">useCallback<\/span><span class=\"p\">(()<\/span> <span class=\"o\">=&gt;<\/span>\n        <span class=\"nx\">game_manager<\/span><span class=\"p\">.<\/span><span class=\"nx\">drop_item<\/span><span class=\"p\">()<\/span>\n    <span class=\"p\">},<\/span> <span class=\"p\">[<\/span><span class=\"nx\">game_manager<\/span><span class=\"p\">])<\/span>\n\n    <span class=\"k\">return<\/span> <span class=\"p\">(<\/span>\n        <span class=\"p\">&lt;<\/span><span class=\"nt\">img<\/span> <span class=\"na\">onClick<\/span><span class=\"p\">=<\/span><span class=\"si\">{<\/span><span class=\"nx\">drop<\/span><span class=\"si\">}<\/span> <span class=\"na\">src<\/span><span class=\"p\">=<\/span><span class=\"si\">{<\/span><span class=\"nx\">game_state<\/span><span class=\"p\">.<\/span><span class=\"nx\">held_item<\/span><span class=\"p\">.<\/span><span class=\"nx\">image<\/span><span class=\"si\">}<\/span> <span class=\"p\">\/&gt;<\/span>\n    <span class=\"p\">)<\/span>\n<span class=\"p\">}<\/span>\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<p><a href=\"https:\/\/res.cloudinary.com\/practicaldev\/image\/fetch\/s--Z4l-8py5--\/c_limit%2Cf_auto%2Cfl_progressive%2Cq_auto%2Cw_880\/https:\/\/mortoray.files.wordpress.com\/2020\/08\/screenshot_20200816_200017.png\" class=\"article-body-image-wrapper\"><img src=\"https:\/\/res.cloudinary.com\/practicaldev\/image\/fetch\/s--Z4l-8py5--\/c_limit%2Cf_auto%2Cfl_progressive%2Cq_auto%2Cw_880\/https:\/\/mortoray.files.wordpress.com\/2020\/08\/screenshot_20200816_200017.png\" alt=\"\"><\/a><\/p>\n\n<p>I provide both the current game state and the game manager in the context. The state is for reading and the context for providing feedback. This is similar to <a href=\"https:\/\/redux.js.org\/basics\/actions\/\">Redux's dispatcher<\/a>; my game manager also uses messages to communicate with the state.<\/p>\n\n<h2>\n  \n  \n  The State\n<\/h2>\n\n<p>Notice <a href=\"https:\/\/reactjs.org\/docs\/hooks-state.html\"><code>useState<\/code><\/a> in that example as well. For React, updating the context is no different than any other use of the state. The extra aspect of the context is providing that value to the descendents of the component. This is what the <code>Provider<\/code> does.<\/p>\n\n<p>State can be used without a context as well. Here's a simple example as a reminder.<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight jsx\"><code><span class=\"kd\">function<\/span> <span class=\"nx\">ExpandInventory<\/span><span class=\"p\">()<\/span> <span class=\"p\">{<\/span>\n    <span class=\"kd\">const<\/span> <span class=\"p\">[<\/span><span class=\"nx\">expanded<\/span><span class=\"p\">,<\/span> <span class=\"nx\">set_expanded<\/span><span class=\"p\">]<\/span> <span class=\"o\">=<\/span> <span class=\"nx\">React<\/span><span class=\"p\">.<\/span><span class=\"nx\">useState<\/span><span class=\"p\">(<\/span><span class=\"kc\">false<\/span><span class=\"p\">)<\/span>\n\n    <span class=\"kd\">const<\/span> <span class=\"nx\">toggle<\/span> <span class=\"o\">=<\/span> <span class=\"nx\">React<\/span><span class=\"p\">.<\/span><span class=\"nx\">useCallback<\/span><span class=\"p\">(()<\/span> <span class=\"o\">=&gt;<\/span> <span class=\"p\">{<\/span>\n        <span class=\"nx\">set_expanded<\/span><span class=\"p\">(<\/span><span class=\"o\">!<\/span><span class=\"nx\">expanded<\/span><span class=\"p\">)<\/span>\n    <span class=\"p\">},<\/span> <span class=\"p\">[<\/span><span class=\"nx\">expanded<\/span><span class=\"p\">,<\/span> <span class=\"nx\">set_expanded<\/span><span class=\"p\">])<\/span>\n\n    <span class=\"k\">return<\/span> <span class=\"p\">(<\/span>\n        <span class=\"p\">&lt;&gt;<\/span>\n            <span class=\"p\">&lt;<\/span><span class=\"nc\">CompactView<\/span> <span class=\"na\">onClick<\/span><span class=\"p\">=<\/span><span class=\"si\">{<\/span><span class=\"nx\">toggle<\/span><span class=\"si\">}<\/span> <span class=\"p\">\/&gt;<\/span>\n            <span class=\"si\">{<\/span><span class=\"nx\">expanded<\/span> <span class=\"o\">&amp;&amp;<\/span> <span class=\"p\">&lt;<\/span><span class=\"nc\">GloriousFullView<\/span> <span class=\"p\">\/&gt;<\/span><span class=\"si\">}<\/span>\n        <span class=\"p\">&lt;\/&gt;<\/span>\n    <span class=\"p\">)<\/span>\n<span class=\"p\">}<\/span>\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<p>When the user clicks on the compact view, the browser calls the toggle function, which modifies the state. When the state is modified React will rerender the control.<\/p>\n\n<p>JSX files create an illusion of close cooperative harmony between this code, the state, and the <a href=\"https:\/\/developer.mozilla.org\/en-US\/docs\/Web\/API\/Document_Object_Model\">HTML DOM<\/a>.  The truth is a lot uglier. The HTML goes through React's diff engine, then is assembled into the browser's DOM tree. The callback function lives in the global heap, connected to DOM object, as well as being a closure over the stack frame in which it was created. The closure will be called in response to a user's click, far away from the stack in which the render code was run.<\/p>\n\n<p>Understanding this structure is the key to making our own selectors. That <code>set_expanded<\/code> function can be called from anywhere and React will figure out how to update the component as a result.<\/p>\n\n<h1>\n  \n  \n  Too many updates\n<\/h1>\n\n<p>Any component that needs the game state can call <code>useContext(GameContext)<\/code>. The problem is that all state changes, whether they'd alter the component or not, cause the component to rerender. In my previous example, the <code>NiftyGameItem<\/code> only needs to update when <code>held_item<\/code> changes, yet currently it'll update anytime anything in the state changes. That's pointless and wasteful.<\/p>\n\n<p>If I were using Redux, I'd use a selector to solve this issue.<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight javascript\"><code><span class=\"kd\">const<\/span> <span class=\"nx\">held_item<\/span> <span class=\"o\">=<\/span> <span class=\"nx\">useSelector<\/span><span class=\"p\">(<\/span> <span class=\"nx\">game_state<\/span> <span class=\"o\">=&gt;<\/span> <span class=\"nx\">game_state<\/span><span class=\"p\">.<\/span><span class=\"nx\">held_item<\/span> <span class=\"p\">)<\/span>\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<p>Only when <code>game_state.held_item<\/code> changes will the component rerender.<\/p>\n\n<p><code>useSelector<\/code> itself isn't magical. It is essentially a layer in between the state and the control. It will listen to every update to the game state, and run the selection function. But it will only update the component if the result of the selection function changes.<\/p>\n\n<p>I wanted the same facility for my game state.<\/p>\n\n<h1>\n  \n  \n  My own selector\n<\/h1>\n\n<p><code>useState<\/code> is the primary hook into React's subscription system. At first, I looked for an explicit subscription API. What I wanted to do isn't directly covered in the state docs. But as I mentioned before, understanding how the callbacks, DOM, and state connect, assures me that my approach is correct.<\/p>\n\n<p>What is the goal? This is what I want my <code>NiftyGameItem<\/code> to look like, ignoring the <code>onClick<\/code> part for a moment.<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight plaintext\"><code>function NiftyGameItem() {\n    const held_item = useGameState( gs =&gt; gs.held_item )\n\n    return (\n        &lt;img src={game_state.held_item.image} \/&gt;\n    )\n}\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<p>I only want to update when <code>held_item<\/code> changes. Let's jump right the almost final code.<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight typescript\"><code><span class=\"kd\">type<\/span> <span class=\"nx\">game_selector<\/span><span class=\"o\">&lt;<\/span><span class=\"nx\">T<\/span><span class=\"o\">&gt;<\/span> <span class=\"o\">=<\/span> <span class=\"p\">(<\/span> <span class=\"nx\">state<\/span> <span class=\"p\">:<\/span> <span class=\"nx\">GT<\/span><span class=\"p\">.<\/span><span class=\"nx\">game_state<\/span> <span class=\"p\">)<\/span> <span class=\"o\">=&gt;<\/span> <span class=\"nx\">T<\/span>\n\n<span class=\"k\">export<\/span> <span class=\"kd\">function<\/span> <span class=\"nx\">useGameState<\/span><span class=\"o\">&lt;<\/span><span class=\"nx\">T<\/span><span class=\"o\">&gt;<\/span><span class=\"p\">(<\/span> <span class=\"nx\">gs<\/span> <span class=\"p\">:<\/span> <span class=\"nx\">game_selector<\/span><span class=\"o\">&lt;<\/span><span class=\"nx\">T<\/span><span class=\"o\">&gt;<\/span> <span class=\"p\">):<\/span> <span class=\"nx\">T<\/span> <span class=\"p\">{<\/span>\n    <span class=\"kd\">const<\/span> <span class=\"p\">[<\/span><span class=\"nx\">_<\/span><span class=\"p\">,<\/span> <span class=\"nx\">game_manager<\/span><span class=\"p\">]<\/span> <span class=\"o\">=<\/span> <span class=\"nx\">React<\/span><span class=\"p\">.<\/span><span class=\"nx\">useContext<\/span><span class=\"p\">(<\/span><span class=\"nx\">GameContext<\/span><span class=\"p\">)<\/span>\n\n    <span class=\"kd\">const<\/span> <span class=\"p\">[<\/span> <span class=\"nx\">state<\/span><span class=\"p\">,<\/span> <span class=\"nx\">set_state<\/span> <span class=\"p\">]<\/span> <span class=\"o\">=<\/span> <span class=\"nx\">React<\/span><span class=\"p\">.<\/span><span class=\"nx\">useState<\/span><span class=\"o\">&lt;<\/span><span class=\"nx\">T<\/span><span class=\"o\">&gt;<\/span><span class=\"p\">(():<\/span><span class=\"nx\">T<\/span> <span class=\"o\">=&gt;<\/span> <span class=\"nx\">gs<\/span><span class=\"p\">(<\/span><span class=\"nx\">game_manager<\/span><span class=\"p\">.<\/span><span class=\"nx\">current_game_state<\/span><span class=\"p\">()))<\/span>\n\n    <span class=\"nx\">React<\/span><span class=\"p\">.<\/span><span class=\"nx\">useEffect<\/span><span class=\"p\">(()<\/span> <span class=\"o\">=&gt;<\/span> <span class=\"p\">{<\/span>\n        <span class=\"kd\">const<\/span> <span class=\"nx\">track<\/span> <span class=\"o\">=<\/span> <span class=\"p\">{<\/span>\n            <span class=\"na\">current<\/span><span class=\"p\">:<\/span> <span class=\"nx\">state<\/span><span class=\"p\">,<\/span>\n        <span class=\"p\">}<\/span>\n\n        <span class=\"k\">return<\/span> <span class=\"nx\">game_manager<\/span><span class=\"p\">.<\/span><span class=\"nx\">listen_game_state<\/span><span class=\"p\">(<\/span> <span class=\"p\">(<\/span><span class=\"na\">game_state<\/span><span class=\"p\">:<\/span> <span class=\"nx\">GT<\/span><span class=\"p\">.<\/span><span class=\"nx\">game_state<\/span><span class=\"p\">)<\/span> <span class=\"o\">=&gt;<\/span> <span class=\"p\">{<\/span>\n            <span class=\"kd\">const<\/span> <span class=\"na\">next<\/span><span class=\"p\">:<\/span> <span class=\"nx\">T<\/span> <span class=\"o\">=<\/span> <span class=\"nx\">gs<\/span><span class=\"p\">(<\/span><span class=\"nx\">game_state<\/span><span class=\"p\">)<\/span>\n            <span class=\"k\">if<\/span> <span class=\"p\">(<\/span><span class=\"nx\">track<\/span><span class=\"p\">.<\/span><span class=\"nx\">current<\/span> <span class=\"o\">!=<\/span> <span class=\"nx\">next<\/span><span class=\"p\">)<\/span> <span class=\"p\">{<\/span>\n                <span class=\"nx\">track<\/span><span class=\"p\">.<\/span><span class=\"nx\">current<\/span> <span class=\"o\">=<\/span> <span class=\"nx\">next<\/span>\n                <span class=\"nx\">set_state<\/span><span class=\"p\">(<\/span><span class=\"nx\">next<\/span><span class=\"p\">)<\/span>\n            <span class=\"p\">}<\/span>\n        <span class=\"p\">})<\/span>\n    <span class=\"p\">},<\/span> <span class=\"p\">[<\/span><span class=\"nx\">game_manager<\/span><span class=\"p\">,<\/span> <span class=\"nx\">set_state<\/span><span class=\"p\">,<\/span> <span class=\"nx\">gs<\/span><span class=\"p\">])<\/span>\n\n    <span class=\"k\">return<\/span> <span class=\"nx\">gs<\/span><span class=\"p\">(<\/span><span class=\"nx\">state<\/span><span class=\"p\">)<\/span>\n<span class=\"p\">}<\/span>\n<\/code><\/pre>\n\n<\/div>\n\n\n\n\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight typescript\"><code>    <span class=\"kd\">const<\/span> <span class=\"p\">[<\/span><span class=\"nx\">_<\/span><span class=\"p\">,<\/span> <span class=\"nx\">game_manager<\/span><span class=\"p\">]<\/span> <span class=\"o\">=<\/span> <span class=\"nx\">React<\/span><span class=\"p\">.<\/span><span class=\"nx\">useContext<\/span><span class=\"p\">(<\/span><span class=\"nx\">GameContext<\/span><span class=\"p\">)<\/span>\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<p>I get the game manger as I did before, but we'll have to come back and fix something here.<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight typescript\"><code>    <span class=\"kd\">const<\/span> <span class=\"p\">[<\/span> <span class=\"nx\">state<\/span><span class=\"p\">,<\/span> <span class=\"nx\">set_state<\/span> <span class=\"p\">]<\/span> <span class=\"o\">=<\/span> <span class=\"nx\">React<\/span><span class=\"p\">.<\/span><span class=\"nx\">useState<\/span><span class=\"o\">&lt;<\/span><span class=\"nx\">T<\/span><span class=\"o\">&gt;<\/span><span class=\"p\">(():<\/span><span class=\"nx\">T<\/span> <span class=\"o\">=&gt;<\/span> <span class=\"nx\">gs<\/span><span class=\"p\">(<\/span><span class=\"nx\">game_manager<\/span><span class=\"p\">.<\/span><span class=\"nx\">current_game_state<\/span><span class=\"p\">()))<\/span>\n    <span class=\"p\">...<\/span>\n    <span class=\"k\">return<\/span> <span class=\"nx\">state<\/span>\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<p>I prep the state for the component. The game manager needs to provide the current state as it'll be needed when the component first renders, not only when the state updates. Here I don't track the entire game state, only the part that is of interest \u2014 the part extracted by the selector.<\/p>\n\n<p>A selector function, <code>gs<\/code> here, takes the global state as input and returns the part to be watched. My <code>useGameState<\/code> code calls  the <code>gs<\/code> selector function with the global state. The selector in my example is <code>gs =&gt; gs.held_item<\/code>, which retrieves only the <code>held_item<\/code>. In the game I have an on-screen indicator showing which item the player is currently holding.<\/p>\n\n<p>I return the state at the end of the function. In the first call, this will be the initial state. In subsequent calls, for each new render of the control, it'll be the current state.<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight typescript\"><code>        <span class=\"k\">return<\/span> <span class=\"nx\">game_manager<\/span><span class=\"p\">.<\/span><span class=\"nx\">listen_game_state<\/span><span class=\"p\">(<\/span> <span class=\"p\">(<\/span><span class=\"nx\">game_state<\/span><span class=\"p\">:<\/span> <span class=\"nx\">GT<\/span><span class=\"p\">.<\/span><span class=\"nx\">game_state<\/span><span class=\"p\">)<\/span> <span class=\"o\">=&gt;<\/span> <span class=\"p\">{<\/span>\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<p>The vital piece of code inside <code>useEffect<\/code> is the call to <code>listen_game_state<\/code>. I added this subscription function to the <code>game_manager<\/code>. The game manager already knows when the state updates, since it has to update the context. Now it updates the context as well as calling all the registered listeners. I'll show this code a bit further below.<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight typescript\"><code>        <span class=\"kd\">const<\/span> <span class=\"nx\">track<\/span> <span class=\"o\">=<\/span> <span class=\"p\">{<\/span>\n            <span class=\"na\">current<\/span><span class=\"p\">:<\/span> <span class=\"nx\">state<\/span><span class=\"p\">,<\/span>\n        <span class=\"p\">}<\/span>\n\n        <span class=\"k\">return<\/span> <span class=\"nx\">game_manager<\/span><span class=\"p\">.<\/span><span class=\"nx\">listen_game_state<\/span><span class=\"p\">(<\/span> <span class=\"p\">(<\/span><span class=\"nx\">game_state<\/span><span class=\"p\">:<\/span> <span class=\"nx\">GT<\/span><span class=\"p\">.<\/span><span class=\"nx\">game_state<\/span><span class=\"p\">)<\/span> <span class=\"o\">=&gt;<\/span> <span class=\"p\">{<\/span>\n            <span class=\"kd\">const<\/span> <span class=\"na\">next<\/span><span class=\"p\">:<\/span> <span class=\"nx\">T<\/span> <span class=\"o\">=<\/span> <span class=\"nx\">gs<\/span><span class=\"p\">(<\/span><span class=\"nx\">game_state<\/span><span class=\"p\">)<\/span>\n            <span class=\"k\">if<\/span> <span class=\"p\">(<\/span><span class=\"nx\">track<\/span><span class=\"p\">.<\/span><span class=\"nx\">current<\/span> <span class=\"o\">!=<\/span> <span class=\"nx\">next<\/span><span class=\"p\">)<\/span> <span class=\"p\">{<\/span>\n                <span class=\"nx\">track<\/span><span class=\"p\">.<\/span><span class=\"nx\">current<\/span> <span class=\"o\">=<\/span> <span class=\"nx\">next<\/span>\n                <span class=\"nx\">set_state<\/span><span class=\"p\">(<\/span><span class=\"nx\">next<\/span><span class=\"p\">)<\/span>\n            <span class=\"p\">}<\/span>\n        <span class=\"p\">})<\/span>\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<p>Each time the state updates, the caller provided selector function is called to select a part of the state. This is compared to what value it had previously, and only if it has changed do we call the <code>set_state<\/code> function.  If we were to call the <code>set_state<\/code> function every time, then it'd be no better than the caller listening for every state change.<\/p>\n\n<p>Note the <code>return<\/code>. The <code>listen_game_state<\/code> function returns an unsubscribe function, which will be called whenever the effect is reevaluated, or the component unmounts. The game manager shouldn't hold on to components that are no longer around.<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight typescript\"><code>    <span class=\"nx\">React<\/span><span class=\"p\">.<\/span><span class=\"nx\">useEffect<\/span><span class=\"p\">(()<\/span> <span class=\"o\">=&gt;<\/span> <span class=\"p\">{<\/span>\n        <span class=\"p\">...<\/span>\n    <span class=\"p\">},<\/span> <span class=\"p\">[<\/span><span class=\"nx\">game_manager<\/span><span class=\"p\">,<\/span> <span class=\"nx\">set_state<\/span><span class=\"p\">,<\/span> <span class=\"nx\">gs<\/span><span class=\"p\">])<\/span>\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<p>The <code>useEffect<\/code> runs once when the control is mounted (or first rendered, more correctly). I have a dependency list of <code>[game_manager, set_state, gs]<\/code> for correctness. Should one of those change the effect needs to be reevaluated to grab the new values. In practice, these dependencies never change.<\/p>\n\n<h2>\n  \n  \n  useState outside of a component?\n<\/h2>\n\n<p>It may seem unusual to call the <code>useState<\/code> function in something other than a react component. This type of chaining is allowed and expected. There's nothing special about calling <code>useState<\/code> directly in the component, or inside a function called by the component. React will understand which component it is in and associate it correctly.<\/p>\n\n<blockquote>\n<p>I've not looked into precisely how this works. My assumption is that it's a global value that tracks the current component. The hook functions inspect that variable to figure out where they are and to register the appropriate listeners. I don't see another option, since <code>useGameState<\/code> is a plain TS\/JS function \u2014 the JSX compiler has no chance to modify it. In threaded languages this stack would need to be thread local, but JS is single threaded (workers, etc. get their own global space, making them effectively thread local).<\/p>\n<\/blockquote>\n\n<p>My selector is a combination of existing React functions: <code>useState<\/code>, <code>useEffect<\/code>, and <code>useContext<\/code>.<\/p>\n\n<h2>\n  \n  \n  Hold on, there's a problem\n<\/h2>\n\n<p>I have an issue in the first line of the <code>useGameState<\/code> function:<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight typescript\"><code>    <span class=\"kd\">const<\/span> <span class=\"p\">[<\/span><span class=\"nx\">_<\/span><span class=\"p\">,<\/span> <span class=\"nx\">game_manager<\/span><span class=\"p\">]<\/span> <span class=\"o\">=<\/span> <span class=\"nx\">React<\/span><span class=\"p\">.<\/span><span class=\"nx\">useContext<\/span><span class=\"p\">(<\/span><span class=\"nx\">GameContext<\/span><span class=\"p\">)<\/span>\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<p>I reused the context from before, the one that provides the game state and the game manager. This is bad. Since it hooks into the game state context, this component will still be updated with every change of the state.<\/p>\n\n<p>To fix this, I added a new context which contains only the game manager.<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight plaintext\"><code>    const game_manager = React.useContext(GameManagerOnly)\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<p>This game manager never changes for the life of the game, thus no needless updates will be triggered by the call to <code>useContext<\/code>.<\/p>\n\n<blockquote>\n<p>At this point I gain nothing by storing the game manager singleton in a context. I could simply refer to the global object from my <code>useGameState<\/code> code. However, to behave as a proper React app, I've left it in the context. This may also be important for your project, where the object isn't a singleton.<\/p>\n<\/blockquote>\n\n<h1>\n  \n  \n  Save the batteries\n<\/h1>\n\n<p>Performance wasn't an issue for my game. Curiousity was part of the reason I wrote the selectors. The selectors do of course help; there were thousands of needless updates to components. Cutting back this processing time should help older machines, as well as saving battery power on tablets.<\/p>\n\n<p>I'll continue to make optimizations where I see them. It may be inconsequential compared to the massive browser SVG rendering overhead, but there's nothing I can do about that. As my games get more complex the calculation will continue to increase. Keeping it performant can only help long term.<\/p>\n\n<p>Plus, you know, curiousity. A solid reason to do something.<\/p>\n\n\n\n\n<p><em>Check out how this all comes together in my game <a href=\"https:\/\/edaqa.link\/EdaqasRoom-Prototype-3\">Edaqa's Room: Prototype<\/a>. A collaborative online escape room full of puzzles, adventure, and probably no vampires.<\/em><\/p>\n\n<p><a href=\"https:\/\/edaqa.link\/EdaqasRoom-Prototype-3\"><img src=\"https:\/\/res.cloudinary.com\/practicaldev\/image\/fetch\/s--Fy41jsic--\/c_limit%2Cf_auto%2Cfl_progressive%2Cq_auto%2Cw_880\/https:\/\/mortoray.files.wordpress.com\/2020\/07\/logo.png\" alt=\"\"><\/a><\/p>\n\n\n\n\n<h1>\n  \n  \n  Appendix: Game Manager subscription code\n<\/h1>\n\n<p>This is the <code>listen_game_state<\/code> code called by <code>useEffect<\/code> in <code>useGameState<\/code>. I've removed details about how I connect to my state object, for simplicity. If you'd like a closer examination of that part, let me know.<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight typescript\"><code><span class=\"k\">export<\/span> <span class=\"kd\">type<\/span> <span class=\"nx\">game_state_listener<\/span> <span class=\"o\">=<\/span> <span class=\"p\">(<\/span><span class=\"nx\">gs<\/span><span class=\"p\">:<\/span> <span class=\"nx\">GT<\/span><span class=\"p\">.<\/span><span class=\"nx\">game_state<\/span><span class=\"p\">)<\/span> <span class=\"o\">=&gt;<\/span> <span class=\"k\">void<\/span>\n\n<span class=\"k\">export<\/span> <span class=\"kd\">class<\/span> <span class=\"nx\">GameManager<\/span> <span class=\"k\">implements<\/span> <span class=\"nx\">StateChanged<\/span> <span class=\"p\">{<\/span>\n\n    <span class=\"nx\">gsl_id<\/span> <span class=\"o\">=<\/span> <span class=\"mi\">0<\/span>\n    <span class=\"nx\">game_state_listeners<\/span><span class=\"p\">:<\/span> <span class=\"nb\">Record<\/span><span class=\"o\">&lt;<\/span><span class=\"kr\">number<\/span><span class=\"p\">,<\/span><span class=\"nx\">game_state_listener<\/span><span class=\"o\">&gt;<\/span> <span class=\"o\">=<\/span> <span class=\"p\">{}<\/span>\n    <span class=\"p\">.<\/span>\n    <span class=\"p\">.<\/span>\n    <span class=\"p\">.<\/span>\n    <span class=\"nx\">listen_game_state<\/span><span class=\"p\">(<\/span> <span class=\"nx\">listener<\/span><span class=\"p\">:<\/span> <span class=\"nx\">game_state_listener<\/span> <span class=\"p\">):<\/span> <span class=\"p\">()<\/span><span class=\"o\">=&gt;<\/span><span class=\"k\">void<\/span> <span class=\"p\">{<\/span>\n        <span class=\"k\">this<\/span><span class=\"p\">.<\/span><span class=\"nx\">gsl_id<\/span> <span class=\"o\">+=<\/span> <span class=\"mi\">1<\/span>\n        <span class=\"kd\">const<\/span> <span class=\"nx\">nid<\/span> <span class=\"o\">=<\/span> <span class=\"k\">this<\/span><span class=\"p\">.<\/span><span class=\"nx\">gsl_id<\/span>\n        <span class=\"k\">this<\/span><span class=\"p\">.<\/span><span class=\"nx\">game_state_listeners<\/span><span class=\"p\">[<\/span><span class=\"nx\">nid<\/span><span class=\"p\">]<\/span> <span class=\"o\">=<\/span> <span class=\"nx\">listener<\/span>\n\n        <span class=\"k\">return<\/span> <span class=\"p\">()<\/span> <span class=\"o\">=&gt;<\/span> <span class=\"p\">{<\/span>\n            <span class=\"k\">delete<\/span> <span class=\"k\">this<\/span><span class=\"p\">.<\/span><span class=\"nx\">game_state_listeners<\/span><span class=\"p\">[<\/span><span class=\"nx\">nid<\/span><span class=\"p\">]<\/span>\n        <span class=\"p\">}<\/span>\n    <span class=\"p\">}<\/span>\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<p>Subscription queues needn't be complex.  On updates to the game state, the function below is called (part of the <code>StateChanged interface<\/code>).<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight typescript\"><code>    <span class=\"nx\">game_state_changed<\/span><span class=\"p\">(<\/span><span class=\"nx\">game_state<\/span><span class=\"p\">)<\/span> <span class=\"p\">{<\/span>\n        <span class=\"k\">if<\/span><span class=\"p\">(<\/span> <span class=\"k\">this<\/span><span class=\"p\">.<\/span><span class=\"nx\">set_game_store<\/span> <span class=\"p\">)<\/span> <span class=\"p\">{<\/span>\n            <span class=\"k\">this<\/span><span class=\"p\">.<\/span><span class=\"nx\">set_game_store<\/span><span class=\"p\">(<\/span><span class=\"nx\">game_state<\/span><span class=\"p\">)<\/span>\n        <span class=\"p\">}<\/span>\n\n        <span class=\"k\">for<\/span> <span class=\"p\">(<\/span><span class=\"kd\">const<\/span> <span class=\"nx\">listener<\/span> <span class=\"k\">of<\/span> <span class=\"nb\">Object<\/span><span class=\"p\">.<\/span><span class=\"nx\">values<\/span><span class=\"p\">(<\/span><span class=\"k\">this<\/span><span class=\"p\">.<\/span><span class=\"nx\">game_state_listeners<\/span><span class=\"p\">))<\/span> <span class=\"p\">{<\/span>\n            <span class=\"nx\">listener<\/span><span class=\"p\">(<\/span><span class=\"nx\">game_state<\/span><span class=\"p\">)<\/span>\n        <span class=\"p\">}<\/span>\n    <span class=\"p\">}<\/span>\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<p>The first line goes back to the <code>game_manager.watch_state(set_game_state)<\/code> call at the start of this article. It's what updates the context storing the game state.<\/p>\n\n<p>The loop is what tells all the <code>useGameState<\/code> listeners that something has changed.<\/p>\n\n","category":["coding","react","games","architecture"]},{"title":"I Wrote an Online Escape Game","pubDate":"Fri, 31 Jul 2020 18:35:28 +0000","link":"https:\/\/dev.to\/mortoray\/i-wrote-an-online-escape-game-2ddh","guid":"https:\/\/dev.to\/mortoray\/i-wrote-an-online-escape-game-2ddh","description":"<p>I\u2019m an escape room enthusiast, some may say addict, and for the past few months I\u2019ve been missing it. A friend of mine, a true addict with over 500 rooms to his name, started organizing online competitions. After playing a few of the online games, I thought, \u201cI want to build my own.\u201d <\/p>\n\n<p>So for that past couple of months I\u2019ve been writing <a href=\"https:\/\/edaqa.link\/EdaqasRoom-Prototype-1\">an online escape game<\/a> \u2014 which you could say is a web puzzle game, but with the exciting flare of escape! It\u2019s suitably called \u201cPrototype\u201d. I assumed that name would let me get away with some rough edges. This will be an evolving project, but the first installment is a success.<\/p>\n\n<p>I\u2019m proud of my game. I want to tell you how I made it.<\/p>\n\n<p><a href=\"https:\/\/edaqa.link\/EdaqasRoom-Prototype-3\"><br>\n<img src=\"https:\/\/res.cloudinary.com\/practicaldev\/image\/fetch\/s--Fy41jsic--\/c_limit%2Cf_auto%2Cfl_progressive%2Cq_auto%2Cw_880\/https:\/\/mortoray.files.wordpress.com\/2020\/07\/logo.png\" alt=\"\"><\/a><\/p>\n\n<h1>\n  \n  \n  Technology for the user\n<\/h1>\n\n<p>I had a few major goals for the game. These sit somewhere in the spectrum between user epics and use cases.<\/p>\n\n<p>-Painless experience <a href=\"https:\/\/mortoray.com\/2019\/02\/05\/the-user\/\">for the user<\/a>: I wanted it all in the browser. These types of games are relatively short, and needing to download something would be a pain.<br>\n-A multi-player team experience: Real rooms admit teams of 2-6 or more players, and I wanted my game to allow the same. Additionally, a certain world crisis is an excellent motivator for remote team play.<br>\n-Painless registration: Beyond paying, I didn\u2019t want any registration at all. This bugged me about many other games. Just let people play as quickly as possible.<\/p>\n\n<p>Obviously clever puzzles and a fun experience were paramount, but it\u2019s harder to quantify those directly. Those would be the product goals, and I felt the above points were critical to supporting those goals.<\/p>\n\n<p>Given these requirements, I set out to write my own engine, as I saw nothing that would come close to what I want. I was picky with my game, not letting it get away with anything I\u2019d complain about in other games. Naturally, a few priorities chipped a few notches in that plan.<\/p>\n\n<p>Overall, I achieved those goals. Let me know where I should elaborate \u2014 priorities again, I don\u2019t want to be writing blindly about everything!<\/p>\n\n<h1>\n  \n  \n  A puzzling web stack\n<\/h1>\n\n<p>A design had been floating in my head for a while before I set down any code. There was some trial and error, but the architecture was stable from the start, with only a few deviations in method.<\/p>\n\n<p>Here are some major pieces of the stack.<\/p>\n\n<p>-<a href=\"https:\/\/reactjs.org\/\">React<\/a>: Just React. No optional modules, no plugins, nothing. The core of React provided what I needed. Since I had a state machine, there was no need for something like Redux.<br>\n-<a href=\"https:\/\/www.python.org\/\">Python<\/a> and <a href=\"https:\/\/flask.palletsprojects.com\/\">Flask<\/a>: The server components, and game processing, are written in Python using Flask, with <a href=\"https:\/\/flask-socketio.readthedocs.io\/\">Flask-SocketIO<\/a>, with <a href=\"http:\/\/eventlet.net\/\">Eventlet<\/a> (always so unavoidable many layers here).<br>\n-<a href=\"https:\/\/redis.io\/\">Redis<\/a>: A small, but essential part to coordinate the multi-player actions.<br>\n-SVG: I\u2019m listing this as it\u2019s a key part of the engine. Everything is based on SVG working well in the browsers. It was a major trouble point, yet surprisingly rewarding.<\/p>\n\n<p>There are also the typical web server bits, using Jinja templates, talking to Mongo, Paypal\u2026 Zzz. Yeah, I\u2019ll mention these bits more, but I suspect there\u2019s nothing novel here.<\/p>\n\n<p>Until late in the project, this was a mass of wiggling bits! I had many stressful days trying to juggle tech in my head. Getting something working was my primary goal, and I did that in stages. Now, as I write more games, I\u2019ll keep refining the stack, but there won\u2019t likely be any major architectural changes.<\/p>\n\n<h1>\n  \n  \n  Languages are what I do\n<\/h1>\n\n<p>As a good friend of mine said, \u201cNo <a href=\"https:\/\/edaqa.com\/\">Edaqa<\/a> project would be complete unless there\u2019s a new language.\u201d I\u2019m too blinded by the beauty of languages to even catch a hint of criticism there.<\/p>\n\n<p>The most important question of the technology is: did I want to write a game engine? The answer is a resounding \u201cno!\u201d I wrote the engine because I wanted to achieve my primary goals, saw nothing else that fit, and knew an engine was <a href=\"https:\/\/mortoray.com\/topics\/writing-a-ui-engine\/\">within reach<\/a>. I wanted to design games. And I wanted to not be overly burdened while designing.<\/p>\n\n<p>Thus there\u2019s a domain-specific language for the games. It\u2019s a high-level <a href=\"https:\/\/mortoray.com\/2017\/07\/14\/what-is-declarative-programming\/\">declarative language<\/a>. I fully expect that long-term I\u2019ll write other engines for it. My goal was to keep the game logic clean, without being bound to the engine. I want to write games and ensure that long-term I can maintain those games.<\/p>\n\n<p>I\u2019ll be happy to show you how the language works, what the preprocessor does, and how the game handles the code.<\/p>\n\n<h1>\n  \n  \n  As they say, \u201cAsk me anything\u201d\n<\/h1>\n\n<p>I hit a lot of knowledge pockets and defects on this project. Not everything I did is obvious \u2014 and I hesitate to say some illogical bits remain. But I\u2019m happy to talk about all of it.<\/p>\n\n<p>Let me know what interests you the most, and I\u2019ll answer what I can, providing more writeups where necessary. And if you like puzzles, or escape rooms, I invite you to <a href=\"https:\/\/edaqa.link\/EdaqasRoom-Prototype-2\">play the game<\/a>.<\/p>\n\n\n\n\n<p><em>I encourage you to try the game, <a href=\"https:\/\/edaqa.link\/EdaqasRoom-Prototype-3\">Prototype: A Game Master is Needed<\/a>. It\u2019s an escape game I wrote, and have lots to say about.<\/em><\/p>\n\n","category":["webdev","svg","game","programming"]},{"title":"How I Evaluate You in a Code Interview","pubDate":"Wed, 30 Oct 2019 10:46:36 +0000","link":"https:\/\/dev.to\/mortoray\/how-i-evaluate-you-in-a-code-interview-42hh","guid":"https:\/\/dev.to\/mortoray\/how-i-evaluate-you-in-a-code-interview-42hh","description":"<p>Many of my interview candidates do poorly as they don\u2019t understand how I evaluate them. While other interviewers\u2019 criteria vary, there are common themes. Knowing the basics helps you judge yourself and improve your technique.<\/p>\n\n<p>At <a href=\"https:\/\/iio.sh\/r\/Yxu7\">interviewing.io<\/a>, I evaluate people on three categories. I think this fairly represents how various companies will rate you. Even if not explicitly considered, they\u2019ll play a role in your overall evaluation.<\/p>\n\n<ul>\n<li> problem solving<\/li>\n<li> technical skills<\/li>\n<li> communication<\/li>\n<\/ul>\n\n<p>I\u2019m asked to give a rating to each one, but they are intimately linked together. A candidate who lacks one ability will also appear to lack the others. Let\u2019s take a look at each point here, trying to isolate its specifics.<\/p>\n\n<h1>\n  \n  \n  Problem Solving\n<\/h1>\n\n<p>Problem solving is the part abstract from writing code. I want to understand your process, and not just the result. This requires that you <a href=\"https:\/\/interview.codes\/success\/listen-carefully-to-your-interviewer\/\">know what the problem is<\/a>, and what I\u2019ll accept as a valid solution. Problem solving is a lot about defining constraints and finding ambiguities, or uncertainties, and resolving them.<\/p>\n\n<p>The challenge varies depending on the question. My <a href=\"https:\/\/interview.codes\/success\/interview-question-a-two-player-card-game\/\">card game<\/a> question requires you to write a small game simulation. I want to see how you approach this problem. I want you to ask questions. What exactly is a \u201ccard\u201d, and how will you represent it? What does \u201cdeal\u201d concretely mean?<\/p>\n\n<p>Problem solving is about taking high-level requirements and translating them into concrete steps. If you\u2019re working on a project, this would be the phase where you write user stories and user journeys. You don\u2019t have to be this formal in an interview, but I want to see that you\u2019ve understood the requirements.  Tell me what you\u2019re thinking and write the key points.<\/p>\n\n<p>Show the inputs and outputs to the problem you\u2019re solving. State whether you\u2019re transforming data, or implementing a process. Describe how to break down the large problem into smaller ones. For an algorithm, mention related algorithms, and tell me how you\u2019d adapt them to this new situation.<\/p>\n\n<p>Identify what parts you recognize and how it\u2019s analogous to something you\u2019ve done before. If there\u2019s a whiteboard, sketch out flow,  data sets, and anything else in your head. Especially if you\u2019re stuck, I want to know how you\u2019re solving the problem. Never fall into a <a href=\"https:\/\/interview.codes\/success\/how-to-write-javascript\/\">blank silent stare<\/a>. I\u2019m there to help you, but I can\u2019t do that if I can\u2019t follow your process.<\/p>\n\n<p>And definitely don\u2019t treat problem solving as a phase. I don\u2019t expect you to come up with the perfect solution at the start of an interview. I want to see you take an idea and run with it. It\u2019ll likely have problems and require corrections. I\u2019ll even change the requirements on you, or criticize your approach. I want you to accept change and adapt.<\/p>\n\n<blockquote>\n<p>\ud83d\udcad Your ability to solve problems depends on your technical skills. While problem solving is not a tangible skill, your designs are based on the real-world limitations of computers. It\u2019s not possible to completely separate problem solving and technical ability. <\/p>\n<\/blockquote>\n\n<h1>\n  \n  \n  Technical Skills\n<\/h1>\n\n<p>There are two major aspects of technical skills that I consider. The first is your experience in coding idioms and wisdom. Do you know what is possible and how to go about coding it? The second is specific language ability. Once you\u2019ve said what you want to do, how well can you implement that in the language you\u2019re using.<\/p>\n\n<p>The first point is about your general coding knowledge. Are you aware of data structures, program flow, and all the things programming languages are capable of doing? This knowledge influences your problem solving ability, as this is your toolbox. Whether your design will work depends on whether the tools are available.<\/p>\n\n<p>I evaluate the candidate here on how much they hesitate, or how quickly they work. I\u2019m not judging the speed itself, but as an indicator of one\u2019s proficiency with the tools. I\u2019m also listening to how you speak, whether your voice carries confidence or is it an inquisitive tone seeking affirmation.<\/p>\n\n<p>The second aspect, knowing the syntax and semantics of a particular language, is vitally linked to the first. How you express yourself in code is my primary window into understanding your general coding knowledge. If you\u2019re struggling to write a loop, I have to decide whether it\u2019s because you don\u2019t know the language, or you\u2019re uncertain of how a loop applies to the problem.<\/p>\n\n<p>I\u2019m looking at how smoothly you translate your ideas into code. For example, from my card game question, there is a phase of dealing the cards. You must split the deck into two stacks, one for each player. Whether you\u2019ve chosen a loop, or a high-level splitting function, you should be able to write the code. Writing the wrong code, forgetting how a loop works, using the wrong splitting syntax, all reflect negatively on the candidate.<\/p>\n\n<p>I don\u2019t punish people for making mistakes. They happen. However, a pattern of similar mistakes, or outright repetition of previous errors, shows a lack of knowledge. If I spot a mistake I may ignore it, or I may ask you about the code. Being able to recognize mistakes reflects positively on your abilities.<\/p>\n\n<p>Languages have a lot of features and I\u2019m looking for how well you use them. Are you using enumerations and constants? Can you create a structure to encapsulate values? Are you correctly passing variables by value or reference? Do you use the standard error mechanisms?<\/p>\n\n<p>I don\u2019t expect you to know everything, but I expect you to know all <a href=\"https:\/\/interview.codes\/reference\/programming-language-preparation-for-a-coding-interview\/\">the basics<\/a>. For individuals claiming to be experienced in the language I\u2019ll be looking for common idioms. For example, in Python I look at the use of list comprehensions and dictionaries. In C++ I\u2019ll look for smart pointers and possibly lambdas. If the code is absent all the common language idioms, then it looks like the candidate is inexperienced in that language.<\/p>\n\n<blockquote>\n<p>\ud83d\udcad When I conduct general interviews I don\u2019t put a strong emphasis on language knowledge. Your code is one way that helps me evaluate you. However, if you\u2019re interviewing for a specific position, that requires specific language knowledge, your language skills will be weighted more heavily.<\/p>\n<\/blockquote>\n\n<h1>\n  \n  \n  Communication\n<\/h1>\n\n<p>Voicing your thoughts correctly is the best way to ensure I properly evaluate you. Candidates that cannot do this perform worse than others. Not only does speaking well improve my evaluation of your communication skills, it also ensures the interview goes smoother and you finish the code on time.<\/p>\n\n<p>Talking is your buffer to mistakes in your code. If you just write code, then my only view on your abilities is through that code. If you tell me why you are writing the code, then I can understand your approach even if the code turns out to be wrong. <\/p>\n\n<p>Which also means it\u2019s pointless to read the code aloud. I can see the code and read it myself. Your words should express thoughts that aren\u2019t explicit in the code. Many candidates recognize the need to talk, but I feel are unaware of what they should talk about. Thus they end up reading the code. Sharing your thoughts is something you can learn, but it takes practice.<\/p>\n\n<p>Pay attention to programming vocabulary. You should know how to talk about software in industry terms. This reveals a lot about your experience and your attention to detail. I want to hear about concrete things, like instance variables, constant values, and calling functions with arguments. I want to hear you talk about popping items off a stack and enumerating the elements of a map. I don\u2019t want to hear about \u201cdoing something with this thing here\u201d.<\/p>\n\n<p>I also don\u2019t want to hear buzzwords. Don\u2019t use terms you don\u2019t know and don\u2019t claim to know words that you don\u2019t. Dishonesty is a major red flag in an interview. Don\u2019t think you can get away with it. I\u2019m good at spotting bullshitters, as are most people working in HR and many programmers who have done a lot of hiring.<\/p>\n\n<p>Don\u2019t worry about not knowing everything. I tailor my expectations based on your level of experience. Junior programmers have not been exposed to many concepts. Be open, honest, and inquisitive. However, I expect a lot from somebody with ten or more years of experience.<\/p>\n\n<p>Interviews are interactive and communication ensures we\u2019re both on the same track. A failure to communicate generally results in a misrepresentation of your problem solving abilities, and general programming knowledge. And the more I know what you are thinking the more I can help you towards solving the problem.<\/p>\n\n<h1>\n  \n  \n  Conclusion\n<\/h1>\n\n<p>Generally it\u2019s a lack of communication skills or problem solving that sinks a candidate. I can\u2019t imagine a company that wants to hire somebody with either skill lacking. Okay, sure there are some cases where specific, and uncommon, technical knowledge is required. But even then, you\u2019ll be competing with other candidates.<\/p>\n\n<p>There are also individuals who can write basic code, but not apply their knowledge to new situations. They appear to have learned one particular approach without understanding any of the concepts behind it. I can\u2019t always say how correct my impression is, as I go on what I see in the interview. This makes it imperative to show all these abilities.<\/p>\n\n<p>You may be wondering how to demonstrate these skills. Obviously, to improve any one of them requires practice and study. But to show them all in an interview requires balance. Don\u2019t keep your head stuck in the code. Be vocal about what your are thinking, this is the key way to reveal your knowledge and show how you\u2019re solving problems. Don\u2019t be afraid to show limitations in your knowledge, instead be open and show how you\u2019re working around it. Talk to the interviewer. Sketch things on the whiteboard. <\/p>\n\n<p>Be aware you are being evaluated, and on three primary things: problem solving, technical skills, and communication.<\/p>\n\n","category":["career","programming","interview"]},{"title":"Merge Sort Quick Reference","pubDate":"Thu, 17 Oct 2019 07:35:59 +0000","link":"https:\/\/dev.to\/mortoray\/merge-sort-quick-reference-5gl5","guid":"https:\/\/dev.to\/mortoray\/merge-sort-quick-reference-5gl5","description":"<p>Merge sort is a useful algorithm for sorting a list of items. It's an efficient and predictable algorithm. Inevitably it'll come up during your <a href=\"https:\/\/interview.codes\/\">interviews<\/a>, either you'll be asked directly, or you'll find it useful to solve a code challenge. This is a quick reference for merge sort.<\/p>\n\n<h1>\n  \n  \n  Algorithm Design\n<\/h1>\n\n<p>Merge sort is a recursive algorithm that takes an unsorted list and produces a sorted version of it.<\/p>\n\n<p>It is a stable sorting algorithm.<\/p>\n\n<p>The divide-and-conquer algorithm splits the input in half, sorts both sides, and then combines the result.<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight plaintext\"><code>Merge-Sort( input )\n    if length( input ) &lt; 1\n        return input\n\n    left-half, right-half = divide-in-half( input )\n    left = Merge-Sort( left-half )\n    right = Merge-Sort( right-half )\n\n    out = []\n    while not empty left or not empty right\n        if front( left ) &lt; front( right )\n            out.append( pop-front(left) )\n        else\n            out.append( pop-front(right) )\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<p>Refer to <a href=\"https:\/\/github.com\/mortoray\/interview.codes\/blob\/master\/merge-sort\/main.py\">sample code in Python<\/a>.<\/p>\n\n<h1>\n  \n  \n  Algorithm Complexity\n<\/h1>\n\n<p>The algorithm has a stack depth of <code>log N<\/code>. This is how deeply the recursive calls to <code>Merge-Sort<\/code> are nested. At each of these levels, it has half the input size, but the call is made twice. So effectively at each level, across all calls, the whole input is operated on. This is useful for the complexity.<\/p>\n\n<p>The merge sort algorithm's complexity is not dependent on the input. The worst-case complexity is the same as the best-case complexity. It always performs the same number of comparisons and makes the same number of copies.<\/p>\n\n<h2>\n  \n  \n  <strong>Time:<\/strong> Comparisons\n<\/h2>\n\n<p>The number of comparisons it the number of times two items are compared with each other. This is typically done using only the less than <code>&lt;<\/code> comparison.<\/p>\n\n<p>At each of the <code>log n<\/code> recursive levels the recombining requires <code>n<\/code> comparisons. This results in <code>\u03b8(n \u00b7 log n)<\/code> comparisons.<\/p>\n\n<h2>\n  \n  \n  <strong>Time:<\/strong> Copies\n<\/h2>\n\n<p>The number of copies is the number of times an item is copied to a new location, either from the input or within the output.<\/p>\n\n<p>At each of the <code>log n<\/code> recursive levels all of <code>n<\/code> items are copied into new output arrays during recombining. This results in <code>\u03b8(n \u00b7 log n)<\/code> copies.<\/p>\n\n<p>The initial splitting may also do the same amount of copying. This does not change the upper bound, though in practical terms doubles the copying time. This is language dependent, it's possible to use a view on the input to avoid copying during the division stage.<\/p>\n\n<h2>\n  \n  \n  Space\n<\/h2>\n\n<p>Each level of the recursion requires a copy of the input, as each level needs to combine the elements into new lists. This requires <code>\u03b8(n \u00b7 log n)<\/code> space.<\/p>\n\n<h1>\n  \n  \n  Notes\n<\/h1>\n\n<p>Merge sort does not require random access to the elements, making it suitable for multiple container types, such as linked lists and arrays.<\/p>\n\n<p>It allows for external sorting, meaning not all the data needs to be in memory at the same time.<\/p>\n\n<blockquote>\n<p>View my <a href=\"https:\/\/edaqa.link\/MergeSort-IC\">video class<\/a> that covers the design, complexity, and code of the algorithm.<\/p>\n<\/blockquote>\n\n","category":["coding","interview","computerscience","tutorial"]},{"title":"Insertion Sort Quick Reference","pubDate":"Tue, 01 Oct 2019 15:03:47 +0000","link":"https:\/\/dev.to\/mortoray\/insertion-sort-quick-reference-47kb","guid":"https:\/\/dev.to\/mortoray\/insertion-sort-quick-reference-47kb","description":"<p>Insertion sort is a commonly taught algorithm for sorting a list of items. Though not commonly used, as it's not efficient, it may come up during <a href=\"https:\/\/interview.codes\/\">interviews<\/a>. It also uses concepts that might be beneficial when writing other algorithms. This is a quick reference for insertion sort.<\/p>\n\n<h1>\n  \n  \n  Algorithm Design\n<\/h1>\n\n<p>Insertion sort takes an unsorted list and produces a sorted version of it.<\/p>\n\n<p>It is a stable sorting algorithm.<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight plaintext\"><code>Insertion-Sort( input )\n    output = empty list\n    for each item in the input\n        find location item in the output using upper bound (binary search)\n        insert item at location in the output\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<p>Refer to <a href=\"https:\/\/github.com\/mortoray\/interview.codes\/blob\/master\/insertion-sort\/main.py\">sample code in Python<\/a>.<\/p>\n\n<h1>\n  \n  \n  Algorithm Complexity\n<\/h1>\n\n<h2>\n  \n  \n  <strong>Time:<\/strong> Comparisons\n<\/h2>\n\n<p>The number of comparisons it the number of times two items are compared with each other. This is typically done using only the less than <code>&lt;<\/code> comparison.<\/p>\n\n<p>For each item in the list <code>n<\/code>, the algorithm searches for the correct location using binary search <code>O(log n)<\/code>. This puts an upper bound on the number of comparisons at <code>O(n \u00b7 log n)<\/code>.<\/p>\n\n<h2>\n  \n  \n  <strong>Time:<\/strong> Copies\n<\/h2>\n\n<p>The number of copies is the number of times an item is copied to a new location, either from the input or within the output.<\/p>\n\n<p>All items must be copied from the input to output, making a minimum of <code>\u03a9(n)<\/code> operations. If the input is in sorted order, this will be the number of copy operations.<\/p>\n\n<p>In the worst-case scenario the list is sorted in reverse order. In this scenario each insertion into the output will need to move all existing items in the list. For each <code>n<\/code> items, there could be up to <code>n<\/code> copy operations. This is <code>O(n\u00b2)<\/code> copy operations.<\/p>\n\n<h2>\n  \n  \n  Space\n<\/h2>\n\n<p>The algorithm needs to allocate space in the output for each item in the output. It needs no additional space. This is a space complexity of <code>\u03b8(n)<\/code>. <\/p>\n\n<h1>\n  \n  \n  Notes\n<\/h1>\n\n<p>Due to its costly time complexity for copy operations, insertion sort is not typically used to sort a list. It is however an efficient way to insert a limited number of items into an already sorted list.<\/p>\n\n<p>Inserting one item with insertion sort is <code>O(log n)<\/code>, whereas adding an item to a list and resorting is <code>O(n \u00b7 log n)<\/code>.<\/p>\n\n\n\n\n<blockquote>\n<p>View my <a href=\"https:\/\/edaqa.link\/InsertionSort-IC\">video class<\/a> that covers the design, complexity, and code of the algorithm.<\/p>\n<\/blockquote>\n\n","category":["coding","interview","computerscience","tutorial"]}]}}