Parallel computation in the browser with observable webworkers

2019-08-18T00:00:00+00:00

In the previous article</a> we dove deep into the use of webworkers to improve the responsiveness of the main thread by offloading computation into a web worker.

To facilitate the message passing, we used the library observable-webworker</code></a> to give us an easy to use RxJS stream-based API.

This strategy worked very well for that use case, however we aren't quite maximising the compute resources we have available to us.

Web browsers can spin up multiple parallel web workers that will each run on different threads of the operating system. This means that any computation we do in one worker will not slow down the computation of another worker. There is a limit to this however, as not only does the computer itself have a parallelisation limit (logical processor core count) browsers will self-limit the amount of concurrency web pages are allowed to use.

The amount of parallel threads that can be run can be determined by navigator.hardwareConcurrency</code>. This number varies depending on both underlying hardware and browser implementation. For example most desktop browser give the core count, some mobile ones will limit this value, and some browsers (Safari) won't tell you at all.

To work out how many workers we can run in parallel, simply do const workerCount = navigator.hardwareConcurrency - 1</code>. The - 1</code> here is to keep a processor core spared for the main ui thread.

Put simply, with the worker pool strategy, we spin up as many workers as we are allowed, and construct a pool of work. Each worker picks up a task from the pool, executes the computation in on it's own core, then returns the results to the main thread, grabbing the next task or shutting down if there is no more work to be done.

To better explain this concept and how to implement it, we will build a simple application that will take a list of files (using an <input type="file" multiple /></code> element) and return the MD5 hash sum of those files to the main thread. This basic concept can be expanded to many different types of application, and is in itself a fairly useful application as you might want to use this strategy for hashing multiple files for upload to AWS S3 for instance.

Let's start off by outlining exactly what we're trying to achieve. I'll use a little diagram syntax that is loosely based on the RxJS marble testing</a> syntax.</p>

-</code> idle/waiting time</li>
^</code> transferring file to worker thread</li>
*</code> processing file</li>
|</code> hash computed, returning results</li>
</ul>
Take the following diagram:</p>
file-1.txt  ----^*******|
</span></code></pre>
With the key above, we can see that we're showing that for file-1.txt</code> there was some idle time, then the file was picked up, processing ran for some time then the result was returned.</p>
We can use this syntax to show what would happen if we were to compute the hashes for the files one at a time:</p>
file-1.txt  ^*******|
</span>file-2.txt  ---------^***|
</span>file-3.txt  --------------^**********|
</span>file-4.txt  --------------------------^********|
</span>file-5.txt  ------------------------------------^*********|
</span>file-6.txt  -----------------------------------------------^****|
</span>
</span>results     --------1----2------------3--------4----------5-----6
</span></code></pre>
I've simulated the files being different sizes/taking different time to hash by extending the processing time (*</code>) randomly.
The last line shows the results as they come back to the main thread, with the number just being the file number.</p>
So from this diagram we can see that the total time to get the hash results of all 6 files is the individual durations of each file processing all added up. With a worker pool we can of course take advantage of parallel computing and run a number of these processes in parallel:</p>
file-1.txt  ^*******|               # worker-1
</span>file-2.txt  ^***|                   # worker-2
</span>file-3.txt  ^**********|            # worker-3
</span>file-4.txt  -----^********|         # worker-2
</span>file-5.txt  ---------^*********|    # worker-1
</span>file-6.txt  ------------^****|      # worker-3
</span>
</span>results     ----2---1---3-4--6-5
</span></code></pre>
In this diagram I've taken the exact same processing durations from above, but shifted the processing times assuming a parallelism count of three. You can see the dramatic effect this has on the overall processing time as once the first file completes (file-2.txt) the next file (file-4.txt) is immediately picked up for processing. You can also see that even while file-3.txt is still occupying one thread, the other two threads are able to complete their unit of work and pick up a new job. One thing to note for later is that the results are returned as soon as possible, which in this case means out of order. We will need to correlate each unit of work with it's result to marry up the data correctly.</p>
Another way to look at the above scenario is with the same diagram syntax, but considering each worker rather than each file:</p>
worker-1    ^*******|^*********|    # file-1.txt, file-5.txt
</span>worker-2    ^***|^********|         # file-2.txt, file-4.txt
</span>worker-3    ^**********|^****|      # file-3.txt, file-6.txt
</span>
</span>results     ----2---1--3--4-6-5
</span></code></pre>
The important thing to note here is that the workers are never idle - as soon as they're finished processing work they pick up another file for processing.</p>
Now in our example we're only using three workers, but let's say the client machine has 8 logical cores and the browser lets you use them, the diagram would look like the following</p>
file-1.txt  ^*******|               # worker-1
</span>file-2.txt  ^***|                   # worker-2
</span>file-3.txt  ^**********|            # worker-3
</span>file-4.txt  ^********|              # worker-4
</span>file-5.txt  ^*********|             # worker-5
</span>file-6.txt  ^****|                  # worker-6
</span>
</span>results     ----26--1453
</span></code></pre>
Not so interesting, but we can see that the overall time took only as long as the longest file (file-3.txt) and also that we only needed six of the seven available workers.</p>
Okay, we've talked through the theory, let's get practical!</p>
I'm going to use Angular here as it's what I'm most familiar with, but the two libraries that we'll be using (observable-webworker</code> and js-md5</code>) are both framework agnostic, so if you're following along feel free to use the frontend framework you like best.</p>

Let's start with a quick component that can load some files from the file system and print them to the console.</p>
Template: files-hash.component.html</code></p>

<</span>input </span>type</span>="</span>file</span>" </span>multiple (change)</span>="</span>hashFiles($event)</span>" />
</span></code></pre>
Component: files-hash.component.ts</code></p>

import </span>{ </span>Component </span>} </span>from </span>'</span>@angular/core</span>';
</span>
</span>@</span>Component</span>({
</span>  selector: '</span>app-files-hash</span>',
</span>  templateUrl: '</span>./files-hash.component.html</span>',
</span>  styleUrls: ['</span>./files-hash.component.scss</span>'],
</span>})
</span>export class </span>FilesHashComponent </span>{
</span>  </span>public </span>hashFiles</span>(</span>$event</span>): </span>void {
</span>    </span>const </span>files</span>: </span>File[] </span>= </span>Array</span>.</span>from</span>(</span>$event</span>.target.</span>files</span>);
</span>    </span>console</span>.</span>log</span>(</span>files</span>);
</span>  }
</span>}
</span></code></pre>
Running this we will see a single file select and when we select a few files we see them logged in the console in an array. Easy.</p>
Next let's create a webworker that will take a file as argument and return the hash of that file. We will not just return the hash string alone as we will need to correlate which hash corresponds to which file (remember from earlier we will get the results out of order).</p>
First though, we define an interface to manage passing back this correlation of filename and hash:</p>

file-hasher.interface.ts</code></p>
export interface </span>FileHashPayload {
</span>  </span>filename</span>: string;
</span>  </span>hash</span>: string;
</span>}
</span></code></pre>
Nothing exciting here, just an interface to hold the filename and the hash.</p>
Now for the worker. We will need both observable-webworker</code> and js-md5</code> to handle the workers and the hashing respectively.</p>
yarn add -E observable-webworker js-md5</code></p>
Worker: file-hasher.worker.ts</code></p>

import </span>* </span>as </span>md5 </span>from </span>'</span>js-md5</span>';
</span>import </span>{ </span>DoWorkUnit</span>, </span>ObservableWorker </span>} </span>from </span>'</span>observable-webworker</span>';
</span>import </span>{ </span>Observable </span>} </span>from </span>'</span>rxjs</span>';
</span>import </span>{ </span>map </span>} </span>from </span>'</span>rxjs/operators</span>';
</span>import </span>{ </span>FileHashPayload </span>} </span>from </span>'</span>./file-hasher.interface</span>';
</span>
</span>@</span>ObservableWorker</span>()
</span>export class </span>WorkerPoolHashWorker </span>implements </span>DoWorkUnit</span><File, FileHashPayload> {
</span>  </span>public </span>workUnit</span>(</span>file</span>: </span>File</span>): </span>Observable<FileHashPayload> {
</span>    </span>return </span>this</span>.</span>readFileAsArrayBuffer</span>(</span>file</span>).</span>pipe</span>(
</span>      </span>map</span>(</span>arrayBuffer </span>=> </span>({ filename: </span>file</span>.name, hash: </span>md5</span>(</span>arrayBuffer</span>) })),
</span>    );
</span>  }
</span>
</span>  </span>private </span>readFileAsArrayBuffer</span>(</span>blob</span>: </span>Blob</span>): </span>Observable<ArrayBuffer> {
</span>    </span>return </span>new </span>Observable(</span>observer </span>=> </span>{
</span>      </span>if </span>(</span>!</span>(</span>blob </span>instanceof </span>Blob)) {
</span>        </span>observer</span>.</span>error</span>(
</span>          </span>new </span>Error(</span>'</span>`blob` must be an instance of File or Blob.</span>'</span>),
</span>        );
</span>        </span>return</span>;
</span>      }
</span>
</span>      </span>const </span>reader </span>= new </span>FileReader();
</span>
</span>      </span>reader</span>.</span>onerror </span>= </span>err </span>=> </span>observer</span>.</span>error</span>(</span>err</span>);
</span>      </span>reader</span>.</span>onload </span>= () </span>=> </span>observer</span>.</span>next</span>(</span>reader</span>.</span>result </span>as </span>ArrayBuffer);
</span>      </span>reader</span>.</span>onloadend </span>= () </span>=> </span>observer</span>.</span>complete</span>();
</span>
</span>      </span>reader</span>.</span>readAsArrayBuffer</span>(</span>blob</span>);
</span>
</span>      </span>return </span>() </span>=> </span>reader</span>.</span>abort</span>();
</span>    });
</span>  }
</span>}
</span></code></pre>
Ok there's a few things going on here, so let's break them down.</p>

@</span>ObservableWorker</span>()
</span>export class </span>WorkerPoolHashWorker </span>implements </span>DoWorkUnit</span><File, FileHashPayload> {
</span></code></pre>
In the class header we decorate it with @ObservableWorker()</code> to register the class as a worker so that we can communicate with it from the main thread.</p>
We also implement DoWorkUnit<File, FileHashPayload></code> which means that we need to implement a public method that takes a file and returns an observable of FileHashPayload</code>.</p>

public </span>workUnit</span>(</span>file</span>: </span>File</span>): </span>Observable</span><</span>FileHashPayload</span>> {
</span>  return this.readFileAsArrayBuffer(file).pipe(
</span>    </span>map</span>(</span>arrayBuffer </span>=> ({ filename: </span>file</span>.name, hash: </span>md5</span>(</span>arrayBuffer</span>) })),
</span>  );
</span>}
</span></code></pre>
This method is pretty straightforward - we read the file as an array buffer (see the private method defined next for the implementation), then map the resulting array buffer to the FileHashPayload</code> we defined earlier with the hash computed synchronously using the js-md5</code> library.</p>
Note that we're returning a completing observable here. This is critical to the functioning of the observable-webworker</code> library when managing thread pools. It uses the completion notification to determine that the worker has completed that unit of work and is ready for another unit of work. We return observable as it allows us to return multiple events from the worker, which is useful for outputting progress events from long running processes.</p>
Now back to the main thread to actually implement the worker pool.</p>

import </span>{ </span>Component </span>} </span>from </span>'</span>@angular/core</span>';
</span>import </span>{ </span>fromWorkerPool </span>} </span>from </span>'</span>observable-webworker</span>';
</span>import </span>{ </span>FileHashPayload </span>} </span>from </span>'</span>./file-hasher.interface</span>';
</span>
</span>@</span>Component</span>({
</span>  selector: '</span>app-files-hash</span>',
</span>  templateUrl: '</span>./files-hash.component.html</span>',
</span>  styleUrls: ['</span>./files-hash.component.scss</span>'],
</span>})
</span>export class </span>FilesHashComponent </span>{
</span>  </span>public </span>hashFiles</span>(</span>$event</span>): </span>void {
</span>    </span>const </span>files</span>: </span>File[] </span>= </span>Array</span>.</span>from</span>(</span>$event</span>.target.</span>files</span>);
</span>    </span>console</span>.</span>log</span>(</span>`</span>files</span>`</span>, </span>files</span>);
</span>    </span>fromWorkerPool</span><File, FileHashPayload>(
</span>      </span>() </span>=> </span>new </span>Worker(</span>`</span>./file-hasher.worker.ts</span>`</span>, { type: </span>'</span>module</span>' </span>}),
</span>      </span>files</span>,
</span>    ).</span>subscribe</span>(</span>(</span>hashPayload</span>: </span>FileHashPayload</span>) </span>=> </span>{
</span>      </span>console</span>.</span>log</span>(</span>'</span>Hashed file</span>'</span>, </span>hashPayload</span>.filename, </span>hashPayload</span>.hash);
</span>    });
</span>  }
</span>}
</span></code></pre>
This is quite similar to what we were doing in earlier articles with fromWorker()</code> but this time we use fromWorkerPool()</code> from observable-webworker</code>. Like fromWorker</code> the arguments are a factory to create a worker, and the input itself.</p>
In this example we've used an Array<File></code> as input, however the fromWorkerPool</code> method also takes an Observable<T></code> or an Iterator<T></code>.</p>
Iterators can be very useful with this pattern as you can use their lazy evaluation feature to know when</em> the work has been picked up for processing, not just that it has been queued.</p>
Observable can be useful as you're probably using observable streams anyway, and fromWorkerPool</code> buffers the stream when the input rate exceeds the rate the worker pool can handle the work.</p>

If we now run our application defined above, and select some filesm, we will see something like the following:</p>
files (10) [File, File, File, File, File, File, File, File, File, File]
</span>Hashed file 1-monospondylic-Notelaea-140MB.txt 8b3e48ad838ba4b3024d42fa10591c82
</span>Hashed file 2-safekeeper-unheedful-307MB.txt d1bef08d19b30f471f161c8f5fbf9a8a
</span>Hashed file 4-pseudochromesthesia-laryngeal-162MB.txt d77b94eea8ad01c4e4b804fe9ecd26a2
</span>Hashed file 3-gnatcatcher-incudes-361MB.txt 30c6b4c2ea8b1dd9e5222e8ab9f2119d
</span>Hashed file 5-amacrine-Nance-188MB.txt 7be20804dde038994a2c0e6630643046
</span>Hashed file 8-ankaratrite-dermatoheteroplasty-149MB.txt 72cb1a511a27ac023b9960dbc3959c40
</span>Hashed file 7-enjambed-escutcheon-256MB.txt 1f5039e50bd66b290c56684d8550c6c2
</span>Hashed file 6-idiograph-freckly-393MB.txt bcee52113567c0040a0db2c678dfe3c3
</span>Hashed file 9-strandage-barrelage-179MB.txt 9eab43b111f3c7ff67536d031d06f69b
</span>Hashed file 10-refinish-mellowy-378MB.txt 212374c37a433a7b06105090002297d0
</span></code></pre>
To test I've generated</a> a bunch of random files with the file size in their filename.</p>
From the output we can see that we've achieved the expected outcome that we theorised earlier - the files are appearing in the logs, out of order and the files that are particularly large are ordered towards the end, as we would expect. Success!</strong></p>
Don't just take my word for it though - I've created a far more interactive demo at https://cloudnc.github.io/observable-webworker</a> which uses basically the exact same strategy outlined here, just with a bunch more progress messages to build a pretty timeline:</p>
</p>
To try it out, jump to the "Multiple Worker Pool" section and select some files. Don't worry - no files are sent out of the browser, feel free to check the source to verify or generate some random files like I did. Once you've selected the files, you'll see a timeline graph of the files being processed with whatever concurrency your machine can handle.</p>
If you inspect the sources tab in your browser devtools you will see multiple workers being created to handle the workload, and they will be shut down as soon as there is no further work in the pool.</p>
Additionally, if you select less files than you have concurrency available, you'll only spin up as many workers as files you've selected.</p>

That's all for this article, I hope you've enjoyed it! Next up in the series we will be digging in to the performance penalties of transferring large objects to webworkers and a strategy to mitigate this cost.</p>

The source for this article & demo is available at https://github.com/zakhenry/blog-posts/tree/master/posts/observable-workers-parallel-computation-in-the-browser</a></p>

Observable Web Workers, a deep dive into a realistic use case

2019-06-30T00:00:00+00:00

In the previous article</a> we took a look at the library observable-webworker</code></a> which lets us use familiar observable patterns to construct and manage Web Workers and the communication between the threads. In this article we will develop an application, first without a web worker, then refactor it to use a web worker to demonstrate the power and usefulness of web workers. If you haven't read the previous article, I do recommend you do so first as it goes over all the prerequisites to getting up and running, and the background of what web workers are, which I will skip over here.

This article is a deep dive for advanced developers, so get yourself a beverage and settle in. I'm assuming moderate to advanced knowledge of Typescript and RxJS in this article, so if you're a beginner this may not be for you I'm afraid. Here's a table of contents so you can come back to this article in stints if you like.

Brief</a></li>
Data Flow Plan</a></li>
Algorithm</a></li>
Application</a>

Common functionality</a></li>
Service</a></li>
Component</a></li>
Main Thread Test</a></li>
Worker</a></li>
Worker Thread Test</a></li>
</ul>
</li>
Performance</a></li>
Wrap Up</a></li>
</ul>
Brief</h2>
The application we're going to build has a few requirements, I'll lay these out as acceptance criteria that we can refer to later.</p>

As a user I want to be able to search through text of a variety of novels fetched from Project Gutenberg</a>. My search results should contain my search phrase highlighted within the context of the matching paragraph. When I typo or misspell words I should still get expected results.</p>
</blockquote>
For fun I'll detail the criteria in the BDD Gherkin syntax</a></p>
Feature: Find phrases in books by searching for them
</span>
</span>    Background:
</span>        Given I'm a user on the home page
</span>        And I have selected the book "Alice in Wonderland"
</span>        And I see a free text field in which I can type my search phrase
</span>    
</span>    # basic search
</span>    Scenario
</span>        When I type "we’re all mad here"
</span>        Then I should see the first search result containing the text "‘Oh, you can’t help that,’ said the Cat: ‘we’re all mad here. I’m mad. You’re mad.’"
</span>    
</span>    # multiple result search
</span>    Scenario
</span>        When I type "mad"
</span>        Then I should see search results with "mad" highlighted at least "10" times
</span>    
</span>    # search with typos
</span>    Scenario
</span>        When I type "were all madd her"
</span>        Then I should see the first search result containing the text "‘Oh, you can’t help that,’ said the Cat: ‘we’re all mad here. I’m mad. You’re mad.’"
</span></code></pre>
Additional to these user journey requirements, we will have the following performance requirements:</p>

the user interface must remain responsive at all times</li>
the user gets their search results live with every keystroke</li>
the user can change book and the search immediate begins returning results for their previous search</li>
</ul>
The difficult part to take particular note of in the requirements is the ability to handle typos and misspellings. This explodes the problem complexity as it is not as simple as finding a substring of a paragraph; rather we need to score candidate substrings to find the best match.</p>
Data Flow Plan</h2>
To start with we will put together a simple typescript file proving out the observable flow, then later build it into the application.</p>
// playground/observable-flow.ts
</span>
</span>import </span>{ </span>from</span>, </span>Observable</span>, </span>of </span>} </span>from </span>'</span>rxjs</span>';
</span>import </span>{ </span>scan</span>, </span>shareReplay</span>, </span>switchMap </span>} </span>from </span>'</span>rxjs/operators</span>';
</span>
</span>/**
</span> * First off, we create a quick enum of the books we will use as a demo. Later
</span> * we will update the urls to be correct
</span> */
</span>enum </span>BookChoice {
</span>  </span>ALICE_IN_WONDERLAND </span>= '</span>http://some-url-to-alice-in-wonderland-text</span>',
</span>  </span>SHERLOCK_HOLMES </span>= '</span>http://some-url-to-sherlock-holmes-text</span>',
</span>}
</span>
</span>/**
</span> * This observable will be something like a regular dropdown, emitted whenever
</span> * the user changes their selection.
</span> */
</span>const </span>userBookSelection$ </span>= </span>of</span>(</span>BookChoice</span>.</span>ALICE_IN_WONDERLAND</span>);
</span>
</span>/**
</span> * This observable represents the stream of search terms the user will enter
</span> * into a text box. We `shareReplay(1)` so that subsequent subscribers will get
</span> * the latest value
</span> */
</span>const </span>userSearchTerm$ </span>= </span>of</span>(`</span>we’re all mad here</span>`).</span>pipe</span>(</span>shareReplay</span>(</span>1</span>));
</span>
</span>/**
</span> * This function will be responsible for fetching the content of the book given
</span> * the enum. We're cheating a little by making the enum value the url to fetch.
</span> * For now we will just pretend the url is the content of the book.
</span> * </span>@todo</span> implement
</span> */
</span>function </span>getBookText</span>(</span>bookChoice</span>: BookChoice): Observable<string> {
</span>  </span>return </span>of</span>(</span>bookChoice</span>);
</span>}
</span>
</span>/**
</span> * This function will be responsible for taking the search term and returning
</span> * the stream of paragraphs found, as soon as they are found.
</span> *
</span> * For the purposes of quick testing, we've hardcoded as if two search results
</span> * were found
</span> *
</span> * We will also likely extend this return type in future to handle highlighting
</span> * the search phrase, but for now just the paragraph is sufficient for testing
</span> * </span>@todo</span> implement
</span> */
</span>function </span>getSearchResults</span>(
</span>  </span>searchTerm</span>: string,
</span>  </span>bookText</span>: string,
</span>): Observable<string> {
</span>  </span>return </span>from</span>([
</span>    </span>searchTerm </span>+ '</span> (this will be the first result)</span>',
</span>    </span>searchTerm </span>+ '</span> (this will be the second result)</span>',
</span>  ]);
</span>}
</span>
</span>/**
</span> * Here we take the user selected book stream and pipe it via switchMap to fetch
</span> * the content of the book. We use switchMap() because we want to cancel the
</span> * download of the book if the user switches to a different book to search
</span> * before the book has finished downloading.
</span> *
</span> * Next we again switchMap() the result of the book content to the user search
</span> * term observable so that if the user has changed books, once it is loaded we
</span> * will cancel the processing of the current search term.
</span> *
</span> * Next we pass that stream of search terms to getSearchResults() which will
</span> * itself be returning a stream of search results for that search string.
</span> *
</span> * Finally, we use a scan() operator to collate the stream of search results
</span> * into an array so that we can present all results to the user, not just the
</span> * most recent one
</span> */
</span>const </span>searchResults$ </span>= </span>userBookSelection$</span>.</span>pipe</span>(
</span>  </span>switchMap</span>(</span>selection </span>=> </span>getBookText</span>(</span>selection</span>)),
</span>  </span>switchMap</span>(</span>bookText </span>=> </span>{
</span>    </span>return </span>userSearchTerm$</span>.</span>pipe</span>(
</span>      </span>switchMap</span>(</span>searchTerm </span>=>
</span>        </span>getSearchResults</span>(</span>searchTerm</span>, </span>bookText</span>).</span>pipe</span>(
</span>          </span>scan</span>((</span>searchResults</span>: string[], </span>searchResult</span>) </span>=> </span>{
</span>            </span>return </span>[...</span>searchResults</span>, </span>searchResult</span>];
</span>          }, []),
</span>        ),
</span>      ),
</span>    );
</span>  }),
</span>);
</span>
</span>/**
</span> * Last but not least, to check our logic, we subscribe to the observable and
</span> * bind the console so we see output.
</span> */
</span>searchResults$</span>.</span>subscribe</span>(</span>console</span>.</span>log</span>);
</span>
</span></code></pre>
Alrighty, we've got a concept of what we're trying to build, so what do we get when we run it? I like to use ts-node</code> for these kind of quick tests, so just run npx ts-node --skip-project playground/observable-flow.ts</code> (tip: the --skip-project</code> bit is just because I'm currently in a working directory that has a tsconfig.json file that is not compatible with just running a plain nodejs script.)</p>
Our output of the above file is as follows:</p>

[ 'we’re all mad here (this will be the first result)' ]
</span>[
</span>  'we’re all mad here (this will be the first result)',
</span>  'we’re all mad here (this will be the second result)'
</span>]
</span>
</span></code></pre>
Okay, so this is expected - we first get one result in an array, then we get both results in the array. Success? Well kinda, but we'd really like to put some realism into this script so that we can see that our observables are having the right behavior.</p>
So, we will now edit the script we just wrote to add a bunch of realistic delays and some more logging to see what is really going on:</p>
// playground/observable-flow-test.ts
</span>
</span>import </span>{ </span>from</span>, </span>Observable</span>, </span>of </span>} </span>from </span>'</span>rxjs</span>';
</span>import </span>{
</span>  </span>concatMap</span>,
</span>  </span>delay</span>,
</span>  </span>map</span>,
</span>  </span>pairwise</span>,
</span>  </span>scan</span>,
</span>  </span>shareReplay</span>,
</span>  </span>startWith</span>,
</span>  </span>switchMap</span>,
</span>  </span>timestamp</span>,
</span>} </span>from </span>'</span>rxjs/operators</span>';
</span>
</span>enum </span>BookChoice {
</span>  </span>ALICE_IN_WONDERLAND </span>= '</span>http://alice.text</span>',
</span>  </span>SHERLOCK_HOLMES </span>= '</span>http://sherlock.text</span>',
</span>}
</span>
</span>/**
</span> * This is a nice little custom operator that spaces out observables by a certain amount, this is super handy for
</span> * emulating user events (humans are slooow!)
</span> */
</span>function </span>separateEmissions</span><T>(</span>delayTime</span>: number) {
</span>  </span>return </span>(</span>obs$</span>: Observable<T>): Observable<T> </span>=> </span>{
</span>    </span>return </span>obs$</span>.</span>pipe</span>(
</span>      </span>concatMap</span>((</span>v</span>, </span>i</span>) </span>=> </span>(</span>i </span>=== </span>0 </span>? </span>of</span>(</span>v</span>) : </span>of</span>(</span>v</span>).</span>pipe</span>(</span>delay</span>(</span>delayTime</span>)))),
</span>    );
</span>  };
</span>}
</span>
</span>/**
</span> * For the book selection, we've piped to separateEmissions() with 4000ms
</span> * defined, this means when subscribed the observable will immediately emit
</span> * Alice in Wonderland content, then 4 seconds later emit Sherlock Holmes content.
</span> */
</span>const </span>userBookSelection$ </span>= </span>from</span>([
</span>  </span>BookChoice</span>.</span>ALICE_IN_WONDERLAND</span>,
</span>  </span>BookChoice</span>.</span>SHERLOCK_HOLMES</span>,
</span>]).</span>pipe</span>(</span>separateEmissions</span>(</span>4000</span>));
</span>
</span>/**
</span> * Slightly different strategy for this one - we're
</span> * 1. Piping delayed user book selection to vary the search phrase depending on
</span> * which book is selected
</span> * 2. creating a streams of individual characters
</span> * 3. spacing out the emissions by 100ms (this is the inter-keystroke time)
</span> * 4. using scan to combine the previous characters
</span> * The result is a pretty good simulation of the user typing the phrase at 10
</span> * keys per second
</span> */
</span>const </span>userSearchTerm$ </span>= </span>userBookSelection$</span>.</span>pipe</span>(
</span>  </span>delay</span>(</span>200</span>),
</span>  </span>switchMap</span>(</span>book </span>=> </span>{
</span>    </span>const </span>searchPhrase </span>=
</span>      </span>book </span>=== </span>BookChoice</span>.</span>ALICE_IN_WONDERLAND
</span>        ? `</span>we’re all mad here</span>`
</span>        : `</span>nothing more deceptive than an obvious fact</span>`;
</span>
</span>    </span>return </span>from</span>(</span>searchPhrase</span>).</span>pipe</span>(
</span>      </span>separateEmissions</span>(</span>100</span>),
</span>      </span>scan</span>((</span>out</span>, </span>char</span>) </span>=> </span>out </span>+ </span>char</span>, ''),
</span>    );
</span>  }),
</span>  </span>shareReplay</span>(</span>1</span>),
</span>);
</span>
</span>/**
</span> * Here, we're guessing it will take about 200ms to download the book. We've
</span> * also put in a console.log so we can make sure we're not going to try download
</span> * the book on every keystroke!
</span> * </span>@param </span>bookChoice
</span> */
</span>function </span>getBookText</span>(</span>bookChoice</span>: BookChoice): Observable<string> {
</span>  </span>console</span>.</span>log</span>(`</span>getBookText called (${</span>bookChoice</span>})</span>`);
</span>  </span>return </span>of</span>(</span>bookChoice</span>).</span>pipe</span>(</span>delay</span>(</span>200</span>));
</span>}
</span>
</span>/**
</span> * With this function we're saying that the search takes (20 milliseconds * the
</span> * length of the search string)
</span> * This is actually totally unrealistic, but the linear variability will help
</span> * when understanding the logs
</span> */
</span>function </span>getSearchResults</span>(
</span>  </span>searchTerm</span>: string,
</span>  </span>bookText</span>: string,
</span>): Observable<string> {
</span>  </span>return </span>from</span>(['</span> (first search result)</span>', '</span> (second search result)</span>']).</span>pipe</span>(
</span>    </span>map</span>(</span>result </span>=> </span>`</span>${</span>bookText</span>} : ${</span>searchTerm</span>} : ${</span>result</span>}</span>`),
</span>    </span>delay</span>(</span>20 </span>* </span>searchTerm</span>.length),
</span>    </span>separateEmissions</span>(</span>200</span>),
</span>  );
</span>}
</span>
</span>/**
</span> * This is unchanged from before
</span> */
</span>const </span>searchResults$ </span>= </span>userBookSelection$</span>.</span>pipe</span>(
</span>  </span>switchMap</span>(</span>selection </span>=> </span>getBookText</span>(</span>selection</span>)),
</span>  </span>switchMap</span>(</span>bookText </span>=> </span>{
</span>    </span>return </span>userSearchTerm$</span>.</span>pipe</span>(
</span>      </span>switchMap</span>(</span>searchTerm </span>=>
</span>        </span>getSearchResults</span>(</span>searchTerm</span>, </span>bookText</span>).</span>pipe</span>(
</span>          </span>scan</span>((</span>searchResults</span>: string[], </span>searchResult</span>) </span>=> </span>{
</span>            </span>return </span>[...</span>searchResults</span>, </span>searchResult</span>];
</span>          }, []),
</span>        ),
</span>      ),
</span>    );
</span>  }),
</span>);
</span>
</span>/**
</span> * Lastly we'd doing a few tricks to make the output express what happened
</span> * better.
</span> * The combination of timestamp and pairwise gives us a stream of when the
</span> * emission happened and bundles it with the previous one so we can compare
</span> * times to get a time taken value. The startWith(null) just gives us the
</span> * startup time as a baseline.
</span> * Lastly we use our old friend map() to output the data in a nice format for
</span> * the logger.
</span> */
</span>searchResults$
</span>  .</span>pipe</span>(
</span>    </span>startWith</span>(</span>null</span>),
</span>    </span>timestamp</span>(),
</span>    </span>pairwise</span>(),
</span>    </span>map</span>(([</span>before</span>, </span>tsResult</span>], </span>i</span>) </span>=> </span>{
</span>      </span>const </span>timeSinceLast </span>= (</span>tsResult</span>.</span>timestamp </span>- </span>before</span>.</span>timestamp</span>) / </span>1000</span>;
</span>      </span>return </span>`</span>${</span>i</span>} : Search Result: [${</span>tsResult</span>.value.</span>join</span>(
</span>        </span>'</span>, </span>'</span>,
</span>      )}] (+${</span>timeSinceLast</span>} seconds)</span>`;
</span>    }),
</span>  )
</span>  .</span>subscribe</span>(</span>console</span>.</span>log</span>);
</span>
</span></code></pre>


getBookText called (http://alice.text)
</span>0 : Search Result: [http://alice.text : w :  (first search result)] (+0.431 seconds)
</span>1 : Search Result: [http://alice.text : we :  (first search result)] (+0.123 seconds)
</span>2 : Search Result: [http://alice.text : we’ :  (first search result)] (+0.128 seconds)
</span>3 : Search Result: [http://alice.text : we’r :  (first search result)] (+0.121 seconds)
</span>4 : Search Result: [http://alice.text : we’re :  (first search result)] (+0.121 seconds)
</span>5 : Search Result: [http://alice.text : we’re all mad here :  (first search result)] (+1.6 seconds)
</span>6 : Search Result: [http://alice.text : we’re all mad here :  (first search result), http://alice.text : we’re all mad here :  (second search result)] (+0.204 seconds)
</span>getBookText called (http://sherlock.text)
</span>7 : Search Result: [http://sherlock.text : n :  (first search result)] (+1.704 seconds)
</span>8 : Search Result: [http://sherlock.text : no :  (first search result)] (+0.125 seconds)
</span>9 : Search Result: [http://sherlock.text : not :  (first search result)] (+0.124 seconds)
</span>10 : Search Result: [http://sherlock.text : noth :  (first search result)] (+0.125 seconds)
</span>11 : Search Result: [http://sherlock.text : nothi :  (first search result)] (+0.122 seconds)
</span>12 : Search Result: [http://sherlock.text : nothing more deceptive than an obvious fact :  (first search result)] (+4.68 seconds)
</span>13 : Search Result: [http://sherlock.text : nothing more deceptive than an obvious fact :  (first search result), http://sherlock.text : nothing more deceptive than an obvious fact :  (second search result)] (+0.201 seconds)
</span>
</span></code></pre>
Alrighty let's dig into this a bit.</p>
We can see straight away that we fetch the Alice in Wonderland book immediately, and never again - this is perfect.</p>
Next as the phrase begins to be typed we first get one result, then the other is appended to the results, good good.</p>
Later on (the line starting with 5</code>) we can see that the search result slowdown has meant that we're getting results less frequently and they are for longer search phrases than just the next character - this is what we expect as it means that the switchMap</code> is unsubscribing from the search processor function as there is different data to be processed. This is a great opportunity to compare the different behaviors of switchMap()</code>, mergeMap()</code> and exhaustMap()</code>:</p>


If we had chosen mergeMap()</code>, we would see every single search result for every keystroke, but likely all overlapping with each other and would be pretty confusing. Also the overall time would be longer, assuming the CPU was saturated while processing the search.</p>
</li>

If we had chosen exhaustMap()</code>, we would get the exhaustive set of results (hence the name!) in the correct order, however the overall time would be way</em> longer as we had to wait sequentially.</p>
</li>
</ul>
In this case I think switchMap()</code> is the correct behavior as the user is not interested in interim search results before they have finished typing, and we gain efficiency by immediately cancelling computation of irrelevant search results.</p>
Back to the output analysis, in the line marked 7</code> we see we've switched to fetching Sherlock Holmes with the new search phrase. Success!</p>
Okay, we're in a state now where we're pretty confident in the general data flow, let's cover the algorithm we will use to score paragraphs, then later start building the application.</p>
Algorithm</h2>
Given the requirement that our search terms may contain typos and misspellings, the algorithm to find the best paragraph to show the user is non trivial. There are a number of different fuzzy string matching algorithms, each with their own strengths & weaknesses. Given our requirement is about typos and misspellings, not word order mistakes, computing the Levenshtein distance</a> sounds like a great candidate.</p>
Levenshtein distance is the number of changes required to change one string to another. The changes can be either insertions, deletions or substitutions. The distance is simply the minimum count of changes.</p>
As an example the distance between hello</code> and helo</code> is 1</code> because there is one insertion required. Two identical strings will have a distance of 0</code>.</p>
Now our inputs don't quite suit this algorithm as we're comparing a small search phrase with a large paragraph - our distance scoring will mostly indicate the difference in length of the search term and the paragraph. Instead we will modify the algorithm to ignore the start and end of the search string such that we compute the minimum Levenshtein distance of the search string when placed anywhere in the paragraph.</p>
Finally, we will normalize Levenshtein distance against the length of the search string in order to compute a similarity score. This will be used to rank the paragraphs by best match.</p>
In order to not make this article any longer than it already is, I'm going to skip over the implementation, but you can dig in to it yourself at https://github.com/zakhenry/blog-posts/tree/master/posts/observable-workers-deep-dive/src/app/book-search/common/fuzzy-substring.ts</p>
What is important to know is that there is a function fuzzySubstringSimilarity</code>:</p>
// src/app/book-search/common/fuzzy-substring.ts#L70-L73
</span>
</span>export function </span>fuzzySubstringSimilarity</span>(
</span>  </span>needle</span>: string,
</span>  </span>haystack</span>: string,
</span>): FuzzyMatchSimilarity {
</span></code></pre>
which returns FuzzyMatchSimilarity</code>:</p>
// src/app/book-search/common/fuzzy-substring.ts#L1-L9
</span>
</span>export interface </span>FuzzyMatch {
</span>  </span>substringDistance</span>: number;
</span>  </span>startIndex</span>: number;
</span>  </span>endIndex</span>: number;
</span>}
</span>
</span>export interface </span>FuzzyMatchSimilarity </span>extends </span>FuzzyMatch </span>{
</span>  </span>similarityScore</span>: number;
</span>}
</span></code></pre>
We will be using this return type to both score the paragraphs of the novel and return the start and end index of the match so we can highlight it in the output.</p>
Let's test the algorithm against the user story defined above:</p>

import </span>{ </span>fuzzySubstringSimilarity </span>} </span>from </span>'</span>../src/app/book-search/common/fuzzy-substring</span>';
</span>
</span>const </span>similarity </span>= </span>fuzzySubstringSimilarity</span>(
</span>  '</span>were all madd her</span>',
</span>  '</span>‘Oh, you can’t help that,’ said the Cat: ‘we’re all mad here. I’m mad. You’re mad.’</span>',
</span>);
</span>console</span>.</span>log</span>(`</span>Similarity: </span>`, </span>similarity</span>);
</span>
</span></code></pre>
All we're doing here is defining the needle as the search string, and the haystack as the expected matching sentence. If we run this file we get the following:</p>


Similarity:  {
</span>  substringDistance: 2,
</span>  startIndex: 42,
</span>  endIndex: 59,
</span>  similarityScore: 0.8823529411764706
</span>}
</span>
</span></code></pre>
As expected, we see a substringDistance of 2</code> (to get to the expected substring we need to add a '</code> and remove a d</code>, two changes, distance is 2</code>). We can also see that the similarityScore</code> is very high (range is 0-1) so we can expect this search to score highly on this paragraph against other paragraphs.</p>
Okay, we now have a algorithm sorted, and a data flow to manage it. Let's build an app!</p>
Application</h2>
So far we've just been working with single typescript files, to work through our ideas, but now we're going to jump into using a framework as this is much more realistic to our real world problems we hope to solve by understanding this article.</p>
I'm going to use Angular, but if you prefer any of the other awesome frameworks don't fret as this is going to be fairly framework agnostic anyway. If you've made it this far, it's probably safe to assume you have the basics of framework development down pat so I'll gloss over that.</p>
ng new observable-workers-deep-dive
</span></code></pre>
ng generate component book-search
</span></code></pre>
Ok, we've got a basic framework scaffolded, given we're trying to test the difference between using observables directly, and using observables with web workers, as we build out the application we will try to extract the common functionality into one place. That way as many variables as possible are controlled during our performance tests as the worker and main thread strategies will be using the exact same core algorithm code.</p>
Common functionality</h3>
To start with we will build a utility function to handle getting a stream of fuzzy-match scored paragraphs:</p>
// src/app/book-search/common/book-search.utils.ts#L8-L25
</span>
</span>interface </span>SearchMatch {
</span>  </span>paragraph</span>: string;
</span>  </span>paragraphNumber</span>: number;
</span>  </span>searchMatch</span>: FuzzyMatchSimilarity;
</span>}
</span>
</span>function </span>getSearchResults</span>(
</span>  </span>searchTerm</span>: string,
</span>  </span>paragraphs</span>: string[],
</span>): Observable<SearchMatch> {
</span>  </span>return </span>from</span>(</span>paragraphs</span>).</span>pipe</span>(
</span>    </span>observeOn</span>(</span>asyncScheduler</span>),
</span>    </span>map</span>((</span>paragraph</span>, </span>index</span>) </span>=> </span>{
</span>      </span>const </span>searchMatch </span>= </span>fuzzySubstringSimilarity</span>(</span>searchTerm</span>, </span>paragraph</span>);
</span>      </span>return </span>{ </span>searchMatch</span>, </span>paragraph</span>, paragraphNumber: </span>index </span>};
</span>    }),
</span>  );
</span>}
</span></code></pre>
Some very important things to note here. We're converting the array of paragraphs to an observable stream of individual paragraphs with from()</code>.</p>
After this we use observeOn(asyncScheduler)</code> - this is critical</em> to the responsiveness of the application. Basically what this is doing is rescheduling the emissions of the from()</code> observable from being synchronous to asynchronous. This allows subscribers to our stream to disconnect from the paragraph stream without having to compute the fuzzy substring scores for the entire book</strong>. This will allow us to discard partial search results when the search string becomes invalid due to the user typing more characters.</p>
Finally we map()</code> to the computation function, and attach the paragraph index number to the output - we will later use this to work out how far along the computation is as a percentage.</p>
Next up for our common functions is an accumulation function that will</p>

take our stream of search results</li>
collate them into an array</li>
order the array of results by score</li>
take the top ten results</li>
slice up the resulting paragraph into non matching text and matching text</li>
</ol>
// src/app/book-search/common/book-search.utils.ts#L27-L77
</span>
</span>export interface </span>MatchingParagraph {
</span>  </span>before</span>: string;
</span>  </span>match</span>: string;
</span>  </span>after</span>: string;
</span>  </span>score</span>: number;
</span>}
</span>
</span>export interface </span>SearchResults {
</span>  </span>paragraphs</span>: MatchingParagraph[];
</span>  </span>searchedParagraphCount</span>: number;
</span>  </span>paragraphCount</span>: number;
</span>}
</span>
</span>function </span>accumulateResults</span>(</span>paragraphCount</span>: number) {
</span>  </span>return </span>(</span>obs$</span>: Observable<SearchMatch>): Observable<SearchResults> </span>=> </span>{
</span>    </span>return </span>obs$</span>.</span>pipe</span>(
</span>      </span>scan</span>((</span>searchResults</span>: SearchMatch[], </span>searchResult</span>: SearchMatch) </span>=> </span>{
</span>        </span>searchResults</span>.</span>push</span>(</span>searchResult</span>);
</span>        </span>return </span>searchResults</span>;
</span>      }, []),
</span>      </span>startWith</span>([]),
</span>      </span>map</span>(
</span>        (</span>searchMatches</span>: SearchMatch[]): SearchResults </span>=> </span>{
</span>          </span>const </span>last </span>= </span>searchMatches</span>[</span>searchMatches</span>.length - </span>1</span>];
</span>
</span>          </span>return </span>{
</span>            searchedParagraphCount: </span>last </span>? </span>last</span>.</span>paragraphNumber </span>: </span>0</span>,
</span>            </span>paragraphCount</span>,
</span>            paragraphs: </span>searchMatches
</span>              .</span>sort</span>(
</span>                (</span>a</span>, </span>b</span>) </span>=>
</span>                  </span>b</span>.</span>searchMatch</span>.</span>similarityScore </span>- </span>a</span>.</span>searchMatch</span>.</span>similarityScore</span>,
</span>              )
</span>              .</span>slice</span>(</span>0</span>, </span>10</span>)
</span>              .</span>map</span>(({ </span>searchMatch</span>, </span>paragraph </span>}) </span>=> </span>{
</span>                </span>return </span>{
</span>                  score: </span>searchMatch</span>.</span>similarityScore</span>,
</span>                  match: </span>paragraph</span>.</span>substring</span>(
</span>                    </span>searchMatch</span>.</span>startIndex</span>,
</span>                    </span>searchMatch</span>.</span>endIndex</span>,
</span>                  ),
</span>                  before: </span>paragraph</span>.</span>substring</span>(</span>0</span>, </span>searchMatch</span>.</span>startIndex</span>),
</span>                  after: </span>paragraph</span>.</span>substring</span>(</span>searchMatch</span>.</span>endIndex</span>),
</span>                };
</span>              }),
</span>          };
</span>        },
</span>      ),
</span>    );
</span>  };
</span>}
</span></code></pre>
Despite the line count, there is nothing particularly complex in here, some points to note however:</p>

The scan()</code> operator is used so that we get results streamed as they come in, if we wanted to wait for completion of the search we would use reduce()</code> (the rxjs operator, not Array.prototype.reduce</code>)</li>
We use startWith([])</code> so that a subscriber immediately gets an empty result set on subscription</li>
</ul>
Lastly we combine these two functions into a single one that accepts a search term and a book, and returns the accumulated stream of search results.</p>
// src/app/book-search/common/book-search.utils.ts#L79-L87
</span>
</span>export function </span>getAccumulatedSearchResults</span>(
</span>  </span>searchTerm</span>: string,
</span>  </span>bookText</span>: string,
</span>): Observable<SearchResults> {
</span>  </span>const </span>paragraphs </span>= </span>bookText</span>.</span>split</span>('</span>\n\n</span>');
</span>  </span>return </span>getSearchResults</span>(</span>searchTerm</span>, </span>paragraphs</span>).</span>pipe</span>(
</span>    </span>accumulateResults</span>(</span>paragraphs</span>.length),
</span>  );
</span>}
</span></code></pre>
Service</h3>
Ok, basic functions that do the heavy lifting are all sorted, we're pretty much on the home stretch! Let's create a service to wrap this logic that we will later use in our component.</p>
// src/app/book-search/main-thread/book-search.service.ts#L11-L44
</span>
</span>@</span>Injectable</span>({
</span>  providedIn: '</span>root</span>',
</span>})
</span>export class </span>BookSearchService </span>{
</span>  </span>constructor</span>(</span>private </span>http</span>: </span>HttpClient</span>) </span>{}
</span>
</span>  </span>public </span>search</span>(
</span>    </span>bookSelection$</span>: </span>Observable<BookChoice>,
</span>    </span>searchTerm$</span>: </span>Observable<string>,
</span>  </span>): </span>Observable<SearchResults> {
</span>    </span>return </span>this</span>.</span>processSearch</span>(</span>bookSelection$</span>, </span>searchTerm$</span>).</span>pipe</span>(
</span>      </span>auditTime</span>(</span>1000 </span>/ </span>60</span>), </span>// emit results at a maximum of 60fps
</span>      </span>share</span>(),
</span>    );
</span>  }
</span>
</span>  </span>protected </span>processSearch</span>(
</span>    </span>url$</span>: </span>Observable<string>,
</span>    </span>search$</span>: </span>Observable<string>,
</span>  </span>): </span>Observable<SearchResults> {
</span>    </span>const </span>sharedSearchTerm$ </span>= </span>search$</span>.</span>pipe</span>(</span>shareReplay</span>(</span>1</span>));
</span>
</span>    </span>return </span>url$</span>.</span>pipe</span>(
</span>      </span>switchMap</span>(</span>url </span>=> </span>this</span>.</span>http</span>.</span>get</span>(</span>url</span>, { responseType: </span>'</span>text</span>' </span>})),
</span>      </span>switchMap</span>(</span>bookText </span>=> </span>{
</span>        </span>return </span>sharedSearchTerm$</span>.</span>pipe</span>(
</span>          </span>switchMap</span>(</span>searchTerm </span>=> </span>{
</span>            </span>return </span>getAccumulatedSearchResults</span>(</span>searchTerm</span>, </span>bookText</span>);
</span>          }),
</span>        );
</span>      }),
</span>    );
</span>  }
</span>}
</span></code></pre>
With this service, we inject an HttpClient</code> into the constructor; this allows us to fetch the book content. Next is a public method that we will use in our component. It takes two streams:</p>

the first is the BookChoice</code>. Recall from earlier this is a string enum with the values being the URl of the book</li>
the second is the search phrase itself that is used for fuzzy matching against the paragraphs of the book</li>
</ul>
We call out to a local processSearch</code> method which simply takes the url and pipes it to a switchMap()</code> that fetches the content, then in turn pipes to the search term observable and finally we switch on that observable to call out to getAccumulatedSearchResults</code> that we built earlier.</p>
Back in the search()</code> method, we pipe the results from processSearch()</code> and first use the auditTime()</code> operator. This restricts the data output rate to 60 frames per second otherwise we could potentially get tens of thousands of SearchResults</code> per second, which would completely oversaturate the change detection strategy of the framework when we try to display the results in the DOM.</p>
Lastly we add a share()</code> to the search results as we don't want multiple subscribers to trigger computation of the search results more than once.</p>
Next let's define the component logic to connect this service to user input.</p>
Component</h3>
Our component requirements are super simple - it needs to provide a form control for both the book selection and the search term, then output the results of the search.</p>
// src/app/book-search/main-thread/book-search.component.ts#L13-L59
</span>
</span>@</span>Component</span>({
</span>  selector: '</span>app-book-search</span>',
</span>  templateUrl: '</span>./book-search.component.html</span>',
</span>  styleUrls: ['</span>./book-search.component.css</span>'],
</span>  changeDetection: </span>ChangeDetectionStrategy</span>.</span>OnPush</span>,
</span>})
</span>export class </span>BookSearchComponent </span>{
</span>  </span>public </span>componentName </span>= '</span>Main thread search</span>'</span>;
</span>
</span>  </span>public </span>bookChoices </span>= </span>[
</span>    {
</span>      url: </span>BookChoice</span>.</span>ALICE_IN_WONDERLAND</span>,
</span>      name: </span>'</span>Alice in Wonderland</span>'</span>,
</span>    },
</span>    {
</span>      url: </span>BookChoice</span>.</span>SHERLOCK_HOLMES</span>,
</span>      name: </span>'</span>Sherlock Holmes</span>'</span>,
</span>    },
</span>  ];
</span>
</span>  </span>public </span>bookSelectionFormControl </span>= new </span>FormControl(</span>null</span>);
</span>  </span>public </span>userBookSelection$</span>: </span>Observable<BookChoice> </span>= </span>this
</span>    .</span>bookSelectionFormControl</span>.</span>valueChanges</span>;
</span>
</span>  </span>public </span>searchTermFormControl </span>= new </span>FormControl(</span>null</span>);
</span>  </span>public </span>userSearchTerm$</span>: </span>Observable<string> </span>= </span>this</span>.</span>searchTermFormControl
</span>    .</span>valueChanges</span>;
</span>
</span>  </span>private </span>searchResults$</span>: </span>Observable<
</span>    SearchResults
</span>  > </span>= </span>this</span>.</span>bookSearchHandler</span>.</span>search</span>(
</span>    </span>this</span>.</span>userBookSelection$</span>,
</span>    </span>this</span>.</span>userSearchTerm$</span>,
</span>  );
</span>
</span>  </span>public </span>searchResultParagraphs$</span>: </span>Observable<
</span>    MatchingParagraph[]
</span>  > </span>= </span>this</span>.</span>searchResults$</span>.</span>pipe</span>(</span>map</span>(</span>result </span>=> </span>result</span>.</span>paragraphs</span>));
</span>
</span>  </span>public </span>searchResultProgress$</span>: </span>Observable<
</span>    [number, number]
</span>  > </span>= </span>this</span>.</span>searchResults$</span>.</span>pipe</span>(
</span>    </span>map</span>(</span>result </span>=> </span>[</span>result</span>.</span>searchedParagraphCount</span>, </span>result</span>.</span>paragraphCount</span>]),
</span>  );
</span>
</span>  </span>constructor</span>(</span>private </span>bookSearchHandler</span>: </span>BookSearchService</span>) </span>{}
</span>}
</span></code></pre>
Points of interest in this component:</p>

we construct a new FormControl</code> for the book selection, then immediately set up an observable that observes the valueChanges</code> of that control</li>
we do the same thing for the search terms</li>
next we construct the private searchResults$: Observable<SearchResults></code> using the previous two observables, it calls out to the service that we just defined earlier</li>
the last two public members observe the searchResults$</code>, the first extracting the matching paragraphs, the second extracting the search progress information</li>
</ul>
Next let's take a look how this will be displayed in the template:</p>
<!-- src/app/book-search/main-thread/book-search.component.html -->
</span>
</span><</span>h2</span>>{{ componentName }}</</span>h2</span>>
</span>
</span><</span>select </span>[formControl]</span>="</span>bookSelectionFormControl</span>">
</span>  <</span>option </span>*ngFor</span>="</span>let book of bookChoices</span>" </span>[value]</span>="</span>book.url</span>">
</span>    {{ book.name }}
</span>  </</span>option</span>>
</span></</span>select</span>>
</span>
</span><</span>input </span>type</span>="</span>text</span>" </span>[formControl]</span>="</span>searchTermFormControl</span>" />
</span>
</span><</span>span </span>*ngIf</span>="</span>searchResultProgress$ | async as searchResultProgress</span>">
</span>  Progress: {{ searchResultProgress[0] }} / {{ searchResultProgress[1] }} ({{
</span>    searchResultProgress[0] / searchResultProgress[1] | percent
</span>  }})
</span></</span>span</span>>
</span>
</span><</span>ng-container </span>*ngFor</span>="</span>let matchingParagraph of searchResultParagraphs$ | async</span>">
</span>  <</span>hr </span>/>
</span>  <</span>blockquote</span>>
</span>    <</span>span</span>>{{ matchingParagraph.before }}</</span>span</span>>
</span>    <</span>strong</span>>{{ matchingParagraph.match }}</</span>strong</span>>
</span>    <</span>span</span>>{{ matchingParagraph.after }}</</span>span</span>>
</span>    <</span>footer</span>>Score: {{ matchingParagraph.score }}</</span>footer</span>>
</span>  </</span>blockquote</span>>
</span></</span>ng-container</span>>
</span>
</span></code></pre>
Nice simple template, we've got</p>

a basic <select></code> to manage the book selection</li>
then an <input></code> to manage the user search</li>
then a <span></code> that outputs the search result progress</li>
lastly a repeated <blockquote></code> to output the results of the search</li>
</ul>
Note that we're using AsyncPipe</code> to managed all</em> subscriptions - this greatly simplifies the component logic, and allows us to use OnPush</code> change detection strategy as the AsyncPipe</code> manages marking the component to be checked.</p>
At long last, we're there, let us test it.</p>
Main Thread Test</h3>
Ok so when we run the app, we're presented with the two controls. So let's select "Alice and Wonderland" and type that typo-ridden string "were all madd her"</code></p>
</p>
Okay! This is good - we've managed to fetch search results, and the first result is the expected match:</p>
were all madd her
</span> Progress: 900 / 901 (100%)
</span>‘Oh, you can’t help that,’ said the Cat: ‘we’re all mad here. I’m mad. You’re mad.’
</span>Score: 0.8823529411764706
</span></code></pre>
However</strong> the responsiveness of the application was awful - that gif is not stuttering, it really did lock up after typing the first character then all the results appeared after a few seconds.
Fortunately we have the incredible tool that is Web Workers up our sleeves, and we can use one to free up this oversaturated main thread to just do UI stuff, and keep our expensive search result computation in a separate thread.</p>
Worker</h3>
So, let's create a worker</p>
ng generate web-worker book-search
</span></code></pre>
This will put a book-search.worker.ts</code> file into our application:</p>

/// <</span>reference </span>lib</span>="</span>webworker</span>" </span>/>
</span>
</span>addEventListener</span>('</span>message</span>', ({ </span>data </span>}) </span>=> </span>{
</span>  </span>const </span>response </span>= `</span>worker response to ${</span>data</span>}</span>`;
</span>  </span>postMessage</span>(</span>response</span>);
</span>});
</span>
</span></code></pre>
We don't want any of that boilerplate, so we will delete it and implement our own worker version of the processSearch</code> method we created in the service earlier:</p>
// src/app/book-search/worker-thread/book-search.worker.ts
</span>
</span>import </span>{ </span>DoWork</span>, </span>ObservableWorker </span>} </span>from </span>'</span>observable-webworker</span>';
</span>import </span>{ </span>Observable </span>} </span>from </span>'</span>rxjs</span>';
</span>import </span>{ </span>ajax </span>} </span>from </span>'</span>rxjs/ajax</span>';
</span>import </span>{
</span>  </span>distinctUntilChanged</span>,
</span>  </span>map</span>,
</span>  </span>shareReplay</span>,
</span>  </span>switchMap</span>,
</span>} </span>from </span>'</span>rxjs/operators</span>';
</span>import </span>{
</span>  </span>getAccumulatedSearchResults</span>,
</span>  </span>SearchResults</span>,
</span>  </span>WorkerInput</span>,
</span>} </span>from </span>'</span>../common/book-search.utils</span>';
</span>
</span>@</span>ObservableWorker</span>()
</span>export class </span>BookSearchWorker </span>implements </span>DoWork</span><WorkerInput, SearchResults> {
</span>  </span>public </span>work</span>(</span>input$</span>: </span>Observable<WorkerInput></span>): </span>Observable<SearchResults> {
</span>    </span>const </span>url$ </span>= </span>input$</span>.</span>pipe</span>(
</span>      </span>map</span>(</span>(</span>{ </span>url </span>}</span>) </span>=> </span>url</span>),
</span>      </span>distinctUntilChanged</span>(),
</span>    );
</span>
</span>    </span>const </span>searchTerm$ </span>= </span>input$</span>.</span>pipe</span>(
</span>      </span>map</span>(</span>(</span>{ </span>searchTerm </span>}</span>) </span>=> </span>searchTerm</span>),
</span>      </span>distinctUntilChanged</span>(),
</span>      </span>shareReplay</span>(</span>1</span>),
</span>    );
</span>
</span>    </span>return </span>url$</span>.</span>pipe</span>(
</span>      </span>switchMap</span>(</span>url </span>=> </span>ajax</span>({ </span>url</span>, responseType: </span>'</span>text</span>' </span>})),
</span>      </span>map</span>(</span>result </span>=> </span>result</span>.</span>response</span>),
</span>      </span>switchMap</span>(</span>bookText </span>=> </span>{
</span>        </span>return </span>searchTerm$</span>.</span>pipe</span>(
</span>          </span>switchMap</span>(</span>searchTerm </span>=>
</span>            </span>getAccumulatedSearchResults</span>(</span>searchTerm</span>, </span>bookText</span>),
</span>          ),
</span>        );
</span>      }),
</span>    );
</span>  }
</span>}
</span>
</span></code></pre>
This ought to look pretty familiar, as it is really just the processSearch</code> function within a @ObservableWorker()</code>- decorated class from the observable-webworker</code></a> package. The only difference is that the input to the work()</code> method is a single stream of Observable<WorkerInput></code> which we split and observe separately.</p>
From there it is essentially identical, though we do use the rxjs ajax()</code> method rather than HttpClient</code> as the Injector</code> is not available in the worker context.</p>
Now for the main thread class to manage this worker:</p>
// src/app/book-search/worker-thread/book-search-worker.service.ts
</span>
</span>import </span>{ </span>Injectable </span>} </span>from </span>'</span>@angular/core</span>';
</span>import </span>{ </span>fromWorker </span>} </span>from </span>'</span>observable-webworker</span>';
</span>import </span>{ </span>combineLatest</span>, </span>Observable </span>} </span>from </span>'</span>rxjs</span>';
</span>import </span>{ </span>map </span>} </span>from </span>'</span>rxjs/operators</span>';
</span>import </span>{ </span>BookSearchService </span>} </span>from </span>'</span>../main-thread/book-search.service</span>';
</span>import </span>{ </span>SearchResults</span>, </span>WorkerInput </span>} </span>from </span>'</span>../common/book-search.utils</span>';
</span>
</span>@</span>Injectable</span>({
</span>  providedIn: '</span>root</span>',
</span>})
</span>export class </span>BookSearchWorkerService </span>extends </span>BookSearchService </span>{
</span>  </span>protected </span>processSearch</span>(
</span>    </span>url$</span>: </span>Observable<string>,
</span>    </span>search$</span>: </span>Observable<string>,
</span>  </span>): </span>Observable<SearchResults> {
</span>    </span>const </span>input$</span>: </span>Observable<WorkerInput> </span>= </span>combineLatest</span>(</span>url$</span>, </span>search$</span>).</span>pipe</span>(
</span>      </span>map</span>(</span>(</span>[</span>url</span>, </span>searchTerm</span>]</span>) </span>=> </span>({ </span>searchTerm</span>, </span>url </span>})),
</span>    );
</span>
</span>    </span>return </span>fromWorker</span>(
</span>      </span>() </span>=> </span>new </span>Worker(</span>'</span>./book-search.worker</span>'</span>, { type: </span>'</span>module</span>' </span>}),
</span>      </span>input$</span>,
</span>    );
</span>  }
</span>}
</span>
</span></code></pre>
Because we've already implemented the search()</code> logic in BookSearchService</code> we simply extend that class and override the implementation of processSearch()</code> in a new BookSearchWorkerService</code> class.</p>
In a real situation you probably would just replace the processSearch</code> method, but we're going to have two implementations in this article so we can compare them side-by-side later.</p>
Our implementation of processSearch</code> is super simple, we combineLatest</code> the two input streams into the form that the Worker required. Then we use the fromWorker()</code> method from 'observable-webworker'</code> to wrap the construction of a new Worker</code> and pass the input stream as the second argument.</p>
Now, while we're trying to preserve the old behaviour, we need a new component to show off this shiny new worker backed service:</p>
// src/app/book-search/worker-thread/book-search-worker.component.ts
</span>
</span>import </span>{ </span>ChangeDetectionStrategy</span>, </span>Component </span>} </span>from </span>'</span>@angular/core</span>';
</span>import </span>{ </span>BookSearchWorkerService </span>} </span>from </span>'</span>./book-search-worker.service</span>';
</span>import </span>{ </span>BookSearchComponent </span>} </span>from </span>'</span>../main-thread/book-search.component</span>';
</span>import </span>{ </span>BookSearchService </span>} </span>from </span>'</span>../main-thread/book-search.service</span>';
</span>
</span>@</span>Component</span>({
</span>  selector: '</span>app-book-search-worker</span>',
</span>  templateUrl: '</span>../main-thread/book-search.component.html</span>',
</span>  styleUrls: ['</span>../main-thread/book-search.component.css</span>'],
</span>  changeDetection: </span>ChangeDetectionStrategy</span>.</span>OnPush</span>,
</span>  providers: [
</span>    { provide: </span>BookSearchService</span>, useClass: </span>BookSearchWorkerService </span>},
</span>  ],
</span>})
</span>export class </span>BookSearchWorkerComponent </span>extends </span>BookSearchComponent </span>{
</span>  </span>public </span>componentName </span>= '</span>Worker thread search</span>'</span>;
</span>}
</span>
</span></code></pre>
Not much going on here! All we're doing is extending the BookSearchComponent</code>, reusing it's template and styles, and providing our new BookSearchWorkerService</code> instead of the default BookSearchService</code>.</p>
Now in our app.component.html</code> template we can insert this new component:</p>
<!-- src/app/app.component.html -->
</span>
</span><</span>app-book-search</span>></</span>app-book-search</span>>
</span><</span>app-book-search-worker</span>></</span>app-book-search-worker</span>>
</span>
</span></code></pre>
So now we have our two implementations of book search - one in the main thread, the other backed by a worker thread service.</p>
Worker Thread Test</h3>
Let's test the new worker component!</p>
</p>
Wow! How's that for performance?! Now I wouldn't call this the nicest of user experiences, but you can really see the power of web workers in action here.</p>
Also in this demo you can really see the reason we had to auditTime()</code> the search results as the DOM update frequency here is super high. In fact, if you comment out that function, you will find that the web worker implementation performance is significantly worse</strong> as it is emitting search results at such a high frequency that it causes a backpressure of change detection cycles for Angular, making the overall performance actually worse. This was a real takeaway lesson for me, and hopefully you too -</p>
Web Workers can make your computation output so fast that displaying the results in the DOM can become a major overall bottleneck</strong></p>
Visual comparisons are all well and good, but let's properly look into how the performance differs between the two strategies</p>
Performance</h2>
To test the relative performance I will search the phrase "There is nothing more deceptive than an obvious fact"</code> within the Sherlock Holmes book. I'll do it three times using the regular main thread strategy, clearing the input each time, then the same for the web worker strategy.</p>

</p>
From these graphs it is clear the difference using a web worker makes - the main thread (in yellow in this graph) holds at 100% for a full 15 seconds, during which repainting will suffer greatly. With the web worker strategy, the main thread barely peaks at 50% utilisation, and is actually mostly idle for the duration.</p>
It is worth noting at this time that the overall duration is roughly the same, and it would be expected that in some circumstances the worker strategy might be slightly slower overall. This is because the strategy requires spinning up a new thread, and structured copying of the data flowing between threads is not zero-cost.</p>
One other consideration is that we're not fully utilising the power of web workers - the computation job that we're running can easily be broken up into smaller tasks for multiple</em> workers, and processed using a thread pool strategy. This is a topic for another article, and I might just be covering that one next.</p>
Wrap Up</h2>
You've reached the end! This was a bit of a marathon I'm afraid but there was an awful lot of content to cover. In summary, we outlined a real world scenario where users want to robustly do full text search over a novel, we then came up with a data flow strategy, then an algorithm, then built it out into an application, then refactored to use an observable web worker strategy, and finally did some performance metrics to prove the utility of web workers. Phew!</p>
The code is all available on Github</a> so please feel free to clone it and have a play. If you spot any errors or room for improvement please do raise an issue or a pull request, I'm learning too!</p>
Next in this series, I will be either demonstrating how to further improve the performance of this application using thread pools, or implementing a new real world use case that does intensive image manipulation which will demonstrate high performance techniques of passing large data between threads. Let me know if you have a preference!</p>
Thanks again for staying with me here, I hope that this article has been useful to you.</p>

npx embedme README.md </code></pre> Now to begin with you're probably not going to get very useful output; you need to annotate your code blocks such that embedme</code> can find the appropriate file to insert So, let us step through an example: Say I have the following README.md: What you can do with embedme</code> is insert a comment in the first line of the code block: Next, if you now run the following: You will see output from the library indicating where it found code blocks and what it did with them (i.e. in this case, inserted the content from path/to/example-usage.ts</code>). Next, to prove the utility of the library, make a change to your source code import { hello } from 'my-amazing-library'; console.log('An update here!'); hello(); // outputs Hello World! </code></pre> Then rerun embedme</code>: You will see your readme file has updated: So, as you've now seen, we're in a situation where out example code is outside of the README, but with a simple utility we can have it be up to date. Not only that, but we can (and should!) unit test this example code against our library, which will ensure that if we cause breaking changes to our library, we are guaranteed that the documentation will be kept up to date with correct code snippets. Embedme has a number of other features, including... extracting specific lines (//path/to/your/file.ts#L10-L20</code>)</li> --verify</code> flag which doesn't actually write to file, but checks there are not changes (good for CI scripts!)</li> --stdout</code> flag to output the change to be stdout in order to be redirected to another file</li> using markdown comment syntax to define code blocks without being required to insert the comment within the code block. (</code>)</li> and a few other handy features...</li> </ul> I encourage you to view the library for more info - https://github.com/zakhenry/embedme and add it to your automated workflow to get documentation that is both a pleasure to write, and guaranteed to be valid.

Tradeoffs</h2>
Unfortunately due to limitations of the lambda runtime, there are tradeoffs to be aware of, and consider up front if they are disqualifying for your intended use-case.</p>

Time limit</h3>
AWS Lambda has a hard time limit of 15 minutes, which makes this approach unsuited for long term streaming requests. However, due to lambda costing more for long durations anyway this is not a good fit for that kind of workload anyway.</p>

Future work</h2>

Zak Henry

Running gRPC services on AWS Lambda with Rust

Parallel computation in the browser with observable webworkers

TC39 Pipeline proposal comparison - RxJS case study

Observable Web Workers, a deep dive into a realistic use case

Observable Web Workers with Angular - Introduction

Ensuring accuracy of README code snippets

Zak Henry

Running gRPC services on AWS Lambda with Rust

Demo</h2> We are going to take the tonic hello world service, enable grpc-web, add a unit test and convert it to a lambda service, and deploy it to AWS Lambda.</p>

Tradeoffs</h2> Unfortunately due to limitations of the lambda runtime, there are tradeoffs to be aware of, and consider up front if they are disqualifying for your intended use-case.</p>

grpc-web</h3> Mentioned earlier, we are required to downgrade the message transport with the grpc-web protocol as AWS Lambda does not support http/2.</p>

Time limit</h3> AWS Lambda has a hard time limit of 15 minutes, which makes this approach unsuited for long term streaming requests. However, due to lambda costing more for long durations anyway this is not a good fit for that kind of workload anyway.</p>

Cost</h3> The niche this approach fills is explicitly low traffic volumes. With sustained traffic, ECS is cheaper and better. This might change with Lambda Managed</em></sup></p>

Future work</h2>

Wrap up</h2> I hope this new capability finds itself some interesting niches for use in the world. I'm intending to use it as a dirt cheap (likely well inside free tier) way of deploying low traffic APIs.</p>

Parallel computation in the browser with observable webworkers

TC39 Pipeline proposal comparison - RxJS case study

Observable Web Workers, a deep dive into a realistic use case

Brief</h2> The application we're going to build has a few requirements, I'll lay these out as acceptance criteria that we can refer to later.</p>

Data Flow Plan</h2> To start with we will put together a simple typescript file proving out the observable flow, then later build it into the application.</p>

Common functionality</h3> To start with we will build a utility function to handle getting a stream of fuzzy-match scored paragraphs:</p>

Observable Web Workers with Angular - Introduction

Ensuring accuracy of README code snippets

Tradeoffs</h2>
Unfortunately due to limitations of the lambda runtime, there are tradeoffs to be aware of, and consider up front if they are disqualifying for your intended use-case.</p>

Time limit</h3>
AWS Lambda has a hard time limit of 15 minutes, which makes this approach unsuited for long term streaming requests. However, due to lambda costing more for long durations anyway this is not a good fit for that kind of workload anyway.</p>