4 Using Events, Listeners, Timers, and Callbacks in Node.js Node,js provides scalability and performance through its powerfull event-driven model. This chapter focuses on understanding the model and how it differs from traditional threading models used by most webservers. Understanding the event model is critical because it may force you to change the design thinking for your applications. However, the changes will be well worth the improvement in speed that you get using Node,js. This chapter also covers the different methods you use to add work to the Node.js event queue. You can add work using event listeners or timers, or you can schedule work directly. You also learn how to implement events in your own custom modules and objects. Understanding the Node.js Event Model Node.js applications are run in a single-threaded event-driven model. Although Node,js implements a thread pool in the background to do work, the application itself doesn’t have any concept of multiple threads. “Wait, what about performance and scale?” you might ask. At first it may seem counterintuitive, but once you understand the logic behind the Node.js event model it all makes perfect sense. Comparing Event Callbacks and Threaded Models In the traditional threaded web model, a request comes in to the webserver and is assigned to an available thread. Then the handling of work for that request continueson that thread until the request is complete and a response is sent. Figure 4.1 illustrates the threaded model processing two requests, Get File and GetData. The Get File request first opens the file, reads the contents, and then sends the data back in a response. All this occurs in order on the same thread. The GetData request connects to the DB, queries the necessary data, and then sends the data in the response.GetFile [Thread 1—>| GetFile(file) ->| Open(file) ma Read(file) PI Send(file) GetDala} Thread 2—>|GetData(db)|-+{connect(ab)-+] query(db) L+/ Send(data) A A Functions are called ina | linear fashion in order on each thread. = i i t 1 i i i t Figure 4.1 Processing two requests on individual threads using the threaded model The Node.js event model does things differently. Instead of executing all the work for each request on individual threads, work is added to an event queue and then picked up by a single thread running an event loop. The event loop grabs the top item in the event queue, executes it, and then grabs the next item. When executing code that is no longer live or has blocking I/O, instead of calling the function directly, the function is added to the event queue along with a callback that is executed after the function completes. When all events on the Node,js event queue have been executed, the Node application terminates. Figure 4.2 illustrates the way Node,js handles the Get File and Get Data requests. The Get File and Get Data requests are added to the event queue. Node.js first picks up the Ge Le request, executes it, and then completes by adding the ) callback function to the event queue. Next, it picks up the GetData request, executes it, and completes by adding the Connect () callback function to the event queue. This continues until there are no callback functions to be executed. Notice in Figure 4.2 that the events for each thread do not necessarily follow a direct interleaved order. For example, the Connect request takes longer to complete than the Read request, so Send (file) is called before Query (db).Event Queue [serie } > Node,js (GetFile(file), Open(file)) [aetDeta}—>| Applcenion! -— (GetData(db),Connect(db)) (Open({file),Read(file)) (Connect(db),Query(db)) (Read(file),Send(file)) (Send(file),none) » (Query(db),Send(data)) (Send(data) none) iD i When the function is executed, Node.js places the callback in the event queue. The order is based ‘on how quickly functions finish. Figure 4.2 Processing two requests on a single event-driven thread using the Node.js event model Blocking I/O in Node.js The Node,js event model of using the event callbacks is great until you run into the problem of functions that block waiting for I/O. Blocking I/O stops the execution of the current thread and waits for a response before continuing. Some examples of blocking I/O are = Reading a file » Querying a database = Socket request « Accessing a remote service The reason Node,js uses event callbacks is not to have to wait for blocking 1/O. Therefore, any requests that perform blocking I/O are performed on a different thread in the background. Node,js implements a thread pool in the background. When an event that requires blocking I/O is retrieved from the event queue, Node.js retrieves a thread from the thread pool and executes the function there instead of on the main event loop thread. This prevents the blocking I/O from holding up the rest of the events in the event queue.The function executed on the blocking thread can still add events back to the event queue to be processed, For example, a database query call is typically passed a callback function that parses the results and may schedule additional work on the event queue before sending a response. Figure 4.3 illustrates the full Node.js event model including the event queue, event loop, and the thread pool. Notice that the event loop either executes the function on the event loop thread itself or, for blocking I/O, it executes the function on a separate thread. The Conversation Example To help you understand how events work in Node.js versus traditional threaded webservers, consider the example of having different conversations with a large group of people at a party. You are acting the part of the webserver, and the conversations represent the work necessary to process different types of web requests. Your conversations are broken up into several segments with different individuals. You end up talking to one person and then another. Then you go back to the first person and then to a third person, back to the second, and so on. This example has many similarities to webserver processing. Some conversations end quickly, like a simple request for a piece of data in memory. Others are broken up into several segments as you go back and forth between individuals, similar to a more complex server-side conversation. Still others have long breaks when you are waiting for the other person to respond, similar to blocking I/O requests to the file system, database, or remote service. Using the traditional webserver threading model in the conversation example sounds great at first because each thread acts like you. The threads/clones can talk back and forth with cach person, and it almost seems as though you can have multiple conversations simultaneously. There are two problems with this model.thread trom the thread fool and exesutes the neler func, “The event bop reroves @ o/ 8 8/8/ 812 3|i|2/2/2|2 Slelels| sls Blaze) 2|2 SJE/SIEE + Schedule anotner event The cattack unction wil ho one the folouirg: + gare ond + Senda response Figure 4.3 In the Node,js event model, work is added as a function with callback to the event queue, and then picked up on the event loop thread. The function is then executed on the event loop thread in the case of non-blocking, or on a separate thread in the case of blocking First, you are limited by the number of clones. What if you only have five clones? To talk with a sixth person, one clone must completely finish its conversation. The second problem is the limited number of CPUs (or “brains”) that the threads (“clones”) must share. This means that clones sharing the same brain have to stop talking/listening while other clones are using the brain. You can see that there reallyisn’t a benefit to having clones when they freeze while the other clones are using the brain. The Node.js event model acts more like real life when compared to the conversation example. First, Node.js applications run on a single thread, which means there is only one of you, no clones. Each time a person asks you a question, you respond as soon as you can. Your interactions are completely event driven, and you move naturally from one person to the next. Therefore, you can have as many conversations going on at the same time as you want by bouncing between individuals. Second, your brain is always focused on the person you are talking to since you aren’t sharing it with clones. So how does Node,js handle blocking I/O requests? That is where the background thread poo! comes into play. Node.js hands blocking requests over to a thread in the thread pool so that it has minimal impact on the application processing events. Think about when someone asks you a question that you have to think about. You can still interact with others at the party while trying to process that question in the back of your mind. That processing may impact how fast you interact with others, but you are still able to communicate with several people while processing the longer-lived thought. Adding Work to the Event Queue As you create your Node.js applications, keep in mind the event model described in the previous section and apply it to the way you design your code. To leverage the scalability and performance of the event model, make sure that you break work up into chunks that can be performed as a series of callbacks. Once you have designed your code correctly, you can then use the event model to schedule work on the event queue. In Node,js applications, work is scheduled on the event queue by passing a callback function using one of these methods: » Make a call to one of the blocking 1/0 library calls such as writing to a file or connecting to a database. » Add a built-in event listener to a built-in event such as an ht: server.connection. request or = Create your own event emitters and add custom listeners to them. = Use the process .nextTick option to schedule work to be picked up on thenext cycle of the event loop. = Use timers to schedule work to be done after a particular amount of time or at periodic intervals. The following sections discuss implementing timers, next Tick, and custom events. They give you an idea of how the event mechanism works. The blocking I/O calls and built-in events are covered in subsequent chapters. Implementing Timers A useful feature of Node,js and JavaScript is the ability to delay execution of code for a period of time. This can be useful for cleanup or refresh work that you do not want to always be running, There are three types of timers you can implement in Nodes: timeout, interval, and immediate. The following sections describe each of these timers and how to implement them in your code. Delaying Work with Timeouts Timeout timers are used to delay work for a specific amount of time. When that time expires, the callback function is executed and the timer goes away. Use timeouts for work that only needs to be performed once. Timeout timers are created using the set Timeout (callback, delayMilliSeconds, [args]) method built into Node,js. When you call setTimeout (), the callback function is executed after delayMillisec expires. For example, the following executes myFunc () after 1 second: setTimeout (myFunc, 1000); The set Timeout () function returns a timer object ID. You can pass this ID to clearTimeout (timeoutId) at any time before the delayMilliSeconds expires to cancel the timeout function. For example: = setTimeout (myFunc, 1 clearTimeout (myTimeout) ; Listing 4.1 implements a series of simple timeouts that call the simpleTimeout () function, which outputs the number of milliseconds since thetimeout was scheduled. Notice that it doesn’t matter which order set Timeout () is called; the results, shown in Listing 4.1 Output, are in the order that the delay expires. Listing 4.1 simp1e_timex. js: Implementing a series of timeouts at various intervals Click here to view code image function simpleTimeout (consoleTimer) { 02 console. timeEnd(consoleTimer) ; 03 } 04 console. time ("twoSecond") ; 05 setTimeout(simpleTimeout, 2000, "twoSecond") ; 06 console. time ("oneSecond") ; o7 Timeo impleTimeout, 1000, "oneSecona"); 08 console.time("fiveSecond") ; 09 setTimeout (simpleTimeout, 5000, "fiveSecond"); 10 console.time ("50Millisecond") 11 setTimeout (simpleTimeout, 50, "50MilliSecond") ; Listing 4.1 Output simple_timex . 4s: Timeout functions executed at different delay amounts Click here to view code image :\books\node\ch04> node simple timer. S0MilliSecond: 50.489ms oneSecond: 1000.688ms Second: 2000.665ms econd: 5000.186ms Performing Periodic Work with Intervals Interval timers are used to perform work on a regular delayed interval. When the delay time expires, the callback function is executed and is then rescheduled for the delay interval again. Use intervals for work that needs to be performed on a regular basis. Interval timers are created using the set Interval (callback, delayMilliSeconds, [args]) method built into Node,js. When you call setInterval (), the callback function is executed every interval after delayMilliSeconds has expired. For example, the following executesmyFunc () every second: setInterval (myFunc, 1000); The set Interval () function returns a timer object ID. You can pass this ID to clearInterval (intervalId) at any time before the delayMil1liSeconds expires to cancel the timeout function. For example: myInterval = set terval (myFune, 100000); cle interval (myInterval) + Listing 4.2 implements a series of simple interval callbacks that update the values of the variables x, y, and z at different intervals. Notice that the values of x, y, and z are changed differently because the interval amounts are different, with x incrementing twice as fast as y, which increments twice as fast as z, as shown in Listing 4.2 Output. Listing 4.2 simple_interval. js: Implementing a series of update callbacks at various intervals Click here to view code image Ol var x=0, y-0, 2-0; 02 function displayValues () ( 03 console.log("X=8d; Y=8d; Z=8a", x, y, z)i 04} 05 tion updatex () 06 x t= 1; 07 } 08 function updateY() 09 yal; 10 } 11 function updatez() 120 24-1; 13 displayValues(); 14} 15 setInterval (updatex, 500) 16 s 17 s Interval (updateY, 1000); Interval (updateZ, 2000); Listing 4.2 Output simple_interval. js: Interval functions executed at different delay amounts Click here to view code imageks\node\ch04> node simple_interval.js x=15; Performing Immediate Work with an Immediate Timer Immediate timers are used to perform work on a function as soon as the I/O event callbacks are executed, but before any timeout or interval events are executed. This allows you to schedule work to be done after the current events in the event queue are completed. Use immediate timers to yield long-running execution segments to other callbacks to prevent starving the I/O events. Immediate timers are created using the set Immediate (callback, [args]) method built into Node,js. When you call set Immediate (}, the callback function is placed on the event queue and popped off once for cach iteration through the event queue loop after I/O events have a chance to be called. For example, the following schedules my#unc () to execute on the next cycle through the event queue: setImmediate (myFunc(}, 1000); The set Immediate () function returns a timer object ID. You can pass this ID to clearImmediate (immediateld) at any time before it is picked up off the event queue. For example: myImmediate = setImmediate (myrunc}; clearImmediate (myImmediate) ; Dereferencing Timers from the Event Loop Often you do not want timer event callbacks to continue to be scheduled when they are the only events left in the event queue. Node.js provides a useful utility to handle this case. The unre () function available in the object returned by set Interval and setTimeout allows you to notify the event loop to not continue when these are the only events on the queue.For example, the following dereferences the myInterval interval timer: myInterval = myInterval.un t Interval (myFunc) 7 QO: Tf for some reason you later do not want the program to terminate if the interval function is the only event left on the queue, you can use the ref () function to re- reference it: myInterval.ref(); Warning When using unref () with setTimout timers, a separate timer is used to wake up the event loop. Creating a lot of these can cause an adverse performance impact on your code, so use them sparingly. Using nextTick to Schedule Work A useful method of scheduling work on the event queue is the process.nextTick(callback) function. This function schedules work to be run on the next cycle of the event loop. Unlike the set Immediate () method, nextTick() executes before the I/O events are fired. This can result in starvation of the I/O events, so Node.js limits the number of next Tick () events that can be executed each cycle through the event queue by the value of process.maxTickDepth, which defaults to 1000. Listing 4.3 illustrates the order of events when using a blocking I/O call, timers, and nextTick (). Notice that the blocking call fs. stat () is executed first, then two set Immediate () calls, and then two next Tick () calls. Listing 4.3 Output shows that both next Tick () calls are executed before any of the others. Then the first set Immediate () call is executed followed by the fs. stat (), and then on the next iteration through the loop, the second set Immediate () call is executed. Listing 4.3 nexttick. js: Implementing a series of blocking £ calls, immediate timers, and next Tick () calls to show the order in which they get executed Click here to view code image 01 var fs = require ("fs");02 £ at ("nextt 03 nsole. log ("nex: ; 04 ye 05 setImmedia on) { 06 onsole.log("Immediate Timer 1 Executed"); 07 })e 08 setImmediate (function() { 12 console.log("Next Tick 1 Executed"); 13 ))F 14 process .nextTick (function () { xt Tick 2 Executed"); Listing 4.3 Output nexttick . js: Executing the nextTick () calls first Click here to view code image c:\books\node\ch04>node nexttick.js Executed Executed Implementing Event Emitters and Listeners In the following chapters you get a chance to implement many of the events built in to the various Node,js modules. This section focuses on creating your own custom events as well as implementing listener callbacks that get implemented when an event is emitted. Adding Custom Events to Your JavaScript Objects Events are emitted using an EventEmitter object. This object is included in the events module. The emit (eventName, [args]) function triggers the eventName event and includes any arguments provided. The following code snippet shows how to implement a simple event emitter: vi events quire ("events"); emitter = new events.Event mitter ()semitter.emit ("simpleEvent") ; Occasionally you want to add events directly to your JavaScript objects. To do that you need to inherit the EventEmitter functionality in your object by calling events.EventEmitter.call (this) in your object instantiation as well as adding the events.EventEmitter. prototype to your object’s prototyping. For example: Function MyOb3 (i { Events. EventEmi ter.call (this); MyOb}.prototype. proto = r.prototype; You then can emit events directly from instances of your object. For example: var myObj = new MyObj(); myObj.emit ("someEvent") Adding Event Listeners to Objects Once you have an instance of an object that can emit events, you can add listeners for the events that you care about. Listeners are added to an ZventEmitter object using one of the following functions: = .addListener(eventName, callback): Attaches the callback function to the object’s listeners. Every time the eventName event is triggered, the cal back function is placed in the event queue to be executed. =.on(eventName, callback): Sameas .addListener(). s.once(eventName, callback): Only the first time the event event is triggered, the callback function is placed in the event queue to be executed. jame For example, to add a listener to an instance of the MyObject EventEmitter class defined in the previous section you would use the following: function myCallback(i{Removing Listeners from Objects Listeners are usefull and vital parts of Node.js programming. However, they do cause overhead, and you should use them only when necessary. Node,js provides server helper functions on the Event Emitter object that allow you to manage the listeners that are included, These include « . listeners (eventName) : Returns an array of listener functions attached to the eventName event. » .setMaxListeners (n) : Triggers a warning if more than n listeners are added to an EventEmitter object. The default is 10. « .xemoveListener (eventName, callback) : Removes the callback function from the eventName event of the EventEmitter object. Implementing Event Listeners and Event Emitters Listing 4.4 demonstrates the process of implementing listeners and custom event emitters in Node.js. The Account object is extended to inherit from the EventEmitter class and provides two methods to deposit and withdraw that both emit the balanceChanged event. Then in lines 15-31, three callback functions are implemented that are attached to the Account object instance balanceChanged event and display various forms of data. Notice that the checkGoal (acc, goal) callback is implemented a bit differently than the others. This was done to illustrate how to pass variables into an event listener function when the event is triggered. The results of executing the code are shown in Listing 4.4 Output. Listing 4.4 emitter_listener.js: Creating a custom EventEmitter object and implementing three listeners that are triggered when the balancedChanged event is triggered Click here to view code image 01 var events = req 02 function Ac re(‘events'); { 03 -balan 0; 04 s.EventEmitter.call (this); 05 deposit = function (amour06 07 08 09 10 1 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 this.balance += amount; this.emit ("balanceChanged') ; te this.withdraw = function (amount) { this.balance -= amount; this.emit ("balanceChanged') ; Me ) Account.prototype.__prote__ = events.EventEmitter.prototype; function displayBalance(){ console.log("Account balance: $%d", this.balance); ) function checkOverdraw() { if (this.balance < 0){ console. log ("Account overdrawn!!!"); } } function checkGoal(acc, goal) { if (acc.balance > goal) { console. log("Goal Achieved! } } var account = new Account (); account.on("balanceChanged", displayBalance); account.on("balanceChanged", checkOverdraw) 7 account.on("balanceChanged", function () { checkGoal (this, 1000); Me account .deposit (220); account .deposit (320); account..deposit (600); account .withdraw (1200) ; Listing 4.4 Output emitter_listener. js: The account statements output by the listener callback functions Click here to view code image Cz\books\node\ch04>node emmiter_listener.js Account balance: $220 Account balance: $540 Account balance: $1140 Goal Achieved! Aci unt bala $-60 Account overdrawn!!!Implementing Callbacks As you have seen in previous sections, the Node.js event-driven model relies heavily on callback functions. Callback functions can be a bit difficult to understand at first, especially if you want to depart from implementing a basic anonymous function. This section deals with three specific implementations of callbacks: passing parameters to a callback function, handling callback function parameters inside a loop, and nesting callbacks. Passing Additional Parameters to Callbacks Most callbacks have automatic parameters passed to them, such as an error or result buffer. A common question when working with callbacks is how to pass additional parameters to them from the calling function. You do this by implementing the parameter in an anonymous function and then call the actual callback with parameters from the anonymous function. Listing 4.5 illustrates implementing callback parameters. There are two sawCar event handlers. Note that the sawCar event only emits the make parameter. Notice that the emitter.emit () function also can accept additional parameters; in this case, make is added as shown in line 5. The first event handler on line 16 implements the logCar (make) callback handler. To add a color for logColorCar (), an anonymous function is used in the event handler defined in lines 17-21. A randomly selected color is passed to the call logColorCar(make, color). You can see the output in Listing 4.5 Output. ¢ 4.5 callback_parameter. js: Creating an anonymous function to add additional parameters not emitted by the event Click here to view code image 0 0 var show = new function logCar (make) { SSSIAFSON11 console. log("Saw a " + make); 12) 13 function logColorCar(make, color) { 14 console. log("Saw a %s %s", color, make); 15 } 16 show.on("sawCar", logCar); 17 show.on("sawCar", function (make) { 18 var colors = ['red', 'blue', 'black']; 19 var co. = colors[Math. floor (Math. random()*3) ]7 20 logCol r(make, color); 21 4)3 22 show. see! rrari"); -see! Porsche’ -seeCar ("Bugatti"); seeCar ("Lamborghini") ; ("Aston Martin"); seeCa Listing 4.5 Output callback_parameter . js: The results of adding a color parameter to the callback Click here to view code image \books\node\ch04>nede callback_parameter.js Ferrari b Bugatti red Bugatt Lamborghi black Lamborghi Aston Martin black Aston Martin Implementing Closure in Callbacks An interesting problem that asynchronous callbacks have is that of closure. Closure is a JavaScript term that indicates that variables are bound to a function’s scope and not the parent function’s scope. When you execute an asynchronous callback, the parent function’s scope may have changed; for example, when iterating through a list and altering values in each iteration. If your callback needs access to variables in the parent function’s scope, then you need to provide closure so that those values are available when the callback is pulledoff the event queue. A basic way of doing that is by encapsulating the asynchronous call inside a function block and passing in the variables that are needed. Listing 4.6 implements a wrapper function that provides closure to the LogCar () asynchronous function. Notice that the loop in lines 7-12 implements a basic callback. However, Listing 4,6 Output shows that the car name is always the last item read because the value of message changes each time through the loop. The loop in lines 13-20 implements a wrapper function that is passed message as the msg parameter and that msg value sticks with the callback. Thus the closure shown in Output 4.6 displays the correct message. To make the callback truly asynchronous, the process .nextTick () method is used to schedule the callback. Listing 4.6 callback_closure. js: Creating a wrapper function to provide closure for variables needed in the asynchronous callback Click here to view code image 01 function logCar(logMsg, callback) { 02. process.nextTick(function() { 03 allback (logMsg) ; = ["Ferrari", "Porse x in cars) { 08 var message = "Saw a" + cars[idx]; 09 = logCar(message, function() { onsole.log("Normal Callbac! » "Bugatti" + mess iL 12 } 13 for (var idx in cars) { 14 var message = "Saw a" + cars[idx]; 15 (function (msg) { 16 logCar(msg, fu Ww console. le back: " + msg); 18 de 19 ) (message) ; 20 } Listing 4.6 Output callback_closure.js: Adding a closure wrapper function allows the asynchronous callback to access necessary variables Click here to view code imageck_closure.js \pooks\node\ch04>node_callb rmal Callba Saw a Bugatti ure Cal Chaining Callbacks With asynchronous functions you are not guaranteed the order that they will run if two are placed on the event queue. The best way to resolve that is to implement callback chaining by having the callback from the asynchronous function call the function again until there is no more work to do. That way the asynchronous function is never on the event queue more than once. Listing 4.7 implements a basic example of callback chaining. A list of items is passed into the function LogCars (), the asynchronous function logCar (} is called, and then the logCars () function is used as the callback when LogCar () completes. Thus only one version of LogCar () is on the event queue at the same time. The output of iterating through the list is shown in Listing 4.7 Output. Listing 4.7 cal back_chain. js: Implementing a callback chain where the callback from an anonymous function calls back into the initial function to iterate through a list Click here to view code image nction logCar (car, console. log(" if (car: allback) { awa Ss", ; logCars (cars); ferrari", "Porsche", "Bugatti",16 "Lamborghini", "Aston Martin"); ) 17 log¢ars (ca Listing 4.7 Output callback_chain. js: Using an asynchronous callback chain to iterate through a list Click here to view code image C:\books\node\ch04>node callback chain.js rs » ze boos Summary The event-driven model that Node.js uses provides scalability and performance. You learned the difference between the event-driven model and the traditional threaded model for webservers, You learned that events can be added to the event queue when blocking 1/0 is called. And you learned that listeners can be triggered by events or timers or called directly using the next Tick () method. This chapter discussed the three types of timer events: timeout, interval, and immediate. Each of these can be used to delay the execution of work for a period of time. You also saw how to implement your own custom event emitters and add listener functions to them. Next In the next chapter you see how to manage data I/O using streams and buffers. You also learn about Node.js functionality that allows you to manipulate JSON, string, and compressed forms of data.