I started out in my software engineering life writing software for embedded real-time systems and communication stacks, so I often think of myself as an async native. I’ve written a lot of code in various languages, mostly not using event-driven programming models and design patterns, but hardware interrupts, callback functions, event loops and finite state machines – I still see these as reassuringly familiar features of my original habitat. Call it the baby duck syndrome.

Who is this post for?

This post is primarily aimed at developers working with (or planning to work with) server-side JavaScript using NodeJS. If you’re already familiar with event loops and state machines then much of this article will be familiar to you. For others, I’m hoping this post will provide you with an understanding of the fundamentals of the event-driven programming model you will need when creating Finite State Machines or working with NodeJS on the server or on a Raspberry Pi or similar device.

Events and Event-driven Programming

A conventional program takes input data and transforms it by processing into the required output data, and has full control of the linear processing thread from the beginning to the end.

In contrast, a lot of systems are naturally event-driven, which means that they are being continuously triggered by the occurrence of some external input or internal event such as a mouse click, a button press, a time tick, or an arrival of a data packet. After recognizing the event, the system reacts by performing some appropriate action. Once the event handling is complete, the system goes back to waiting for the next event.

A McAnalogy

To illustrate this contrast between a classic linear, “thread-based” model of handling program execution and the event-driven model, lets look at how you order food at a typical fast-food restaurant. (or “quick service restaurant” as they prefer to call themselves)

The linear, “thread-based” way of implementing the service would be to take an order, take payment, cook the food, give it to the customer, and then turn to the next customer. I actually worked in a burger bar that worked this way while paying my way through engineering school. This model is simple and works fine at low scale. And to service more customers; simply add another staff member doubles throughput.

Modern fast food restaurants don’t work that way. They are designed to scale (handle larger customer volume without growing staff numbers linearly) using an event-driven model. When you place your order, it’s sent out back for a cook to fulfill while the cashier takes your payment. The cashier then takes the next order. At some point, your cashier is notified of your order being ready, and hands it to you (this is analogous to the triggering of callback function).

The key point is that cooking is not “blocking” the receiving of new orders. Non-blocking functions are a key feature of successful event-driven systems.

Event-driven programming is the paradigm most used in GUI frameworks that are built to take specific actions in response to naturally unpredictable user input. Typically in event-driven systems, you need to deal with the fact that the appropriate action to take upon a particular event depends on actions that have taken place previously, and on their results. And like the restaurant, a key goal of event-driven programming is scalability (handle larger volume without growing CPUs linearly)

The Paradigm

A state machine is an abstraction of the basic event-driven model. An event-driven state machine can be simply compared to somebody with amnesia who is constantly asking:

• Where am I?
• What just happened?

and…

• So what do I do (or where do I go) next?

In embedded systems, where you don’t have an operating system to get in the way …I mean support you… you use hardware interrupts to trigger interrupt handler functions directly.

In higher-level systems, you start with a simple main loop that takes events from a queue.

In a Finite State Machine (FSM), you let your state machine know “where am I?” and “what to do I do next?” by defining a finite list of states your system can be in, the events that can occur, and the functions which are triggered by events and which in turn drive state transitions. Then you start up your main loop and away it goes.

In an event-driven system like Windows, or the JavaScript runtime engine, there is an in-built background event loop that listens for events, and then triggers your callback functions when one of those events you’re listening for is detected. In such systems, instead of implementing a state machine, event-driven applications can be implemented using languages supporting high-level abstractions such as event listeners, callbacks and closures. More on closures later.

Finite State Machines

You can skip this section if it’s not FSMs you are interested in. You don’t need to understand FSMs to develop Node applications.

In an FSM, the response to an event generally depends on both the type of the event and on the internal state of the system.

As a by-product of the FSM model, it’s easy to represent your code behavior visually. My FSMs usually look like this one, used to implement a TCP/IP stack:

where the circles represent states and the arrows represent actions and state transitions. The concept of the FSM is important in event-driven programming precisely because it makes the event handling explicitly dependent on both the event-type and on the state of the system.

The good news is that using the JavaScript engine server-side doesn’t require you to implement a FSM in your application – maintenance of state is decentralized and implicit when you use closures in setting up your callbacks, as we’ll see below.

The traditional techniques for selecting the next action based on an event and a state and perhaps a condition will result in convoluted conditional logic (in C/C++, you’ll see these coded as deeply nested if-else or switch-case statements). Doing away with these conditional branches to make your code easier to write, understand, test, and maintain, will simplify your job by an order of magnitude – and that’s what a well-designed state machine will do.

Most of my early software was written in C, and it turns out that C has a feature that is very useful for writing simple and efficient finite state machines – the function pointer. You can always implement an FSM in any language using nested switch/case or if-then-else statements. But being able to define a variable that was a pointer to (i.e. the address of) a function, means you can write a simple FSM loop that processes events using a struct like this:

---

/* FSM States */
#define STATE_START 0
#define STATE_INITIALIZED 1

/* FSM Events */
#define EVENT_INIT   0
#define EVENT_ERROR 1
…
/* FSM action functions */
int initialize(void) {
   /* Initialize FSM here */
   return (STATE_INITIALIZED);
}
…
/* FSM actions and transitions */
const struct {
   int state;
   int event;
   bool condition;
   (int)(*fn)();
} fsm [] = {
   {STATE_START, EVENT_INIT, initialize},
   {STATE_START, EVENT_ERROR, error_handler},
   {STATE_INITIALIZED, EVENT_ERROR, error_handler},
…
};

---

I wrote an FSM using this technique for a public Bildshirmtext (“Btx”) terminal while working in Germany in the late 1980s. These were pre-Internet days, but the terminal had keyboard and block-graphics UI and used an embedded dial-up modem and the Btx protocol to implement some features pretty familiar to Web 1.0 users, like company sites, online shopping, information pages, and even online banking.   

The downside of FSMs

The FSM is a great model for tackling problems where there are a manageable number of states – say up to a dozen or so states. The complexity of the FSM table quickly gets out of control beyond that. In the past, I’ve implemented nested hierarchies of FSMs to cope with more complex real-time solutions, but its really stretching the paradigm.

The use of an event-driven language and framework like JavaScript and Node can overcome this issue.

Node and JavaScript

Although Node is mostly seen as a way to run JavaScript on a server, JavaScript isn’t actually the point of Node; it was just a means to achieve a non-blocking event-driven platform. There is much more to Node than just JavaScript.

The nodejs.org homepage boldly declares:

Node.js is a platform built on Chrome’s JavaScript runtime for easily building fast, scalable network applications. Node.js uses an event-driven, non-blocking I/O model that makes it lightweight and efficient, perfect for data-intensive real-time applications that run across distributed devices.

Well, that’s pretty much Node in a nutshell right there. Node is a server-side JavaScript Runtime Environment designed on the event-driven paradigm. The Google V8 JavaScript engine used in the Chrome browser is used as its JavaScript interpreter, complemented with a complete non-blocking API library, based on an asynchronous I/O model. On top of this API, Node provides a whole raft of modules to help build your network-enabled applications quickly.

JavaScript was specifically designed for use in the browser and thus for an event-driven GUI context. It handles asynchronous events through callbacks attached to objects and events. If you’ve written any client-side JavaScript at all you will have used event listeners and callbacks.

In this simple example, when the user clicks on the button, “Hello World” appears in the browser window:

---

document.getElementById("myButton").onclick = function(){
    document.getElementById("myTextElement").innerHTML = "Hello World";
};

---

There are even multiple ways to achieve the same event-driven behaviour :

---

document.getElementById("myButton").addEventListener("click", function(){
    document.getElementById("myTextElement").innerHTML = "Hello World";
});

---

That makes JavaScript a good choice for event-driven programming in general.

In a browser context, the DOM is an API – effectively it’s the browser’s asynchronous I/O library, and the callbacks and listeners in its library functions deliver the events driving your application.

Once Node provided a corresponding asynchronous I/O framework for JavaScript to run on the server side, JavaScript was able to escape from the browser.

Callbacks

Where do my callbacks go when I submit them in an asynchronous function call ?

The JavaScript runtime engine has an event or message queue, which stores events to be processed, and their associated callback functions. In an event loop, the queue is polled for the next event and the callback for that event is executed. Callback functions processing an event can also fire off new callbacks, which result in new events appearing in the queue, often making for some pretty convoluted-looking code.

If you come from a “traditional” background in multi-threaded backend development in C/C++/C#, Java, etc., then you may have to go through some mental recalibration before you start to get asynchronous. Luckily, Node makes thinking asynchronously simpler.

Closures

JavaScript’s async libraries and closures mean there is no need for you to implement a formal FSM to explicitly manage states in your application. Closures make asynchronous programming elegant. To understand closures, you have to understand function scope.

In C, the parentheses { } always define scope, called block scope. JavaScript also uses parentheses, but they do not give you block scope. In JavaScript, functions provide scope – known unsurprisingly as function scope.

Function scope means that everything between parentheses operators { } of a function provides your scope. Use of this function scope is very common in JavaScript code, even as far as defining a “dummy” function purely to limit scope and prevent module variables having global scope.

As we’ll see, JavaScript being functionally scoped encourages asynchronous, event-driven programming.

In JavaScript, a function has access not only to its own local variables and global variables, but also has access to its “referencing environment” – that means a function defined within an outer function has access to all the variables defined in the scope of the outer function. In JavaScript, that inner function is a “closure”.

JavaScript’s closures allow you to submit callbacks that, when triggered, will still have access to the variables scoped by the outer function they were spawned from, even though the outer function has since terminated. That means you can simply maintain state in an outer function instead of having to remember to pass it all via the callback.

To illustrate how this allows a callback to access the environment it was initially created in, take a look at this browser-based JavaScript example, where we again display “Hello World” but this time 5 seconds after clicking a button:

---

function delayedHelloWorld () {
 var textElem = document.getElementById("MyTextElement");

 setTimeout(function helloWorld() {
   textElem.innerHTML = "Hello World";
   return false;
 }, 5000);
 return false;
}

document.getElementById("myButton").addEventListener("click", delayedHelloWorld );

---

In the example, the delayedHelloWorld callback function includes a local variable textElem. The async function setTimeout is invoked in this function, firing off another callback called helloWorld. This adds the hellloWorld callback to the event queue managed within the JavaScript runtime engine, and this callback is triggered after approximately 5 seconds. Our delayedHelloWorld function then returns. But here’s the thing: the textElem variable is still referenced via a closure and so is not garbage collected by the JavaScript runtime. When the timer event triggers the helloWorld function, it still has access to the textElem variable declared in the outer function’s scope, even though that function has already terminated. Once the helloWorld function has finished processing the event, the textElem variable can be garbage collected.

Using Node, you can get the same effect of private state using JavaScript outside the browser environment. The following example will display “Hello World” on the console, with a 5sec delay.

---

function delayedHelloWorld() {
 var state = 64;

 setTimeout(function () {
   if (state === 64) {
      console.log("Hello World");
   }
   return false;
 }, 5000);
 return false;
}

delayedHelloWorld();

---

Node is retaining the callback’s context, ensuring the callback has access to the value of the state variable defined in the outer function.

In contrast, C doesn’t have closures, making it a bit more difficult to write this kind of non-blocking code in C.

To Node or not to Node ?

This post was mainly about understanding the event-driven programming model behind Node, rather than recommending the use of Node for a specific application. But if you are still considering whether to use Node for you next project, here are a few other pros and cons to consider.

More Advantages of Node

As a server-side framework, Node has a lot more going for it than just being an event-driven framework :

• It’s fast. Do the math.
• It was made for real-time communications. And not just http.
• It fits seamlessly with noSQL databases like MongoDB.

Challenges using Node

• It’s what’s euphemistically called “non-trivial” to learn.
• The API is still evolving, which makes the top of that learning curve a bit of a moving target.
• If you’re a JavaScript developer you may struggle with Node because of the low level nature and the level of control you have. It’s JavaScript, but it feels more like C in that regard. There’s lot of flexibility and power – and with great power comes great responsibility…

What Next ?

To try Node out, first install on your Mac as I described here.

Then if you want to create your first Node-based web application on Amazon, follow my 3-step how-to guide – “A Node.JS Application on Amazon Cloud”:

• Part 1: Installing Node on an EC2 instance
• Part 2: Adding a Database
• Part 3: A simple Webserver in Javascript using Node, Express, and MongoDB

Finally, if your target device is a Raspberry Pi, read how to install Node on your Raspberry Pi.