Getting started with Node.js

 Published on 2018-11-23

Node.js is an open-source runtime that enables us to create applications that run outside of a browser (for example, for dynamic server-side scripting). It is based on JavaScript. There are a lot of companies that use Node.js for various projects, including IBM, Groupon, Microsoft, Netflix, PayPal, SAP, Walmart, Yahoo, and others.

Node.js is based on an event-driven architecture, where requests and I/O operations are handled asynchronously. In short, this means that when a request arrives that requires a (relatively) long processing time (like connecting to another server, a database, reading a file, etc), Node.js handles it asynchronously (for example, sends it to the file system for processing), and then continues with the next request. This enables Node.js to handle multiple requests (at once), even though it's single-threaded.

Installing Node.js

It is very easy to get started with Node.js. Just head out to the official Node.js website at https://nodejs.org and download the latest LTS version (recommended for most users, offering Long Term Support). If you're on Windows (where most users are), you just need to install the file that you download. For Linux and other systems, you just follow the instructions presented on the official website.

To verify that your installation is working correctly, open a command prompt and type "node –version". If you don't get an error message, that means you have Node.js setup correctly. Otherwise, restart your computer and try to run "node –version" again. If you get an error (very rare), use Google to search for a solution to the problem by typing the exact error you got.

Just to see what you can do with Node.js, let's create a file called "test.js". In the file, copy the content provided below (please notice how this is plain JavaScript, nothing complicated).

var a = 3;
var b = 5;

var result = (a+b) / 2;
console.log("The result is:", result);

Save the file, open a Command Prompt (or a Terminal, if you use Linux), go to the directory where you created the "test.js" file, and type the following command "node test.js". You should see the result of the calculation.

Awesome, we've created our first Node.js script. Now, let's get down to the interesting stuff.

Setting up Express.js

We can use Node.js to create backend applications that serve dynamic content. This is a very common use case of Node.js. To get started, we must decide which web framework we are going to use, because there are multiple options available.

In this tutorial, we are going to focus on Express.js – a fast, minimalist web framework for Node.js, which provides a large set of features through a very simple API. Express is the most popular web framework for Node.js. As a matter of fact, a lot of other frameworks are based on Express, allowing you to easily transition there if you decide so in the future.

When you installed Node.js, you automatically installed NPM – the package manager for JavaScript (in practice, it's also the world's largest software registry). When working with Node.js, we use NPM a lot of the time, to install various libraries and frameworks – for example, a library to handle database operations, emails, command-line tools, etc. In other words, in Node.js world, we can install modules created from other companies and users, directly from the NPM repository.

Let's see how NPM works, by installing a tool on our system for creating application skeletons (starter code for our new applications). This will make the creation of new applications very easy and fast. Open a Command Prompt or Terminal, and execute the following command "npm install -g cloudapps". This will install the cloudapps command-line tool/interface globally (hence the -g flag), allowing us to create new applications in any directory we like. It's that easy.

Now, to create a new Express.js application, just create a folder where you want to store your Express.js applications (let's say it's C:/apps), go into it with a Command Prompt or Terminal, and write "cloudapps new". This will start the cloudapps cli, after which we're going to be asked for a name (of our new project, for example "First test"), and the type of project we like to create (we want a simple Express.js app with Gulp - i.e. nothing fancy).

That's it! Our new Express.js app has been created. We just need to switch to the folder where it is (it depends on the name you chose in the previous step), run "npm install" there to install all dependencies, and run "npm run start" to run it. If you now open a browser (Chrome, Firefox, etc) and type the url "http://localhost:3000" there, you're going to see your first app. Pretty cool, right?
(Btw, you should know that the ":3000" part in the url specifies a port - basically, for development purposes, our application runs at port 3000 so it doesn't interfere with anything else).

IDEs

Please note that you can use IDEs (Integrated Development Environments) to edit the files inside your new project, or any text based editor (because we're going to be editing JavaScript files that are source text files – i.e. not binary files). Some good IDEs include: Visual Studio Code or Webstorm. Other tools you can use include Atom or Brackets. You can look up all of these tools on Google and download them from their official website. They offer easier file editing, code competition and a faster development experience.

Creating endpoints

Whether or not you're using an IDE, open the "routes" folder of your project, and then edit the "index.js" file. Please notice how it contains several definitions of endpoints (i.e. what happens when a certain route in our application is hit). For example, when someone hits the "/" route with a GET request, that actually corresponds with someone opening the homepage of our application (http://localhost:3000, or http://yourdomain.com). You can put anything you want inside that function, and you will notice (once your application restarts), how the result of hitting that endpoint will change. For example, let's edit the router.get('/') method, and let's put the following code there:

let a = 5;
let b = 3;
let result = 3*(a+b);
res.send('The result is: ' + result);

Now, if your application is up and running (note how we mentioned that you use "npm run start" to run it), you can type the development url in your browser to see the result: http://localhost:3000. You should get the response that we defined. This is the first route you implemented by yourself, and it shows how we can use Node.js to serve dynamic content.

Please note that, if you add a console.log() statement somewhere in the method we edited, the logs will be available in the terminal where you run the "npm run start" command, and will not be visible inside the browser. In the browser you'll be able to see the data sent from the res.send(…) call.

Applications like this one are usually run on a server (in a datacenter), with a domain like "yourdomain.com" pointing to them. But, the way your application works on your computer when you test it (the http://localhost:3000 url), will be exactly the same when the app is run on a real server. This allows us to develop apps faster, and to test things out before pushing them to production, where real users access them.

Modules

If you read the routes/index.js file more carefully, you'll notice that on the top of the file we have some require(…) function calls. In Node.js we use require(…) to include modules that we want to use. You can think of modules as libraries that contain various functions we can use.

Node.js itself has some built-in modules for dealing with the file system, http, cryptography and other common tools we might need. However, we can use NPM to import other modules to our project, or create our own modules. Similarly, we can create simple .js files, export some functions from them, and then require(…) them in other places. This makes our code much easier to read and test.

As an example, imagine you want to read a file during a call to some route. The first thing you need to do is import/require the fs module, with something like [ var fs = require('fs'); ]. Then, in any location in your source code (in the same file), you can call a function from that module – for example, fs.readFile, fs.writeFile, etc. It's that simple.

Events

Because Node.js is single-threaded, we often need to handle various events (like, a file has finished writing, etc). In other words, when we handle a request, we don't want to stop the execution of everything else while we wait for something to complete (like, a large file to be written), but instead, we want to handle other requests in the meantime, and return back to our current function when the file has been successfully written.

Node.js is great at handling events. In Node.js, we handle long-running tasks with callbacks, promises or async/await (shown below), but a common module to handle other events (for example, something we implement ourselves) is also available. Let's take a look at EventEmitter, which is available from the "events" module.

With an EventEmitter, we fire events with the emit() method, and we handle events with the on(…) method. By the way, If you've used libraries like jQuery, you'll notice the similarity here (in browsers, we also have events - clicks, changes in a text input, etc).

var events = require('events');
var emitter = new events.EventEmitter();


//Let's create a function to handle the event once it's complete
emitter.on('testEvent', function(){

   console.log('Event fired. Calculation has been completed.');
});

//Fire/emit the event after 3 seconds
setTimeout(function(){
   emitter.emit('testEvent');
}, 3000);

In the example above, we used setTimeout() to fire an event (which we named "testEvent") after 3 seconds (3000 milliseconds) to simulate a long-running task. In practice, long-running tasks might include talking to another server (possibly available in a different datacenter, or even on a different continent), processing large amounts of data, etc.

Please note that you can run the example above by creating a file "test.js" and then running the command "node test.js" (from the directory where the file is located). Of course, events and EventEmitter can also be used in Express.js.

Callback functions

Ok, this part might confuse you at first, but it's the key to understanding how Node.js works. Remember when we said that Node.js is asynchronous and single-threaded? Well, what happens when we have a long-running task? How do we let Node.js continue running other stuff (for example, serving other requests), while our long-running task is active, and then continue where we left off?

The key to understanding this comes down to understanding callbacks. Callbacks are functions that we pass as arguments to other parts of our code, that will (later) be called back – once a task has been completed. In Node.js, by definition, the first parameter of a callback function is an error object (and that allows us to know if an error occurred or not, and what that error is). Let's see how we can read a file in Node.js asynchronously, using callbacks.

fs.readFile(filename, function(err, data) {
    if(err) {
        return console.log('ups, an error occurred!', err);
    }

    console.log("data: ", data)
});

Let's take our time here, as this is rather important. Please notice how the fs.readFile(…) function takes two parameters: a filename (which specifies the file we want to read, like 'C:/file.txt'), and a callback function that will be called once the file has been read.

When the function completes it's job, it will call that function with two arguments: an error object (sometimes a string), and data (a buffer with the file content, or the contents as a string – if we specify an encoding). Please notice how we check if we have received an error first (this is very common with callback functions – because we want to handle errors first – errors, in this case, might be a missing file, a file that is open by some other process which is currently writing data to it, etc).

Callbacks are used when we have a long running task. Think of I/O operations with the file system, emails, database operations, etc. Because these tasks take time, we want to use callbacks, in order to know when the event is done (event-driven programming).

It's really not that complicated once you try to think about this in more detail. We want to handle a long-running task asynchronously, and callbacks are a nice way for us to be informed when the task has finished executing. As it turns out though, callbacks are rather messy, and really painful when we want to do multiple tasks – for example, read one file, then another file, and then contact some server (in this case, we will end up with callbacks inside callbacks).

fs.readFile(filename1, function(err1, data1) {
    if(err1) {
        return console.log('ups, an error occurred!', err1);
    }

    //else, read the second file
    fs.readFile(filename2, function(err2, data2) {
        if(err2) {
            return console.log('ups, an error occurred!', err2);
        }

        //else, read the third file
        fs.readFile(filename3, function(err3, data3) {
            if(err3) {
                return console.log('ups, an error occurred!', err3);
            }

            //now, do something with the file contents
            processData(data1, data2, data3);
        });
    });
});

Not very pretty, and very difficult to read, right? It was very common to use Node.js in this way though, even as recently as one year ago. Let's see how we can improve on this example with promises and async/await.

Promises and async/await

Promises allow us to link the producing code (the code that takes time to complete a task, and produce a result or error) and consumers or handlers (the code/functions that wait for a result). To be more specific, a promise is a proxy for a value not known when the promise is created. We use the .then() method to wait for a result, and the .catch() method to catch an error.

Most libraries available today support promises by default - and for others, we can use libraries like Bluebird (or Node.js util module) to convert callback functions to promises (or create our own promises without any library – though, this is not really something we should aim for).

Let's see how the code above (where we read three files) can be rewritten using promises. Please note how we use the promisify method (from "util") to convert a function that expects a callback, to a promise.

var fs = require('fs');
var util = require('util');
var readFileAsync = util.promisify(fs.readFile);

readFileAsync(filename1)
    .then(function(data1) {
        return readFileAsync(filename2);
    })
    .then(function(data2) {
        return readFileAsync(filename3);
    })
    .then(function(data3) {
        //all three files have been read
        processData();
    })

    .catch(function(error) {
      console.log(error);
    });

This technique of calling one .then() method after another is called chaining. The code is much nicer now, and much easier to read, test and debug. Promises are really cool, and we can do various things with them. However, there is another step that will make our code even better – using async/await.

var fs = require('fs');
var util = require('util');
var readFileAsync = util.promisify(fs.readFile);

Promise.resolve()
  .then(async function(){
      var data1 = await readFileAsync(filename1);
      var data2 = await readFileAsync(filename2);
      var data3 = await readFileAsync(filename3);

      processData();
    })
  .catch(function(error){
    console.log(error);
  });

Now, this is even better, and shows the power of async/await when we have promises. Our code looks very linear now, almost as if though we're dealing with synchronous execution (there is no need for chaining – as with promises, or nesting – as we did with callbacks). Please note that we can only use await inside async functions (therefore, we created one ourselves).

The catch() method on the bottom will only execute if there is an error (like, a file doesn't exist). Promises and async/await make programming with Node.js much easier. Just remember that long-running tasks use callbacks by design (old school), but some/most newer libraries and methods (available today) support promises by default. If a library doesn't support promises by default, you can use the "util" module (or a library called "bluebird") to make functions work with promises.

EJS

Ok, so far, we've outlined the basics of developing web applications with Node.js. However, we've skipped one major issue: how to actually render HTML. There are two common solutions that are popular today: using Single Page Applications (where we use Node.js to return JSON data or text, as we did so far), or use Express to render pages. Let's see how we can do the latter.

EJS (Embedded JavaScript templating) is one solution we can use (though, Express.js supports several others). EJS is a templating language that let's us generate HTML markup with plain JavaScript (so, we can use conditionals, for loops, etc). It's very fast, enables us to use things we already know (the JavaScript language), and supports a lot of features out of the box - like partials, caching or delimiters.

Using EJS is very straightforward, especially with the project boilerplate generated for us through the "cloudapps" command line interface we saw previously. Let's do a simple EJS page.

Start by opening the "routes/index.js" file, and edit the last line of any get method (say GET '/'), to the following:

   return res.render('test', {
      name: 'Donald Trump',
      children: ['Eric', 'Barron', 'Ivanka']
   });

This allows us to see that we can send arguments to EJS (in this case, "name" and "children"). With the code above, we ask EJS to render something called "test" for us. By using names like this, we can create multiple views (a home page 'home', a login page 'login', etc). To see how we can do the "test" view, simply go to the "views" directory (it already exists with our Express.js project boilerplate) and create a file called "test.ejs" there. You can put the following content inside the file:

<h3><%= name %></h3>

<p>
  <% for (var child of children) { %>
    <%= child %><br />
  <% } %>
</p>

This might look complicated at first, but trust me, it's not. We just use the tag <%= … %> to output values (like "name", and "child"), and we use the tag <% … %> (without the equals sign '=') to be able to use JavaScript code (in our case, for the "for" loop, where we loop through the "children" array). The final thing that you should notice is how we've opened the for loop inside the first <% … %> tag, and that we closed it with the second <% … %> tag. This enables us to go from JavaScript world, into simple HTML rendering.

Try a couple of examples (especially with an IDE, since it offers syntax highlighting), and you will really get the basics of EJS very quickly.