How To Go Full Stack with ExpressJS and APIs

Post Full Stack World – What Happens Now?

Around 2011-2014, the idea of a full stack JavaScript developer became mainstream. Browsers became more standardized, hybrid native app frameworks, like Cordova, grew in sophistication.
 
Then Node.js continued to grow in popularity. We could work on independent small JavaScript projects. Each project had its own client, server, and database.
 
Yet, there were three main factors that slowed the pace of the rise of full stack JavaScript:
 
  • Increasing fragmentation in the client-side JavaScript space, and
  • Continued superiority of native mobile apps over hybrid mobile apps,and
  • The power of specialization

What’s Next with Express?

Now, that promise of a single unified client is no longer an imminent possibility.
 
APIs, REST or otherwise, are more pivotal than ever for unifying disparate clients.  
 
Node.js has a powerful concurrency model, along with a prolific npm and Express ecosystem.  So these factors make Express the ideal tool to build your next API.

Shouldn’t We All Just Be Using Firebase?

First of all, there was a time when AngularFire meant the end of backend development. Firebase’s client library maintained a socket connection to a realtime database.
 
Also, AngularFire integrated with Angular 1’s dirty checking. You know what else? It also synced your changes in JavaScript to the realtime database with no work on your part. So, you just write some HTML and you’re done.
 
Additionally, you add in over-the-air updates with Ionic 1 and you had a mobile app too. That’s some magic, right?
app.controller('MyController', function($scope, $firebaseObject) 
{
  var ref = firebase.database().ref();
  $scope.data = $firebaseObject(ref);
});

<div controller="MyController">
  <h1>{{data.title}}</h1>
</div>

Now what?

Unfortunately, it also became clear that Angular 1 was difficult to optimize and extend beyond building simple forms. Furthermore, Cordova is proving to have similar issues. Not to mention that some sophisticated clients require developers to specialize.

So let’s assume you need disparate clients like an iOS app, an Android app, an Electron desktop app, and a web client and you need a centralized logic layer.

In case you are wondering why – if you use a realtime database directly and your Android app’s logic doesn’t keep up with your iOS app’s logic, you’ve got a problem.

In order to find a good solution, it’s time to bring on the APIs.

What Should APIs Do?

KeenIO summed it up best when they said APIs “boil complex processes down to simple commands that magically do lots of work for you.” 

Additionally, like Browserling put it, an API is like a Kraken that lurks under the surface of your apps and ties together logic and data sources so your apps don’t have to.

In order to help you get started, here’s a few examples of tasks, with code snippets, that APIs should take care of for their clients.

Security

In case you missed the memo, clients are insecure. Recently, I was reminded of this when I booked a blocked-off slot for car maintenance. Because the dealer’s site relied on Angular 1 form validation resulting in the fact that the slot I chose was not verified as “available” on their server.

How can developers solve issues like this and prevent bad user experiences?

Express middleware makes it easy to define security rules for a group of routes without repeating yourself.  Also, the express-jwt library makes it easy to set up JSON Web Token authentication and you can write middleware to define your own custom rules.

const app = require('express')();
const bodyParser = require('body-parser');

const jwt = require('express-jwt')({ secret: 'my secret key' });

// Now all HTTP endpoints that start with `/admin` and `/user`
// require JWT authentication
app.use('/admin', jwt);
app.use('/user', jwt);

// Add an additional layer of security to `/admin` endpoints to
// make sure only admin users reach it
app.use('/admin', function (req, res, next) {
  // `express-jwt` sets `req.user` for you. Conceptually, jwt's
  // encrypt the JSON representation of the user using the secret key.
  // In other words, your access token for the API is your encrypted
  // user data!
  if (!req.user.isAdmin) {
    return res.status(401).json({ err: 'Must be admin!' });
  }
  return next();
});

Concurrency and Locking with Express Middleware

Importantly, Node.js is non-blocking and there’s no standard Node.js notion of a “lock” (as opposed to languages like Java) In addition, when you have multiple servers on different machines, the standard in-memory locks you might remember from undergrad systems programming are not very useful.

Let’s face it – managing distributed locking across different clients is a nightmare. But you need distributed locking, so what to do?

For example, Express middleware makes it easy to lock a resource for a certain group of endpoints and locking a user every time the client hits an endpoint that updates a user.  So you might have separate endpoints like updateAdmin for doing special types of updates.

As a good rule of thumb, you’d like to lock the user every time someone hits a PUT endpoint under /user.

MongoDB to the rescue!

In order to help get you started, here’s an example using MongoDB as the store for the distributed lock.

const { MongoClient } = require('mongodb');
const app = require('express')();
const bodyParser = require('body-parser');

async function run() {
  const db = await MongoClient.connect('mongodb://localhost:27017/test');

 app.put('/user/:id/*', async function(req, res, next) {
    // If we successfully upserted, that means we acquired the lock. Otherwise,
    // means the resource is already locked
    const result = await db.collection('Lock').findOneAndUpdate(
      { resource: 'User', id: req.params.id },
      { $setOnInsert: { createdAt: new Date() } },
      { upsert: true, returnOriginal: false });

   if (!result.lastErrorObject.updatedExisting) {
      // Acquired the lock!

     res.on('finish', () => {
        // Release the lock by deleting the lock document when the request
        // handler is done
        db.collection('Lock').deleteOne({ _id: result.value._id });
      });
      return next();
    }

   res.status(409).json({ error: `Resource ${req.params.id} locked` });
  });

 app.put('/user/:id/updateAdmin', function(req, res) {
    res.json({ ok: 1 });
  });

 app.put('/user/:id/update', function(req, res) {
    res.json({ ok: 1 });
  });

 app.listen(3000);
}

run().catch(error => console.error(error.stack));

Data Validation

Since data formats change fast, and apps can’t keep up unless you force upgrade your users regularly. Because JavaScript is a dynamically typed language, the JavaScript community has a myriad of well-adopted type casting and data validation libraries, including mongoose, joi, ajv, and others.

In addition to the benefits of JavaScript, Express error handling middleware can help you out by enabling you to handle data validation errors in a standard way across your application.

const Joi = require('joi');
const { MongoClient } = require('mongodb');
const app = require('express')();
const bodyParser = require('body-parser');

async function run() {
  const db = await MongoClient.connect('mongodb://localhost:27017/test');
  // `schema` lets you validate that objects match the given schema
  const schema = Joi.object().keys({
    email: Joi.string().required().regex(/^.+@.+\..+$/),
    name: Joi.string().required()
  });

  // Express body parser
  app.use(bodyParser.json());
  // Sample endpoint that inserts a user in the database if it is valid
  app.post('/user', async function(req, res, next) {
    const result = schema.validate(req.body);
    if (result.error) {
      return next(result.error);
    }
    await db.collection('User').insertOne(req.body);
    return { user: req.body };
  });
  // Express error handling middleware. Will execute if an error was passed to
  // `next()`. When you add more endpoints, they will still have the same
  // way of reporting errors.
  app.use(async function(err, req, res, next) {
    if (err.isJoi) {
      return res.status(400).json({ err: err.message, details: err.details });
    }
    next(err);
  });
  // Catch-all error handler
  app.use(function(err, req, res, next) {
    return res.status(500).json({ err: err.message });
  });

  app.listen(3000);
}

run().catch(error => console.error(error.stack));

Interfacing With Externally Facing APIs

For both security and maintainability, APIs should be responsible for most interactions between your software and external APIs.

Also, leveraging external APIs is difficult and error prone, having a centralized layer for communicating with external APIs and reporting errors is critical.

So, keeping potentially sensitive API keys out of the hands of insecure clients is also important.

Error Handling with Express

If you’re interested in error handling, Express could also be a good option due to Express error handling middleware.

Express makes it easy to handle errors from external APIs in a standardized way. This is true as long as the errors are reported through next().

Check out an example of error handling middleware for the Twilio API.

const Twilio = require('twilio');
const app = require('express')();

const twilio = new Twilio(process.env.TWILIO_ACCOUNT_SID, process.env.TWILIO_AUTH_TOKEN);

async function run() {
  app.post('/sms', async function(req, res, next) {
    try {
      await twilio.messages.create({
        body: 'Hello',
        // From number is invalid, this will cause an error
        from: '+12015550123',
        to: '+5555555555'
      });
    } catch (error) {
      // Mark this error as a Twilio error, because the Twilio API doesn't
      // have a canonical error class
      error.isTwilio = true;
      return next(error);
    }
    res.json({ ok: 1 });
  });

  // Express error handler for handling twilio errors
  app.use(function(err, req, res, next) {
    if (err.isTwilio) {
      // Handle Twilio errors from all endpoints
      return res.status(err.status).json({ err: `Twilio error: ${err.message}` });
    }
    next(err);
  });

  // Catch-all error handler
  app.use(function(err, req, res, next) {
    return res.status(500).json({ err: err.message });
  });

  app.listen(3000);
}

run().catch(error => console.error(error.stack));

Moving Along

Since clients are increasingly specialized, APIs are becoming increasingly important. APIs provide consistency across different clients.

The 2013 dream of a single unified JavaScript client for mobile, browser, and desktop is not feasible for most companies.

Due to the prolific npm ecosystem, Node.js’ elegant concurrency model, and code sharing with browser, Electron, and React Native, a Node.js API written in Express is still the way to go.


  • Sign up for our private beta – your feedback helps prioritize our roadmap with the most value realized within the shortest amount of time
  • Learn about the inaugural feature set we’re striving for to make APIs repeatedly fast, easy and manageable as you evolve through the API lifecycle itself.
  • Sign up for the latest development on APIs and microservices.