Node.js: Abusing Express 3.* to Enable Late Addition of Middleware

March 2nd, 2013 Permalink

One of the more entertaining aspects of Javascript is that - barring good encapsulation - you can reach in and tinker with everything. This also means that you, the developer, can dig yourself a far deeper hole than would otherwise be the case. So let us open this post by noting that public APIs exist for a reason, and that if you base critical code on manipulating the internals of someone else's package then you're creating a bunch of work for someone down the line. Internals change far more readily than APIs, and today's hack is tomorrow's technical debt - or more likely tomorrow's cascade of thrown errors.

So that said, onwards. At a high level the internals of the Express web application framework for Node.js are fairly straightforward. You set up by doing something like the following, in order:

var express = require("express");
var http = require("http");

// 1) Create the application
var app = express();

// 2) Configure the application. e.g.
app.enable("trust proxy");
app.set("parameter name", "parameter value");
// Etc.

// 3) Add middleware. e.g.
app.use(express.bodyParser());
// Etc.

// 4) Add routes.
app.get("/somepath", function (req, res, next) {
  // Send a page to the client here.
});
// Etc.

// 5) Add the 404 route.
app.all("*", function (req, res, next) {
  res.statusCode = 404;
  // Send a 404 page to the client here.
});

// 6) Start listening.
var server = http.createServer(app).listen(80);

Under the hood, Express is essentially just a stack of middleware functions and a separate stack of route functions. Every request runs through the middleware functions in the order they were added:

  • middleware A
  • middleware B
  • ...
  • middleware Z
  • Express router

As soon as the first route is added to the Express application, it adds its own router middleware function to the tail end of that stack. This feeds the request into the stack of route functions by matching on path - and doesn't pass the request on to any following middleware functions. What this means for the purposes of this post is that once the first route is added, any further middleware is added to the stack after the Express router and thus completely ignored.

This is usually not an issue - in most cases you control the horizontal and the vertical, and certainly the ordering of Express configuration on application start. Refrain from adding middleware and routes out of order and everything will work just fine. There are always those days, however, when you have to integrate someone else's application that encapsulates its own setup, days in which altering third party code to insert your middleware in the right place looks to be more painful than the options I'm about to present here. Software development is, if nothing else, all about weighing options and pain.

So let us say that you are in the situation of adding middleware late, after the server is up and running. This is actually pretty straightforward; if you are certain that no later middleware will be added, then you can just manipulate the Express stack of middleware functions directly to put yours exactly where it needs to be:

var myMiddleware = function (options) {
  return function (req, res, next) {
    // Middleware code goes here.
    next();
  };
}

// You could put it right up front to run first, but this isn't all that common -
// your middleware likely depends on other middleware running first.
app.stack.unshift(myMiddleware(options));

// More often you'll want to be the last item before the router middleware.
app.stack.splice(stack.length - 1, 0, myMiddleware(options));

How about the more general situation, however? Perhaps you have no control over when your middleware will be added - it might be first, last, too late, or somewhere in the middle. Yet it still needs to be in the right slot at the end of the day. In which case, you might try something like this:

var myMiddleware = function (options) {
  var middleware = function (req, res, next) {
    // Middleware code goes here.
    next();
  };
  middleware.isMyMiddleware = true;
  return middleware;
}

// Add the middleware the ordinary way. Perhaps this will even put in the
// right place, but given the way the world works, this is probably now following
// the Express router and thus ignored.
app.use(myMiddleware(options));

/**
 * This function will continually reposition the middleware until the Express
 * router has been added.
 */
function positionMiddleware () {
  // Assume the stack will be at least length 1, as our middleware has been put
  // there before this function is called.
  var desiredIndex = app.stack.length - 1;
  // Get the index of the router. Should be last, but if we can add
  // middleware out of sequence, so can other people.
  for (var index = 0; index < app.stack.length; index++) {
    // TODO: a less sketchy way of identifying the router middleware.
    if (app.stack[index].handle === app._router.middleware) {
      // If the router middleware is added, then routes have been
      // added, and thus no more middleware will be added.
      clearInterval(positionMiddlewareIntervalId);
      desiredIndex = index;
      break;
    }
  }

  // If it is already in the right place, then we're good.
  if(app.stack[desiredIndex].handle.isMyMiddleware) {
    return;
  }

  // Otherwise, remove from the present position.
  var stackItem;
  app.stack = app.stack.filter(function (element, index, array) {
    if (element.handle.isMyMiddleware) {
      stackItem = element;
    }
    return !element.handle.isMyMiddleware;
  });

  // And insert it into the correct position.
  app.stack.splice(desiredIndex, 0, stackItem);
}
// This will run until the Express router middleware is added, which happens
// when the first route is set.
var positionMiddlewareIntervalId = setInterval(positionMiddleware, 100);

If you think about what would happen if every application did this, you'll have one of the reasons why doing this sort of thing is a terrible idea. But it works.