Node.js and Forever as a Service: Simple Upstart and Init Scripts for Ubuntu

February 9th, 2013 Permalink

Forever is a useful tool for running a Node.js process with monitoring; it can be set to restart a failed process, and has a few other helpful features along the same lines. In this post you'll find a couple of exceedingly simple scripts for running a Node.js process as a service on Ubuntu using Forever.

One thing I've found in the course of building and tinkering with Node.js is that there's a lot to be said for explicitly setting all the necessary environment variables in your service script. That way the same basic script will serve just fine for locally built versus package manager installations of Node.js, and for code that moves from being a local development copy to a package installed via NPM. Just change the relevant environment variables in your service script as appropriate, and it will remain perfectly clear from the contents of the script as to the circumstances under which it will work. No more puzzling over whether the node binary is in PATH, or what NODE_PATH is set to when Forever forks its child process.

Install Forever Globally

Forever must be installed globally via NPM for the example scripts to work, such that its binary is in the same directory as the Node.js binary:

npm -g install forever

Upstart Script

Upstart is an alternative to using standard issue init scripts and is installed by default on Ubuntu. A script resides in the /etc/init folder and has a .conf file extension, e.g. /etc/init/my-application.conf. Run or check on the script and its process with the following commands:

start my-application
status my-application
restart my-application
stop my-application

Most upstart scripts are set to automatically start and stop for specific run levels in the same way as init scripts. Here is an example Upstart script for a Node.js service running under Forever:

#!upstart
#
# An example upstart script for running a Node.js process as a service
# using Forever as the process monitor. For more configuration options
# associated with Forever, see: https://github.com/nodejitsu/forever
#
# You will need to set the environment variables noted below to conform to
# your use case, and should change the description.
#
description "Example upstart script for a Node.js process"

start on startup
stop on shutdown

# This line is needed so that Upstart reports the pid of the Node.js process
# started by Forever rather than Forever's pid.
expect fork

# The following environment variables must be set so as to define where Node.js
# and Forever binaries and the Node.js source code can be found.
#
# The example environment variables below assume that Node.js is installed by
# building from source with the standard settings.
#
# It should be easy enough to adapt to the paths to be appropriate to a package
# installation, but note that the packages available in the default repositories
# are far behind the times. Most users will be  building from source to get a
# more recent Node.js version.
#
# The full path to the directory containing the node and forever binaries.
# env NODE_BIN_DIR="/usr/local/bin"
# Set the NODE_PATH to the Node.js main node_modules directory.
# env NODE_PATH="/usr/local/lib/node_modules"
# The application startup Javascript file path.
# env APPLICATION_PATH="/home/node/my-application/start-my-application.js"
# Process ID file path.
# env PIDFILE="/var/run/my-application.pid"
# Log file path.
# env LOG="/var/log/my-application.log"
# Forever settings to prevent the application spinning if it fails on launch.
# env MIN_UPTIME="5000"
# env SPIN_SLEEP_TIME="2000"

env NODE_BIN_DIR=""
env NODE_PATH=""
env APPLICATION_PATH=""
env PIDFILE=""
env LOG=""
env MIN_UPTIME=""
env SPIN_SLEEP_TIME=""

script
    # Add the node executables to the path, which includes Forever if it is
    # installed globally, which it should be.
    PATH=$NODE_BIN_DIR:$PATH
    # The minUptime and spinSleepTime settings stop Forever from thrashing if
    # the application fails immediately on launch. This is generally necessary
    # to avoid loading development servers to the point of failure every time
    # someone makes an error in application initialization code, or bringing
    # down production servers the same way if a database or other critical
    # service suddenly becomes inaccessible.
    exec forever \
      --pidFile $PIDFILE \
      -a \
      -l $LOG \
      --minUptime $MIN_UPTIME \
      --spinSleepTime $SPIN_SLEEP_TIME \
      start $APPLICATION_PATH
end script

pre-stop script
    # Add the node executables to the path.
    PATH=$NODE_BIN_DIR:$PATH
    # Here we're using the pre-stop script to stop the Node.js application
    # process so that Forever is given a chance to do its thing and tidy up
    # its data. Note that doing it this way means that each application that
    # runs under Forever must have a different start file name, regardless of
    # which directory it is in.
    exec forever stop $APPLICATION_PATH
end script

Init Script

Init scripts hopefully need no explanation. Scripts reside in /etc/init.d with root ownership and executable permissions. e.g.:

sudo su
cp my-application-script /etc/init.d/my-application
chmod a+x /etc/init.d/my-application

After putting the script in place, you should update the system service definitions:

update-rc.d my-application defaults

Run or check on the script and its process with the following commands:

service my-application start
service my-application status
service my-application restart
service my-application stop

Here is an example init script for a Node.js service running under Forever:

#!/bin/bash
#
# An init.d script for running a Node.js process as a service using Forever as
# the process monitor. For more configuration options associated with Forever,
# see: https://github.com/nodejitsu/forever
#
# This was written for Debian distributions such as Ubuntu, but should still
# work on RedHat, Fedora, or other RPM-based distributions, since none of the
# built-in service functions are used. So information is provided for both.
#
### BEGIN INIT INFO
# Provides:             my-application
# Required-Start:       $syslog $remote_fs
# Required-Stop:        $syslog $remote_fs
# Should-Start:         $local_fs
# Should-Stop:          $local_fs
# Default-Start:        2 3 4 5
# Default-Stop:         0 1 6
# Short-Description:    My Application
# Description:          My Application
### END INIT INFO
#
### BEGIN CHKCONFIG INFO
# chkconfig: 2345 55 25
# description: My Application
### END CHKCONFIG INFO
#
# Based on:
# https://gist.github.com/3748766
# https://github.com/hectorcorrea/hectorcorrea.com/blob/master/etc/forever-initd-hectorcorrea.sh
# https://www.exratione.com/2011/07/running-a-nodejs-server-as-a-service-using-forever/
#
# The example environment variables below assume that Node.js is installed by
# building from source with the standard settings.
#
# It should be easy enough to adapt to the paths to be appropriate to a package
# installation, but note that the packages available in the default repositories
# are far behind the times. Most users will be building from source to get a
# suitably recent Node.js version.
#
# An application name to display in echo text.
# NAME="My Application"
# The full path to the directory containing the node and forever binaries.
# NODE_BIN_DIR="/usr/local/node/bin"
# Set the NODE_PATH to the Node.js main node_modules directory.
# NODE_PATH="/usr/local/lib/node_modules"
# The application startup Javascript file path.
# APPLICATION_PATH="/home/user/my-application/start-my-application.js"
# Process ID file path.
# PIDFILE="/var/run/my-application.pid"
# Log file path.
# LOGFILE="/var/log/my-application.log"
# Forever settings to prevent the application spinning if it fails on launch.
# MIN_UPTIME="5000"
# SPIN_SLEEP_TIME="2000"

NAME=""
NODE_BIN_DIR=""
NODE_PATH=""
APPLICATION_PATH=""
PIDFILE=""
LOGFILE=""
MIN_UPTIME=""
SPIN_SLEEP_TIME=""

# Add node to the path for situations in which the environment is passed.
PATH=$NODE_BIN_DIR:$PATH
# Export all environment variables that must be visible for the Node.js
# application process forked by Forever. It will not see any of the other
# variables defined in this script.
export NODE_PATH=$NODE_PATH

start() {
    echo "Starting $NAME"
    # We're calling forever directly without using start-stop-daemon for the
    # sake of simplicity when it comes to environment, and because this way
    # the script will work whether it is executed directly or via the service
    # utility.
    #
    # The minUptime and spinSleepTime settings stop Forever from thrashing if
    # the application fails immediately on launch. This is generally necessary to
    # avoid loading development servers to the point of failure every time
    # someone makes an error in application initialization code, or bringing down
    # production servers the same way if a database or other critical service
    # suddenly becomes inaccessible.
    #
    # The pidfile contains the child process pid, not the forever process pid.
    # We're only using it as a marker for whether or not the process is
    # running.
    #
    # Note that redirecting the output to /dev/null (or anywhere) is necessary
    # to make this script work if provisioning the service via Chef.
    forever \
      --pidFile $PIDFILE \
      -a \
      -l $LOGFILE \
      --minUptime $MIN_UPTIME \
      --spinSleepTime $SPIN_SLEEP_TIME \
      start $APPLICATION_PATH 2>&1 > /dev/null &
    RETVAL=$?
}

stop() {
    if [ -f $PIDFILE ]; then
        echo "Shutting down $NAME"
        # Tell Forever to stop the process.
        forever stop $APPLICATION_PATH 2>&1 > /dev/null
        # Get rid of the pidfile, since Forever won't do that.
        rm -f $PIDFILE
        RETVAL=$?
    else
        echo "$NAME is not running."
        RETVAL=0
    fi
}

restart() {
    stop
    start
}

status() {
    # On Ubuntu this isn't even necessary. To find out whether the service is
    # running, use "service my-application status" which bypasses this script
    # entirely provided you used the service utility to start the process.
    #
    # The commented line below is the obvious way of checking whether or not a
    # process is currently running via Forever, but in recent Forever versions
    # when the service is started during Chef provisioning a dead pipe is left
    # behind somewhere and that causes an EPIPE exception to be thrown.
    # forever list | grep -q "$APPLICATION_PATH"
    #
    # So instead we add an extra layer of indirection with this to bypass that
    # issue.
    echo `forever list` | grep -q "$APPLICATION_PATH"
    if [ "$?" -eq "0" ]; then
        echo "$NAME is running."
        RETVAL=0
    else
        echo "$NAME is not running."
        RETVAL=3
    fi
}

case "$1" in
    start)
        start
        ;;
    stop)
        stop
        ;;
    status)
        status
        ;;
    restart)
        restart
        ;;
    *)
        echo "Usage: {start|stop|status|restart}"
        exit 1
        ;;
esac
exit $RETVAL