28
Feb 11

nodechat.js – Using node.js, backbone.js, socket.io, and redis to make a real time chat app

UPDATE 5 – This tutorial refers to intermediate builds of nodechat, referenced by tags. These will not work quite right due to breaking changes in Socket.io 0.8.X. The latest commit in Github has been updated to be fully functional and I will continue to maintain it as long as there is interest. I do not plan to fix the intermediate tags, so please keep that in mind as you go.

UPDATE 4 – The hosted instance of nodechat is no longer. The last round of changes from socket.io and Google Chrome have broken it and I am not particularly interested in fixing it. IRC does the multi-user realtime chat job quite nicely!

UPDATE 3 – There is now a follow up tutorial dealing with user profiles and socket.io authentication: Nodechat.js continued – authentication, profiles, ponies, and a meaner socket.io.

UPDATE 2Joyent has kindly give me a permanent home for nodechat.js: http://nodechat.no.de/. I made some more FUN minor improvements. Continue being ware.

UPDATE – For the fun of it, here is the demo code hosted on a Joyent no.de smart machine: http://nodechat.no.de/ . Have fun. Beware.

Sorry folks. I know it has been awhile since I have posted anything. Even longer since I have posted something worth reading. Life, busy, etc. No excuses.

I’ll be better from now on, I promise. To start, here is something so cool it hurts my brain. I present nodechat.js!

nodechat.js is a simple, realtime chat app that leverages node.js, backbone.js, socket.IO, and redis. I wrote it as an exercise and I am sharing it because there are relatively few working examples using all these pieces together.

node.js

If you haven’t been reading about node.js, you might want to start. node.js describes itself as “Evented IO for V8 JavaScript”. Put simply, it is a server-side implementation of the V8 JavaScript engine. Yes, you read correctly. JavaScript on the server. Weird huh? That’s what I thought.

The big advantage of node.js is that it is easy to write fast code that scales. This is partly because code is written in a functional language (JavaScript) which nicely skirts the issue of side effects and partly because node.js does (almost) entirely non-blocking I/O. node.js uses an event loop and callbacks to manage this feat.

With node.js, we can eschew the use of a webserver altogether while writing apps that scale better. I can’t say I am sorry to stop fussing around with IIS or Apache configs.

node.js has already attracted a vibrant community of developers who are cranking out new packages at a frightening rate. More on this later.

Intrigued? You should be. You should go read more about it after you finish my tutorial. :)

backbone.js

If you haven’t been reading about backbone.js, you might want to start (heh, see what I did there?). Backbone is a JavaScript MVC framework. It allows you to stop worrying about the DOM and to start focusing on clean separation of concerns and good application design. If you are anything like me, it will save you from horrible, buggy, spaghetti JS code sprinkled all over your app.

Backbone can stand alone or work with any web framework or service that can supply it a RESTful JSON api. Recently, I have been playing around with Django-Tastypie to supply the api. When I started reading about node.js, I figured there must be a way to combine them. And The Google told me there was (thanks Google!): Henrik Joreteg of &yet recently posted some code providing me exactly what I needed to squish the two together.

Henrik did all the heavy lifting, but he did not post a full working application. I thought it would be helpful to those coming after me if I shared mine.

Socket.IO

Socket.IO is the last critical piece in this toolchain. Socket.IO abstracts away the pain of providing realtime connections to almost any browser. It will detect the capabilities of the client and use a variety of transports to facilitate a connection. All we have to do is handle the messaging.

Install Stuff

We are going to use a few more components to round out the toolchain: redis (a non-plain key-value store), npm (node.js package manager), and a few node.js packages. First off, let’s get everything installed if you haven’t already:

Now sprinkle in a few Node.js packages with npm:

  • sudo npm install socket.io
  • sudo npm install express
  • sudo npm install jade
  • sudo npm install underscore
  • sudo npm install backbone
  • sudo npm install redis

Assuming nothing blew up, we are ready to write some code.

nodechat.js

If you are totally new to Node.js and Socket.IO, I recommend walking through Adam Coffman’s excellent tutorial – Getting your feet wet with node.js and socket.io – Part 1 which I used as a jumping off point for nodechat.js. By the end of the tutorial, you should have a working Node.js instance which broadcasts the number of current clients each time a client connects/disconnects.

structure

Create the following subdirectories in your project:

  • controllers
  • lib
  • models
  • views

models.js

Models first! Create a new file in the models directory named models.js. This is where we will store the backbone.js models that are going to be used by our server AND our client code.

(function () {
    var server = false, models;
    if (typeof exports !== 'undefined') {
        _ = require('underscore')._;
        Backbone = require('backbone');

        models = exports;
        server = true;
    } else {
        models = this.models = {};
    }

Here is where we start to see the Henrik magic. Node.js implements the CommonJS specification to make it easy to write modules. Since we are want our clients and our server to see the same models, we need to act like a .js include file if we are getting referenced on the client or a CommonJS module if we are being loaded by Node.js.

models.ChatEntry = Backbone.Model.extend({});

models.ClientCountModel = Backbone.Model.extend({
    defaults: {
        "clients": 0
    },

    updateClients: function(clients){
        this.set({clients: clients});
    }
});

models.NodeChatModel = Backbone.Model.extend({
    defaults: {
        "clientId": 0
    },

    initialize: function() {
        this.chats = new models.ChatCollection(); 
    }
});

models.ChatCollection = Backbone.Collection.extend({
    model: models.ChatEntry
});

The models are simple enough. We have a single top level model that has a collection of simple chat models. We also have a model for keeping track of how many clients are connected. This is not part of the top level model because we calculate the value at runtime and do not need to persist it.

Backbone.Model.prototype.xport = function (opt) {
    var result = {},
    settings = _({recurse: true}).extend(opt || {});

    function process(targetObj, source) {
        targetObj.id = source.id || null;
        targetObj.cid = source.cid || null;
        targetObj.attrs = source.toJSON();
        _.each(source, function (value, key) {
        // since models store a reference to their collection
        // we need to make sure we don't create a circular refrence
            if (settings.recurse) {
              if (key !== 'collection' && source[key] instanceof Backbone.Collection) {
                targetObj.collections = targetObj.collections || {};
                targetObj.collections[key] = {};
                targetObj.collections[key].models = [];
                targetObj.collections[key].id = source[key].id || null;
                _.each(source[key].models, function (value, index) {
                  process(targetObj.collections[key].models[index] = {}, value);
                });
              } else if (source[key] instanceof Backbone.Model) {
                targetObj.models = targetObj.models || {};
                process(targetObj.models[key] = {}, value);
              }
           }
        });
    }

    process(result, this);

    return JSON.stringify(result);
};

Backbone.Model.prototype.mport = function (data, silent) {
    function process(targetObj, data) {
        targetObj.id = data.id || null;
        targetObj.set(data.attrs, {silent: silent});
        // loop through each collection
        if (data.collections) {
          _.each(data.collections, function (collection, name) {
            targetObj[name].id = collection.id;
            _.each(collection.models, function (modelData, index) {
              var newObj = targetObj[name]._add({}, {silent: silent});
              process(newObj, modelData);
            });
          });
        }

        if (data.models) {
            _.each(data.models, function (modelData, name) {
                process(targetObj[name], modelData);
            });
        }
    }

    process(this, JSON.parse(data));

    return this;
};

This is the crux of what makes it possible for us to reuse the models. Basically, we need to recurse through the models’ object graphs and collapse them into a form that can be serialized and sent over the wire. All the “mad props” for this code must go to Henrik. I made a few small changes. I chose to extend the Backbone.Model prototype instead of using a base model, and I made them take in and spit out json strings instead of partially collapsed object graphs.

If you are cutting and pasting this code, don’t forget to end this file with:

})()

to complete the closure.

views/index.jade

Our jade template looks as follows:

!!! 5
html(lang="en")
  head
    title nodechat
    script(type="text/javascript", src="/lib/jquery-1.5.1.js")
    script(type="text/javascript", src="/lib/underscore.js")
    script(type="text/javascript", src="/lib/backbone.js")
    script(type="text/javascript", src="http://cdn.socket.io/stable/socket.io.js")
    script(type="text/javascript", src="/models/models.js")
    script(type="text/javascript", src="/controllers/controllers.js")
    script(type="text/javascript", src="/views/views.js")
      body
    #heading
      h1 nodechat
    #content
      p 
        | connected clients
        span#client_count 0
      p
        | Fun Chat Messages
        ul#chat_list

      form(method="post", action="#", onsubmit="return false")#messageForm
        p
          label Name:
            input(type='text', name='user_name')
        p
          label Message:
            input(type='text', name='message')
            input(type='submit', value='send')

    #footer

The template has the links to all our client-side libraries (yes, you need to get these separately from the npm modules) and our backbone.js controllers, views, and models. We also stub out some html elements for our views to reference and some input fields to handle the chatting.

views.js

In the views directory, create a file named views.js and add to it the following:

var ChatView = Backbone.View.extend({
    tagName: 'li',

    initialize: function(options) {
        _.bindAll(this, 'render');
        this.model.bind('all', this.render);
    },

    render: function() {
        $(this.el).html(this.model.get("name") + ": " + this.model.get("text"));
        return this;
    }
});

ChatView works with ChatEntry. It tells it how to render itself and nothing else. This view expects its bound model to supply two fields: name and text.

var ClientCountView = Backbone.View.extend({
    initialize: function(options) {
        _.bindAll(this, 'render');
        this.model.bind('all', this.render);
    },

    render: function() {
        this.el.html(this.model.get("clients"));
        return this;
    }
});

ClientCountView just renders a div with the client count.

var NodeChatView = Backbone.View.extend({
    initialize: function(options) {
        this.model.chats.bind('add', this.addChat);
        this.socket = options.socket;
        this.clientCountView = new ClientCountView({model: new models.ClientCountModel(), el: $('#client_count')});
    }

    , events: {
        "submit #messageForm" : "sendMessage"
    }

    , addChat: function(chat) {
        var view = new ChatView({model: chat});
        $('#chat_list').append(view.render().el);
    }

    , msgReceived: function(message){
        switch(message.event) {
            case 'initial':
                this.model.mport(message.data);
                break;
            case 'chat':
                var newChatEntry = new models.ChatEntry();
                newChatEntry.mport(message.data);
                this.model.chats.add(newChatEntry);
                break;
            case 'update':
                this.clientCountView.model.updateClients(message.clients);
                break;
        }
    }

    , sendMessage: function(){
        var inputField = $('input[name=message]');
        var nameField = $('input[name=user_name]');
        var chatEntry = new models.ChatEntry({name: nameField.val(), text: inputField.val()});
        this.socket.send(chatEntry.xport());
        inputField.val('');
    }
});

NodeChatView does most of the heavy lifting on the client side. The most interesting pieces here are the msgReceived and sendMessage functions. msgReceived handles three types of message events. In the ‘initial’ case, it expects to get a json serialized NodeChatView model (with the historical chat data). The ‘chat’ event signals that it should expect a single, serialized ChatEntry model.The ‘update’ event just expects a new number for the current client count.

sendMessage grabs the values from the input fields, creates a new ChatEntry model, serializes it, and sends it through the socket. This implementation works because we never render our own chat messages, but if we were dealing with bigger, more complex objects, we might consider two way syncing of models.

controller.js

Now in the controller directory, create a new file called controller.js. It looks like this:

var NodeChatController = {
    init: function() {
        this.socket = new io.Socket(null, {port: 8000});
        var mysocket = this.socket;

        this.model = new models.NodeChatModel();
        this.view = new NodeChatView({model: this.model, socket: this.socket, el: $('#content')});
        var view = this.view;

        this.socket.on('message', function(msg) {view.msgReceived(msg)});
        this.socket.connect();

        this.view.render();

        return this;
    }
};

The controller is responsible for opening up the socket, creating the initial models, and squishing them into corresponding views.

Finally, we need to load the controller to kick things off. This can live in the controller.js or its own file if you prefer:

$(document).ready(function () {
    window.app = NodeChatController.init({});
});

core.js

Now we switch to the server side of things. Create a new file named core.js. This is where our Node.js code will live. I’ll go through it a chunk at a time. *I have left out any error handling for the sake of clarity. Grab the source from github to get the full picture.

var app = require('express').createServer()
    , jade = require('jade')
    , socket = require('socket.io').listen(app)
    , _ = require('underscore')._
    , Backbone = require('backbone')
    , redis = require('redis')
    , rc = redis.createClient()
    , models = require('./models/models');

Node.js uses require(‘somelib’) to include external libraries. Here we are including all the libraries we need and instantiating any needed objects.

app.set('view engine', 'jade');
app.set('view options', {layout: false});

app.get('/*.(js|css)', function(req, res){
    res.sendfile('./'+req.url);
});

app.get('/', function(req, res){
    res.render('index');
});

This is basically the same as Adam’s tutorial, except I am not using the public/ directory for my static files.

var activeClients = 0;
var nodeChatModel = new models.NodeChatModel();

Ok, now things are getting interesting. Remember above, where we required the (‘./models/models’)? That tells Node.js to include our local models.js file. The same one we just used for our client side code! We have all of our models available for use in this context.

rc.lrange('chatentries', -10, -1, function(err, data) {
    if (data) {
        _.each(data, function(jsonChat) {
            var chat = new models.ChatEntry();
            chat.mport(jsonChat);
            nodeChatModel.chats.add(chat);
        });

        console.log('Revived ' + nodeChatModel.chats.length + ' chats');
    }
    else {
        console.log('No data returned for key');
    }
});

Now redis comes into play. This is probably needlessly complicated for the chat demo, but I thought it would be fun to persist and reload the serialized models. To do that, we use a redis list. When nodechat.js starts up, it checks for historical chat data in the redis instance. If if finds them, it rehydrates them into memory.

socket.on('connection', function(client){
    activeClients += 1;
    client.on('disconnect', function(){clientDisconnect(client)});
    client.on('message', function(msg){chatMessage(client, socket, msg)});

    client.send({
        event: 'initial',
        data: nodeChatModel.xport()
    });

    socket.broadcast({
        event: 'update',
        clients: activeClients
    });
});

Remember, Node.js is event driven. Here we tell it to react to connection events by incrementing the active client count and binding more client specific events. We also send over an ‘initial’ message with the current state of the server’s nodeChatModel (e.g. the chat history). Finally, we broadcast the ‘update’ message to any listening clients (including the one that just connected) to update their client counts.

function chatMessage(client, socket, msg){
    var chat = new models.ChatEntry();
    chat.mport(msg);

    rc.incr('next.chatentry.id', function(err, newId) {
        chat.set({id: newId});
        nodeChatModel.chats.add(chat);

        console.log('(' + client.sessionId + ') ' + chat.get('id') + ' ' + chat.get('name') + ': ' + chat.get('text');

        rc.rpush('chatentries', chat.xport(), redis.print);
        rc.bgsave();

        socket.broadcast({
            event: 'chat',
            data:chat.xport()
        }); 
    }); 
}

Messages received over the socket are handled using this callback. We de-serialize the ChatEntry model, generate a unique identifier and then push it onto our list in redis. Finally, we broadcast the chat back out to all the clients (again including the one that just sent it).

function clientDisconnect(client) {
    activeClients -= 1;
    client.broadcast({clients:activeClients})
}

Here we just handle decrementing the client count.

app.listen(8000)

And finally, we tell our app to listen on port 8000.

Does it work?

That’s a lot of code and lot of jabber in between it. If you are having issues (or are lazy like me), then go grab the source from github.

UPDATE Make sure you grab the v0.1 tag. I have been working on a follow up tutorial and I have some half-baked code in there right now… it works, just needs some extra love on the redis side. Shoot me an email or tweet @jdslatts if you can’t wait :)

To make it work:

  1. Start redis: redis-server
  2. Start node: node core.js
  3. Fire up browser and go to http://localhost:8000/
  4. Optional: Fire up another tab and go to http://localhost:8000/ so you can talk to yourself
  5. Engage in fun, family friendly chat!

Well, that about wraps it up for today. I have a few more fun ideas for nodechat.js and if I get around to them, I’ll post a follow up or two. In the meantime, I am absolutely in love with this stack. I intend on using it for my for real stuff real soon.

UPDATE – Feeling ambitious? Head on over here for round two: Nodechat.js continued – authentication, profiles, ponies, and a meaner socket.io.

If you have any questions, feel free to email me at comments@fzysqr.com or stop by nodechat.

-jslatts