Although I'm now a huge advocate of test-driven-development, I was not an immediate convert to the school of thought. I understood the necessity of unit testing:
- As a way to describe functionality to other members of the team
- As a way to protect against regressions in a production system
- As a way to test edge-cases in complex logic.
I did not, however, buy into a “test-first” approach when designing software. I had been using the same basic approach to designing software since I was 12 years old on GeoCities. TDD (Test-Driven Development) was strange and new.
It took experimentation, experience, and some good mentors to win me over. Ultimately, I realized that TDD helps me write better code.
It took experimentation, experience, and some good mentors to win me over. Ultimately, I realized that TDD helps me write better code because:
- If tests are written first, I'm forced to think about an ideal API independently from implementation details. This ultimately results in much cleaner APIs.
- Architectural flaws become apparent while writing tests, prior to implementation. For example, tightly coupled parts of a system are innately hard to test.
Developers within the Java, Ruby, and C# communities (to name a few) have embraced TDD for years; but the JavaScript community is just beginning down this road. Why is this?
JavaScript APIs, both in the browser and Node.js, tend to be event-driven. This makes testing more difficult; TDD requires a different mindset than testing synchronous logic. How can you make this challenge more manageable?
- Use TDD from the get-go. I find that TDD can help unravel the complexity that arises in an evented paradigm.
- Know your tools. JavaScript testing frameworks, such as Mocha, are designed to make testing asynchronous behavior manageable.
- Practice. I've been hacking in JavaScript since the late 90s; I still spend a lot of my time honing my skills on toy projects (such as the one outlined in this article).
Throughout this article, I build a program called Nostrabot, a Twitterbot that tracks up-to-the-minute news, posting the most interesting articles. Nostrabot is used to demonstrate the approach I take to TDD in JavaScript. It also provides an introduction to Node.js, and the Mocha testing framework.
Throughout this article, I build a program called Nostrabot, a Twitterbot that tracks up-to-the-minute news, posting the most interesting articles.
Creating a Skeleton Project
Before diving headlong into building a real-world application, I think it's useful to go over the structure I use for Node.js libraries. I've posted the skeleton of a project here: https://github.com/bcoe/node-mocha-skeleton.
Let's walk through the basic structure of this skeleton project:
- lib/ is where all of your applications code is stored. I approach my design in an OOP manner;
hello.js
contains aHello
class,banana.js
contains aBanana
class, etc. - test/ mocha runs any tests you place inside the test directory. Generally, there is a corresponding library file for each test, e.g.,
hello.js
has a matchinghello-test.js
. - bin/ files in the
bin
folder expose a command-line-interface to your application. Whennpm install -g
is run, these scripts are installed globally. - Package.json serves several functions: it provides meta-information surrounding your project, author, version, description, etc; it describes the directory structure of your project; it defines your project's dependencies (running
npm install
installs the dependencies described inpackage.json
). - Makefile is used to execute the mocha test framework. Running
npm test
looks for this Makefile, and uses it to execute Mocha's test suite. - README.md a great README file is an important part of any project; make sure you take some time to learn the ins and outs of markdown.
There are several third-party library dependencies that I frequently pull into my projects. These libraries are included in the package.json
of the skeleton project:
- Mocha a lightweight testing-framework for JavaScript. It provides both behavior-driven and test-driven interfaces; choose whichever you feel more comfortable with.
- Sinon is a stand-alone testing framework. I use it specifically for its stubbing functionality. It provides great constructs for mocking and asserting against objects.
- Underscore provides functional programming constructs: map, reduce, extend, etc. It really helps cleanup the logic surrounding array and object manipulation in JavaScript.
- Async provides useful constructs for simplifying the parallel execution of code.
- Optimist is a simple, powerful, command-line-argument parser. It's great for simplifying your
/bin
scripts.
The skeleton project detailed above provides the foundation for Nostrabot, the real-world example built throughout the rest of this article.
Roughing Out the Design
Before writing a single line of code, I jot down ideas in my README. This is a great place to: describe the overall goal of your project; outline what you think the public API should look like; and plan out the classes that will compose your application.
Before writing a single line of code, I jot down ideas in my README.
The idea for Nostrabot was to create a Twitterbot that monitors the RSS feeds of major news sources, and periodically posts the most alarming articles to Twitter. The following class structure jumped out at me:
- RSS-Reader this class handles loading in news articles from the various news sources.
- Tweeter this class handles formatting and posting the selected news articles to Twitter.
- Story-Chooser this class handles selecting the appropriate news articles for publication.
As you sit down and start writing code, this structure will change: more classes jump out at you, you'll think of better names, etc.
Phew, that was a lot of groundwork; let's start writing some code!
Writing the First Failing Test
At this point, you're faced with a paradox of choice, which class should be implemented first? Given the underlying goal (keeping classes as decoupled as possible), it doesn't really matter which we choose. I opted to implement the RSS-Reader class first:
- I created
rss-reader.js
in the/lib
folder. This holds all the logic necessary for grabbing and parsing RSS feeds. - I created
rss-reader-test.js
in the/test
folder. - I updated
index.js
, in the/lib
folder, importing the newRSS-Reader
class.
It is now time to write the first mocha test. I decided that an RSS reader should have a method called readAllFeeds()
. This method iterates over an array of RSS feeds, reads a list of articles from the news source, and returns an array of all of the combined stories. I structured my first test as shown in Listing 1.
Listing 1: rss-reader-test.js: a synchrnous test using Mocha and Sinon
var assert = require('assert'),
RSSReader = require('../lib').RSSReader,
sinon = require('sinon');
describe('RSSReader', function() {
describe('#readAllFeeds', function() {
it('should call #readFeed for each feed', function() {
// Create a new instance of an RSSReader
// passing in a set of feeds.
var rssreader = new RSSReader({
feeds: [
'http://example.com/foo.rss',
'http://example.com/bar.rss'
]
});
// mock the readFeed method using sinon.
rssreader.readFeed = sinon.spy();
rssreader.readAllFeeds();
// test that readFeed was called twice, using the
// assert module in Node's built-in API.
assert.equal(rssreader.readFeed.callCount, 2);
});
});
});
This first test introduces mocha, sinon, and assert. Let's look at it in detail:
- describe() is used to group together similar tests in Mocha. All of the tests for the
readAllFeeds()
method are enclosed withindescribe('#readAllFeeds')
. - It() describes a specific use-case for the method being tested. The assertions themselves are contained within this closure.
- rssreadr is a specific instance of our
RSS-Reader
class. You assert against methods on this object. - sinon.spy() Is used to mock the
readFeed()
method on rssreader. It's expected thatreadAllFeeds()
, when invoked, callsreadFeed()
once for each of the news sources. - assert.equal() assert raises an exception if the two arguments provided are not equal, causing the unit tests to fail.
At this point, when you run npm test
, the test fails because you've yet to write any code. Using a test-driven approach, you write tests before writing code; this helps you to think of the optimal API without getting weighed down by implementation details.
Using a test-driven approach, you write tests before writing code; this helps you to think of the optimal API without getting weighed down by implementation details.
Listing 2 includes the minimal amount of code I wrote, to satisfy the first test.
Listing 2: rss-reader.js: minimum code necessary to create passing test
var _ = require('underscore');
function RSSReader(opts) {
_.extend(this, {
feeds: [
'http://feeds.foxnews.com/foxnews/latest.rss'
]
}, opts);
}
RSSReader.prototype.readAllFeeds = function() {
var _this = this;
this.feeds.forEach(function(feed) {
_this.readFeed(feed);
});
};
RSSReader.prototype.readFeed = function() {};
At this point, when you run npm test
, everything is in the green. Great! But the code doesn't actually do much yet. I wanted readAllFeeds()
to return a list of articles from an RSS feed, so:
- I created a
/fixtures
directory where I stored the XML contents of several RSS feeds. - I did some research and found a library called Feedparser for parsing the RSS content.
It was at this point that I hit a bit of a hiccup. So far all of the code written was synchronous. The Feedparser interface was asynchronous. I would need to rework the RSS-Reader
tests, and the API, to handle this.
Writing Asynchronous Tests with Mocha
To indicate that a test is asynchronous in Mocha, you simply pass a callback as the first argument to the it()
method:
it('should be asynchronous', function(done) {
setTimeout(function() {
done();
}, 500);
});
Tests will time out, if done()
is not executed within a specified number of milliseconds (the default being two seconds).
It is time to rework readAllFeeds()
to handle an asynchronous paradigm. Rather than returning an array of articles, readAllFeeds()
now accepts a callback. This callback will, at some later time, be executed with a list of articles.
Before rewriting any code, I modified my unit tests to describe these new requirements as in Listing 3.
Listing 3: rss-reader-test.js: an asynchrnous test using Mocha and Sinon
var assert = require('assert'),
RSSReader = require('../lib').RSSReader,
sinon = require('sinon'),
fs = require('fs');
describe('RSSReader', function() {
describe('#readAllFeeds', function() {
it('should call #readFeed for each feed', function(done) {
var rssreader = new RSSReader({
feedUrls: [
'http://example.com/foo.rss',
'http://example.com/bar.rss'
]
}),
fakeNewsArticle = {
title: 'a fake article title',
desciption: 'a fake article description.'
};
// mock the readFeed method. Sinon now
// readFeed now takes a callback.
rssreader.readFeed = sinon.mock()
.callsArgWith(1, null, [fakeNewsArticle])
.twice();
rssreader.readAllFeeds(function(err, articles) {
assert.equal(articles.length, 2);
done();
});
});
});
});
Let's look at what's changed from the synchronous version of the same test:
- it() now accepts the argument
done
. Mocha times out if this is not executed. - readAllFeeds() now accepts a callback, this is executed with the combined articles loaded from all of the news sources.
- sinon.mock() continues to mock the
readFeed()
method. Now it executes a callback with a list of news articles rather than returning the array immediately.
Running the npm test
, you will find that tests are failing again. Nostrabot's code must also be modified to support these new asynchronous requirements as in Listing 4.
Listing 4: rss-reader.js: asynchronous impementation
var _ = require('underscore'),
FeedParser = require('feedparser'),
async = require('async');
function RSSReader(opts) {
_.extend(this, {
feedUrls: [
'http://feeds.foxnews.com/foxnews/latest.rss',
'http://rss.cnn.com/rss/cnn_world.rss'
]
}, opts);
}
RSSReader.prototype.readAllFeeds = function(callback) {
var _this = this,
readWork = [];
// Create read work for each of the feed URLs.
this.feedUrls.forEach(function(feedUrl) {
readWork.push(function(callback) {
_this.readFeed(feedUrl, callback);
});
});
// Use the async library to do
// all of the work in parallel.
async.parallel(
readWork,
function(err, articlesByFeed) {
var allArticles = [];
(articlesByFeed || []).forEach(function(articles) {
allArticles = allArticles.concat(articles);
});
// return the articles to the
// caller of readAllFeeds().callback(err, allArticles);
}
);
};
You've now seen examples of synchronous and asynchronous tests in action, and the process necessary to convert tests from a synchronous API to an asynchronous API. Where do you go from here?
Implementing Nostrabot
I've put all of the code for Nostrabot online, so that you can see what the finished product looks like. It's here: https://github.com/bcoe/nostrabot.
Implementing the remaining classes (the Tweeter and the Story-Chooser) followed a pattern identical to the one outlined for the RSS-Reader:
- Write an initial “it should” statement describing a method exposed in a class' API, e.g., the Story-Chooser's
chooseArticle()
method, which should pick the most alarming story from a list of news articles. - Implement a minimal solution that gets your tests passing.
- Build out the functionality further, ensuring that your tests continue passing.
- Find appropriate data to test against. For example, the XML content from an RSS feed, or the JSON payloads from curl requests to the Twitter API.
- Identify any necessary refactoring, e.g., our conversion of
readAllFeeds()
from a synchronous API to an asynchronous API.
Advanced Tips
Here are some useful tips that jumped out at me while implementing Nostrabot:
- Even when you're a JavaScript guru, asynchronous code is painful to test. Don't jump through hoops to use callbacks when returning a value suffices. As an example, the code for
tweeter.js
does not use a single callback. - Wrap third-party dependencies that are difficult to test in methods that can be mocked. In
rss-reader.js
, the logic to download an RSS feed over HTTP is pulled into a method called_getRawFeed()
, and mocked within tests. - Take your time and research third-party dependencies: create appropriate fixtures, look up projects on Github, and read the source code.
Final Thoughts
It took me a while to get the hang of it, but adopting a test-driven approach to writing JavaScript has been well worth it. At the end of the day, I find my code is cleaner, more stable, and I release bugs into production far less frequently.
Resources
- Sinon: https://sinonjs.org/ (provides detailed information about mocks, stubs, and (in general) writing unit tests using Sinon as a tool).
- Mocha: https://github.com/mochajs provides information about writing unit tests using Mocha.
- Nostrabot: https://github.com/bcoe/nostrabot, the toy project built throughout the article.