Software applications and the development processes that create them have matured and become far more complex than they were just a few years ago. Web application architecture has also changed quite a bit and matured over the past two decades. Modern Web applications are a combination of a heavy top and bottom with a thin middle. Complexities live in the browser and server where, in the past, most complexities were isolated on the server. The client is a rich user experience composed of advanced CSS, JavaScript, and HTML techniques. Many server-side tasks have shifted to the client in the form of JavaScript libraries and functionality. At the same time, the way modern Web-serving platforms are architected have changed as well.
As Web technologies have matured, so have the ways both the server and client are architected. When the Web first became mainstream, we used large foundations like Classic ASP, Java, Coldfusion, or PHP to build data-driven Web sites. In the early 2000s, Microsoft introduced ASP.NET, which has grown and morphed several times with the original WebForms, followed by MVC and now vNext. In more recent years, the Web development community has also created other development stacks like Ruby and Node.
Even as we matured as developers, the server-side architecture browsers also changed. Browsers should be thought of as a thin, virtual-client application operating system. Over the past five to six years, HTML5, CSS3, and ECMAScript 5 and 6 have become widely supported by all browsers. The differences between browsers have almost gone away and developers are free to focus more on programming to a common standard.
Browsers should be thought of as a thin, virtual-client application operating system.
Just as programming techniques and platforms have evolved, so have the devices customers are using. We no longer live in a world dominated by desktops. Today, mobility is king. This has given rise to the mobile app and a certain level of usability for the end user. Two primary things users say they expect in applications are performance and easy navigation. For this article, I will expand upon performance, which means that customers want the application to load and respond instantly. Mobile devices have not created a new set of problems, but rather amplified the impact that performance issues have with client applications.
How Has the Web Server Architecture Changed?
I mentioned modern Web server platforms like Node and ASP.NET vNext. These platforms are designed to use a component-based architecture. Each platform uses a configurable processing pipeline where developers define middleware that processes each request and response. This gives developers and architects control over the way their application works. If you don't like a particular logging library, no problem. Just replace it with a couple of lines of code and redeploy.
Each platform uses a configurable processing pipeline where developers define middleware that processes each request and response. This gives developers and architects control over the way their applications work.
ASP.NET developers should be familiar with NuGet, a package manager for Visual Studio and .NET. Here, developers have packaged up their libraries and registered them with nuget.org. Developers have the option of using a command-line interface or Visual Studio's NuGet Package Manager to find, install, and update packages. These libraries are smaller, task-focused systems or utilities that enhance the application or development experience.
In the NodeJS world, developers rely on the node package manager to find and install modules. Modules are small, task-focused JavaScript libraries. You can search the https://www.npmjs.com/ repository to find modules and add them to your package.json configuration file. If you're not familiar with Node yet, I just introduced several new concepts. Let me take a moment to describe them.
Node is a platform built on top of the WebKit JavaScript engine and V8, and, of course, uses JavaScript as the programming language. Everything in the Node world is a module. The main Node module repository is npmjs.org. This is where developers register their public modules. When you install Node, you also have the npm command line interface or CLI installed. This is your gateway to install these modules.
You can open up a command line and manually install modules by using the npm install command:
>npm install underscore
Or you can use a package.json file and define all your application's module dependencies. A package.json file is a simple configuration file using JSON (JavaScript Object Notation) syntax to define information about your application. The Node package manager parses this file, looking for the dependencies and devDependencies objects. Npm reads the name and version of each dependency and installs the desired module from npm. Instead of executing a request for each module by hand, running npm install installs all the dependencies at once. See Listing 1.
Listing 1: An Example Package File
{
"name": "My Node JS Web Application",
"description": "a example of configuring a node/express based web site",
"version": "0.0.1",
"private": true,
"dependencies": {
"express": "4.x",
"body-parser": "~1.3.0",
"chai": "~1.9.1",
/* truncated for brevity */
"underscore": "^1.6.0"
},
"devDependencies": {
"grunt": "^0.4.5",
"grunt-cafe-mocha": "^0.1.12",
"grunt-contrib-cssmin": "^0.10.0",
/* truncated for brevity */
"zombie": "~2.0.0-alpha31"
}
}
To keep your project up-to-date with the latest module updates, you can run npm update from the command line.
The package file format comes from the CommonJS specification for packages . This specification defines various properties that can be included in a package file. Its purpose is to describe your application so that different applications can parse the information.
Visual Studio users can install the Node.js Tools for Visual Studio and get a similar experience for npm as the Nuget Package Manager, as you can see in Figure 3.
Task Runners
The concept of a package file is not limited to Node itself. Grunt and Gulp, popular task runners, or Web compilers as I like to call them, use this concept as well. Grunt is my tool of choice, so I'll focus on how it works. Both Grunt and Gulp are built on Node and use the package.json file to list the modules (small JavaScript libraries with a single purpose) that they use. Both also use a custom configuration file. Grunt uses a Gruntfile.js file.
The Gruntfile.js is a Node module that defines how Grunt should run. The file defines configurations for each Grunt module. Listing 2 is a simple Gruntfile.js that automatically runs QUnit tests, lints the application's JavaScript and bundle, and minifies the CSS and JavaScript for deployment. You can see how each module has a unique configuration, mostly defining source and destination files in Listing 2. Each module has a different set of possible configurations, so check each module's documentation.
Listing 2: Example Gruntfile.js
module.exports = function (grunt) {
require("matchdep").filterDev("grunt-*").forEach(grunt.loadNpmTasks);
// Project configuration.
grunt.initConfig({
pkg: grunt.file.readJSON('package.json'),
qunit: {
all: ['js/specs/**/*.html']
},
jshint: {
options: {
browser: true
},
files: ['Gruntfile.js', 'js/debug/*.js', 'js/libs/*.js']
},
cssmin: {
sitecss: {
options: {
banner: '/* My minified css file */'
},
files: {
'css/site.min.css': [
'css/dev/site.css'
/* truncated for brevity */
'css/dev/views/*.css',
]
}
}
},
uglify: {
options: {
compress: true
},
applib: {
src: [
'js/libs/dollarbill.min.js',
'js/libs/backpack.js',
'/* truncated to save space */
'js/dev/movie.app.bootstrap.js'
],
dest: 'js/applib.js'
}
}
});
// Default task.
grunt.registerTask("default", [jshint', 'qunit', "uglify", "cssmin"]);
};
Where can you find Grunt modules? Just like npm, Grunt.com maintains a repository of plugins (modules). There are currently almost 3400 Grunt plugins. This makes it a very rich ecosystem.
In Figure 4, you'll notice that the first plugins listed start with contrib. These are plugins maintained by the Grunt team and are part of a collection of over two dozen common plugins. Don't feel like these are the only ones you should use. I often use only two or three of these plugins and a handful of other community-contributed plugins, so you should feel free to experiment to find the best solution for your project. The odds are that you will find a plugin to solve just about anything you need.
Not only has the nature of Web server software changed, but so has the server's architecture. Modern Web server platform software is no longer large and monolithic. The modern Web server has become more of an extensible platform, open to customization with small, task-focused libraries to accomplish the exact tasks needed by the application and nothing more. This does not mean that developers and architects have more responsibility than before; modularity means they have more control than before. It also means that applications can be more easily configured for optimal performance and maintainability.
What About the Client?
This recent shift in capabilities from the server to the client has created a vast amount of churn and confusion among developers. A common misconception developers have is the assumption they must use a large god framework to build rich user experiences. These are libraries and frameworks like Angular, Backbone, and Ember. Angular is an all-inclusive framework. Backbone and Ember are composed of several large libraries. Although these libraries are popular, there are two key drawbacks: size and development approach.
This recent shift in capabilities from the server to the client has created a vast amount of churn and confusion among developers.
Data size has been well documented to be a cause of slow loading times. A high-performance website optimizes requests as much as possible. This means reducing the number of HTTP requests and the size of each request, which are two fundamental Web development rules. A number of independent studies show that performance is directly correlated to sales and conversions. A recent report by WebPerfromanceToday.com demonstrates the correlation between performance and online success (). Performance is also a search engine optimization factor. So every millisecond you can save is important. With the three example frameworks (Angular, Ember, and Backbone) weighing in between 111-341kb, your application starts with a disadvantage.
The modular approach allows you to better control the user experience and work closer with the browser's API to create an optimal experience.
OK, so you understand performance and how large libraries can affect performance, but what does “development approach” mean? A few years back, Lou Carbone gave a presentation at Mix where he coined the term “outside-in development.” I call it “user-first development.” JavaScript god frameworks appeal to developers by implementing patterns and practices that are popular and successful on the server, but they do not always lead to good user experiences when applied to the client. I feel the difference when I use a framework like Angular compared to a modular approach. The modular approach allows the developer to better control the user experience and work closer with the browser's API to create an optimal experience.
A large framework disadvantage can easily be demonstrated by their size. Large frameworks are designed to include features and functionality to cover just about every application scenario, not just the ones for your application. This means that the customer must download the entire framework when your application uses 10% of the framework's code. This is bad for performance, especially when the customer is on a mobile device using a cellular connection. Despite the popular notion that common libraries like jQuery and Angular will get cached because everyone uses them, it's not always true.
Developers assume that libraries and frameworks are cached because “everyone uses them” and they're referencing CDN hosted versions. But you can never guarantee that your customer has already cached your god framework of choice. These frameworks continually update and fragment. Over a two-week period this August, Angular added 11kb to its size. Often jQuery versions are referenced that are four and five years old. You're putting your trust into an environment that you don't control: other web sites.
Mobile devices purge cached resources more frequently than desktop computers. They do this because disk space and memory are limited. At one point, mobile Safari never cached any resource over 26kb. That was a harsh limit to honor. Fortunately, that limit seems to have been lifted, but the reliability of these caches is still spotty. I still believe that limiting resources to the 26kb limit is achievable and forces developers to use a more modular approach, ultimately producing a high-performance application. My experience has proven to me that an even better strategy is controlling my application's resources as much as possible and keeping my data footprint as small as possible. This not only improves performance, but makes development much simpler.
“...It's Baltimore, gentlemen; the gods will not save you.” –Chief Burrell, The Wire
I like to say, “It's the Web, developers and architects; the gods will not save you.”
If you've done any Web development in the past seven or so years, you probably know a little about the jQuery library and jQueryUI. These aren't frameworks but rather libraries. Because it has been around so long, jQuery has built up some pretty extensive scar tissue supporting old and obsolete browsers. At the same time, browsers have implemented native features that make many aspects of jQuery obsolete. Fortunately, tools like jQuery Builder () are available to make a customized and smaller version of jQuery possible. In addition, jQueryUI has long had the ability to create a custom version through the Download Builder (). This modular way is how larger libraries and frameworks should be offered to developers.
Despite being designed for all sorts of applications, you must still create your own extensions to the god frameworks. Although extensibility is good for any library, when the library is designed to meet so many needs, a large ecosystem can be symptomatic of a poorly designed library. Each one adds more and more weight to an already large application footprint.
What's a MicroJS Library?
A MicroJS library is a small JavaScript library with a single purpose; you've already seen me use a variation of this phrase several times in this article. Often these libraries are less than 5-6kb after minification and gzipping. This means they're small and agile. Just like the Node and .NET examples I gave earlier, MicroJS libraries can be composed to make a nice, thin application foundation. These applications load faster and foster a good modular development approach.
If you have kids, you've no doubt told them to only take what they can eat. The same concept applies to software development and is even more important for Web applications. MicroJS libraries let you order exactly what you need to make your application work and nothing more.
Another important MicroJS attribute is independence. Ember, Backbone - even Bootstrap to a degree - have hard dependencies on other libraries. For example, all three rely on jQuery. A good MicroJS library stands by itself with no dependencies. There are exceptions to the rule, but in general, any dependency is another small MicrojJS library. This approach is a good one because it keeps your application's overall dependencies in check, making it easier to manage.
Although the large god frameworks have a single authoritative source to locate them, MicroJS libraries are spread around the Web, making them sometimes harder to find. Like Node's npm or Grunt's plugin repository, there are a few good resources around the Internet to help developers locate libraries. The first one should be your favorite search engine. This is often my starting point for any library search. I often find myself searching for a specific task in JavaScript or even CSS and find possible libraries or, better yet, a blog post or Stackoverflow question. These posts often point me to a library that solves my problem. Sometimes the answer is as simple as a single function.
Next, I go to GitHub and perform a similar search. Again, I'm often able to find a suitable solution to my problem. You can clone the entire repository, which is what I often do, fork the repository, download the library's distributable version or just star or bookmark it for later. Like the search engine, I usually find several solutions to my problem and compare how each one works and decide what is best, using one of the libraries or, sometimes I create a similar one from scratch.
I also search through Node's npm modules. Often, the modules designed for the server were either derived from a client-side library or have a suitable client-side solution available. Right now, the most referenced Node module is Underscore (), which was first created as a client-side utility library. The Underscore library is designed for you to make a custom version.
Another source is MicroJS.com, a searchable repository of small JavaScript libraries. Type in what you're looking for and you'll most likely find a helpful, small library. The rules for a library to be included are simple, it must be less than 5kb minified and compressed. As you can see in the Figure 6, a search for “ajax” returned around two dozen libraries, the largest a whopping 4.5kb, but most less than 1.5kb.
The last place I look is bower.io. Bower is a library-management utility that uses Node, built by the folks at Twitter. Similar to npm, it can also be used to install libraries and frameworks at the command line or based on a config file. I've found the search functionality to be rather limited unless you already know the name of the library you want.
Designing a MicroJS Architecture
God frameworks appeal to developers because they provide a known architecture that they can follow to create applications. What does a modular approach look like? A modular approach provides the flexibility to create an architecture that best suits your needs.
An example of how I approach modular Web application architecture can been seen in the bootstrap.js file in the demo application for my High Performance Single Page Applications book. Here, various small libraries are instantiated and injected into other libraries as dependencies. I follow a pattern that relies on implied interfaces, which makes it possible for me to change libraries as I desire. Grunt and Gulp use a similar concept for their plugin models.
In the example in Listing 3, backpack, rqData, and other dependencies are instantiated and injected to other libraries. Movie is my instantiated application object and is injected into the SPA library. Some of these libraries I wrote myself, others abstract other third-party libraries. The rqData abstracts reqwest, which is a small AJAX library that I use on many projects. MustacheViewEngine abstracts MUSTACHE, which is a popular template library. I take advantage of JavaScript's dynamic nature to perform dependency injection. I prefer this form of dependency injection over other, often more complicated dependency injection libraries and techniques. It also helps keep my code footprint small.
Listing 3: Example Modular Application Dependency Injection & Instantiation
var movie = MovieApp({
services: {
"viewEngine": MustacheViewEngine(),
"dataProvider": movieData({
RottenTomatoes: RottenTomatoes({ data: rqData()}),
fakeTheaters: fakeTheaters()
})
}
});
_spa = spa({
"appContext": movie,
"bp": backpack(),
"defaultPage": "homeview",
"viewWrapper": "#main",
"viewTransition": "slide",
"defaultTitle": "Modern Web Movies"
});
If you have experience with Angular, you might see a pattern that you recognize: modularity. If I break down the above application to a diagram you will see a main framework called spa, and an application called movie. What you don't see in the code are the controllers and models as they are instantiated and destroyed as they're used. Each library or service used can be thought of as its own MicroJS library, an example of which is shown in Figure 7.
In Summary
Using a modular MicroJS approach keeps data footprints much smaller, improving application load and runtime experience. Most applications I build using small JavaScript and CSS libraries have around 50kb of JavaScript and less than 25kb of CSS total before compression. This is less than half the size of the smallest god framework's footprint, a size that doesn't include application-specific code.
Another benefit I find with a MicroJS approach is the ability to focus on my application-specific code. When using a god framework, you spend much of your time fussing with the framework's opinionated approach. For example, many Angular directives wrap functionality inside small UI components or worse, complicate these simple libraries.
There are tradeoffs with a small modular approach. Developers and architects do need to understand the platform, or the browser, better. A god framework abstracts the browser's details away, but at the same time you spend as much - if not more - time learning the details of any given god framework and still need to understand the browser's native API to troubleshoot problems. In the end, I find it's better to cut out the middle man and buy directly from the source. This is a clear advantage that a MicroJS library approach provides.
I hope you can see how small modular libraries are being used to build server-side applications, tools, and Web clients. I challenge you give it a try the next time you spin up a new Web project.
