For developers, the Web has been in a constant state of change. Starting with static Web pages, developers quickly started building dynamic websites using a combination of server-side- and client-side-rendered content. Arguably, this reached an apex with the advent of Single Page Applications (SPAs). The beauty of building Web applications is how far they can reach. Today, it's a simple process to bring a website online with the capability to reach literally billions of eyeballs.
The downside of “pure” websites is their inability to create a truly rich application that can be encompassed by native applications. That's because 10 years ago, the Web landscape changed radically. What happened 10 years ago? The iPhone happened (along with the Android a short time later). The creation of the iPhone demonstrated to users that portable and rich applications were possible using the Web as a backbone. Although native applications have true advantages over Web applications, they aren't without their disadvantages. They're complex to build, must be built for multiple platforms, are more complex to deploy (because of app stores), and don't have “reach” right out of the box. Figure 1 contrasts the reach of a traditional website versus the capabilities of a native app on the Web.
The question is: “Can we build applications with both reach across the breadth of the Internet and capabilities similar to native applications?” It's my opinion that the answer to this question is yes. A brand-new way of building Web apps has come to the forefront. This concept is known as a Progressive Web Application (PWA). PWAs combine reach, flexibility, lower development costs, lower deployment costs (no app store hassle), and powerful features that were previously limited to native applications. Figure 2 demonstrates how PWAs move the bar between capability and reach.
The nice thing about PWAs is that it's not some “pie in the sky” technology. A number of recognizable brands have already jumped on the PWA train. These companies include Starbucks, Lyft, The Weather Channel, The Washington Post, Twitter, and loads of other big names. So how do you jump on the train yourself?
You've come to the right place. Let's take a look at how to build PWAs.
PWA Basics
The promise of PWAs is that they're fast, reliable, secure, engaging, and integrated. There are at least three primary requirements that your website must meet to be considered a PWA. It must be:
- Served via HTTPS
- A valid Web manifest
- A registered service worker with a fetch event handler
These are the big three, although there are many more, and more subtle, requirements to truly take advantage of the PWA spirit. The browser vendors behind PWAs are hoping that we'll soon start delivering Web experiences that implement a battery of best practices.
“PWAs are a new level of thinking about the quality of your user experience.” –Chrome's Chris Wilson
Service workers provide an extensible mechanism to enable many platform features, like push notifications and background sync. Browsers also trigger Add to Homescreen experiences that make PWAs look and behave like native applications with a first-class presence. (I'll explain more about Add to Homescreen in a minute.)
“Homescreen” means just what you think it means: a first screen that leads into the rest of the website's pages.
Microsoft accepts PWAs to the Windows Store and gives them equal status to Universal Windows Platform (UWP) applications. They're even using the Bing spider data (an SEO crawler) to identify PWAs. If the PWA meets a certain quality level, they automatically submit it to the store.
So far, Microsoft has identified 1.5 million PWAs to include in the store.
They've been accepting hosted Web apps to the store since Windows 10 was released in 2015. Up to this point, they were called Hosted Web Apps. They have similar requirements, except for a service worker. Now that Edge supports service workers, Microsoft is changing the name to PWA to sync with the rest of the industry.
Secure by Default
PWAs must be served using HTTPS. This is part of a bigger trend to make the Web secure by default. Many new and existing APIs, like geolocation (see Figure 3), Web payments, and log-in forms are being gated behind HTTPS. Service workers and HTTP/2 require TLS certificates before they are enabled.
Browsers are getting more aggressive with visual cues to users if a site isn't secure. For example, most browsers now detect credit card numbers and password fields. If present, they make it very apparent to the user if the site isn't served via HTTPS (see Figure 4).
Browsers are getting more aggressive in visually indicating to users that a website isn't secure. This summer, Chrome 68 will display a “not secure” message in the address bar for any site using HTTP. In the near future, any website served via HTTP and Firefox will show a visible insecure warning to the consumer. Each browser has a road map to increase insecure warning visibility. I advise you to upgrade now, rather than wait, as this alone will severely limit your ability to function online in the near future.
Search engines also use HTTP as a ranking signal and HTTPS as a positive ranking signal. Combining search engine ranking with on-screen warnings causes business stakeholders to aggressively upgrade external and internal sites to HTTPS. Today, over 60% of the top 100,000 websites are served via HTTPS and the number grows every day.
If you don't believe me, look at the top 100 search results the next time you perform a search. It's becoming more unusual to see a page served via HTTP.
In the past, HTTPS has been an unfortunate burden for the Web. SSL certificates have been cost prohibitive and a technical challenge for the masses. The vast majority of websites are hosted on a shared server, limiting how much control site owners have over managing TLS certificates. If HTTPS was supported, it's been a premium upgrade, often more than the cost of the content hosting. See Figure 5 for how things have changed in only the last three years.
Those barriers have been removed in recent years. Let's Encrypt (https://letsencrypt.org) alone has issued over 60 million free SSL certificates. And cloud service providers like AWS and Azure offer free SSL to their CDN customers.
The technical barriers are also being removed. When I add an AWS SSL certificate to a website, it takes me about 30 seconds, and most of that time is waiting on an e-mail to confirm the request.
A big reason behind the rise in HTTPS is that WordPress.com converted all of their customers to HTTPS over a year ago. This was a big chunk of the Web. This forced migration ensured that their customer's sites can take advantage of all the platform features the Web offers while providing a safe experience to visitors.
Using HTTP/2 for Performance
If you're worried about HTTPS causing performance issues, you can stop. HTTP/2 is the latest version of HTTP, correcting many performance limitations of HTTP/1.1. We have used HTTP 1.1 for almost two decades with only minor enhancements. To counter many inherent performance issues, we created many performance hacks, like bundling.
HTTP/2 features request multiplexing, header compression, and much more. Many of the performance hacks, like domain sharding and bundling are now anti-patterns. HTTP/2 eliminated the need for these hacks by focusing on optimizing file and packet delivery.
Like service workers, HTTP/2 requires HTTPS. The good news is that any good CDN provider has HTTP/2 enabled by default and most make TLS easy to implement. This means that everyone should have support for HTTP/2.
The Weather Channel upgraded to HTTP/2 and saw a significant increase in their Web performance metrics. The increased performance increased their consumer engagement and made their content more consumable on mobile devices.
“When we launched [HTTPS], we saw an average of a 50ms hit for negotiation...it was more than offset when we activated HTTP/2...a month later we saw a 250ms drop per pageview.” –Weather.com
The Add to Homescreen Experience
PWAs are designed to be installable. To match native apps on the mobile platform, browsers are implementing an Add to Homescreen experience.
Chrome, FireFox, Opera, and Samsung Internet all offer similar Add to Homescreen prompts. When a user opens your PWA, after a period of time or possibly a repeat visit, the browser may automatically prompt them to add the site to the home screen. They can also manually install a PWA that creates the home screen icon and launch experience.
When a PWA is installed using Chrome on Android, it's automatically converted to a WebAPK. This process turns the PWA into a native application on Android. Under the hood, what Chrome does is something similar to a hybrid or Cordova application. At this time, you don't get access to Android-specific platform APIs, but the user can manage the PWA just like any other app.
This is done because Google wanted to avoid writing parallel plumbing to enable PWAs to have a similar experience to native apps. Now your PWA is displayed in the app drawer on Android and shows up like a regular application in the application list. Installed PWAs are included in the Android application settings. This makes it manageable, just like any other native application.
You can perform the same process on the desktop. Of course, it's more of an Add to Desktop experience. At this time, no one is offering an Add to Start Menu option. But that should change in the future. (See Figure 6.)
The Add to Homescreen functionality is driven by the Web manifest file. This is a JSON document that you host on your Web server and reference from your Webpages. It provides metadata to the mobile device and browser that describes your brand's experience.
The primary fields are name, a short description, and an array of icons. But you can also control how your application will launch once it's been added to the home screen. For example, if you wanted to launch without any of the browser accoutrements like an address bar, you can tell it to do so.
Another important property you define is the Start URL. This is a key feature that differentiates a PWA from merely bookmarking a site to the desktop. When you bookmark a site, it creates a shortcut to the current URL. When a PWA is installed, it opens to the designated start or home page defined in the manifest file.
The following code snippet is an example Web manifest file. It's a simple JSON object with a set of properties that describe the PWA to the platform. This example is a minimal manifest file; there are additional optional properties that you may also want to include.
{
"name": "Fast Furniture",
"short_name": "Furniture",
"icons": [{
"src": "/meta/24d5b086-693c-a559-1926-8fa98f0b5684.webPlatform.png",
"sizes": "24x24",
"type": "image/png"
},... ],
"description": "Fast Furniture is a Progressive Web Application Demonstration Site",
"orientation": "portrait-primary",
"start_url": "/?utm_source=homescreen",
"display": "fullscreen",
"background_color": "#F99D36",
"theme_color": "#55B4F6",
"scope": "/"
}
You can also limit the display mode to landscape or portrait, color scheme, language, and serval other properties (see Figure 7). There's an official manifest standard, but browsers may also support custom fields. For example, Windows uses extra properties to describe your application to the store.
If you want to launch with all the browser features, you can do that as well. This means that your progressive Web app can launch from the home screen and take up the entire viewport of the consumer's device, just like a native application.
Service Workers
Service workers are a brand-new addition to the Web platform. They were first introduced about three years ago by the Google Chrome team. Since then, all major browsers have shipped support for service workers at least in early preview additions.
Service workers are a JavaScript, registered by a Web page, that runs in the background. They can intercept requests to the network, interact with the Cache API, and manage native push notifications and background synchronization. They execute in a separate thread from a browser tab or client.
Service workers are asynchronous, which means they require the use of promises, and activate in response to events. An event can be a tab loading a page within the service worker's scope, a push notification or background sync event. They are designed to be an extensible platform, which means that more functionality can be added later.
Recently, Microsoft and Apple both shipped preview editions of Edge and Safari with service worker support turned on by default (https://love2dev.com/blog/apple-and-microsoft-ship-service-worker-support-in-preview-builds/). Once these two browsers ship support, all major browsers will have service worker support. I don't have exact dates at this time, but I suspect that by autumn, we'll have ubiquitous platform support.
Service workers must be registered from a Web page. You should place the registration within a feature-detection block. This prohibits the service worker registration code from executing in older browsers, avoiding some unwanted exceptions.
if ('serviceWorker' in navigator) {
navigator.serviceWorker
.register('/sw.js')
.then(function(registration) {
// Registration was successful
})
.catch(function(err) {
// registration failed :(
});
}
You can only register a single service worker per origin (domain). The origin is also known as the scope. A service worker can only control pages and “clients” within the origin that registered the worker. The service worker file must also reside within the scope.
A site could have multiple service workers where they each manage a scope, partitioned by folder structure. For example, the top-level site (example.com) could have a service worker registered. The finance department (example.com/finance) could have a separate service worker. The finance service worker can't access the site's top level and the top-level worker can't access the finance department scope.
The top-level service worker could control the finance department content if it didn't have its own service worker. A service worker's scope can't creep outside of its origin. This makes them secure and prevents bad third-party script providers from injecting themselves into your application.
In addition to Web pages, a service worker client could be any process that initiates a service worker. This includes push notifications, background sync, and other features currently going through the standardization process.
A service worker can control multiple clients, like browser tabs, at the same time. Each client can only have a single service worker.
Each client can only have a single service worker.
Confused?
I hope not. These concepts belong to service worker life cycle management. This can be a tricky and often overlooked aspect of service workers.
Besides scope, life cycle refers to how service workers are registered, updated and removed. By default, a new service worker can't become active until all existing clients are closed. You can also programmatically force a new service worker to take control by calling the skipWaiting method in the install event. (See Figure 8.)
When a service worker is registered, there are a couple of events to which you can bind handlers: install and activate. When the service worker is registered, the install event triggers. When it becomes active, the activate event triggers.
self.addEventListener('install', function (e) {
//do something
});
self.addEventListener('activate', function (event) {
//do something
});
Don't discount the importance of managing your service worker's life cycle. These events give you the ability to perform tasks to set up your service worker. The most common tasks are pre-caching network resources and cleaning named caches when the schema changes.
Service workers run in an external process or thread from the browser. They can intercept every request to the network, allowing you to control the response. They also enable other features, like push notifications, even when the browser isn't running.
Part of the service worker specification is a robust cache infrastructure. The service worker cache gives you the ability to persist network responses in the browser and return responses from cache instead of hitting the network. By caching resources locally, you make the network an optional requirement. Now, there's no network latency when you request a resource.
This feature alone enables rich off-line and instant loading experiences. The cache API makes the network an enhancement and not a requirement.
How Service Worker Caching Works
Service workers can intercept every request to the network and determine how that request will be handled. It does this through the fetch event handler.
The reason why it happens in the fetch event is that the service worker API depends on the Fetch API being implemented in the browser (https://developer.mozilla.org/en-US/docs/Web/API/Fetch_API/Using_Fetch). If you haven't heard of the Fetch API, you should take some time to learn how it works. It's a modern replacement for XMLHttpRequest, or, as we commonly know it, Ajax.
Ajax is a completely rewritten implementation that leans on promises and a much simpler API. It also includes custom Response, Request, Header, and Body objects that you can manipulate to craft custom versions as needed. The Cache API depends on these fetch objects to persist network responses across sessions.
The fetch event handler allows you to create your logic workflow to determine how requests will be handled. The fetch event has an Event object that contains a reference to the Request. By passing the Request object to the Cache API, you can determine if a response is already cached for that particular request. If so, you have the option of returning the cached response or hitting the network, or, if you are so inclined, doing both. This means that you have complete control over how your application is persisted in the browser.
Asset Pre-Caching
When you first install a service worker, there are several events that you can use, one of which is the install event. Here, you can perform many different tasks. A very common one is to precache common assets.
The following code snippet fetches a list of important URLs from the network and places them in a named cache. These assets typically belong to the app shell (the common site layout) or are identified as frequently needed or above the fold resources. This list contains common HTML, JavaScript, CSS, fonts, and images.
By caching them when the service worker is installed, you're ensuring these resources are immediately available when the app is loaded.
self.addEventListener('install', function (e) {
e.waitUntil(
caches.open(cacheName).then(function (cache) {
return cache.addAll(filesToCache)
.catch(function (error) {
console.log(error);
});
})
);
});
This is what is called pre-caching and you need to select files that are used frequently, like your main site's CSS and JavaScript files, common images, and popular pages. Pre-caching these files insures that they're available whenever someone requests them. This means that those pages and those assets can load almost instantly, without having to download from the network (see Figure 9).
This is a huge advantage because many of the files I mentioned are commonly accessed across your website. Because they're stored locally, you no longer need to hit the network. This is a valuable feature for traditional desktops on broadband as well as mobile devices on cellular networks. The fastest request you can make is the one that never happens, or at least that's the way the saying goes. Accessing responses from cache has negligible latency, measured in a few milliseconds. Often, my experience is that the latency is much less than a single frame-refresh on the display.
Cache First Falling Back to the Network
Let's look at a common caching pattern, the cache first and network fallback strategy. This is one of my favorite caching patterns and allows me to return network assets instantly to the client (see Figure 10).
This pattern requests a resource and the service worker intercepts the request. It then checks to see if there's a cached response. If there is a cached response, it's returned and the network access is never accessed.
If a response isn't currently cached, the request is passed along to the network just as if the service worker didn't exist. When the response is received, the service worker can then clone the response and store it in cache for later access (see Listing 1).
Listing 1: Clone the response and store it in cache
self.addEventListener("fetch", event => {
event.respondWith(caches.match(event.request).then(function (response) {
if (response) {
return response;
}
return fetch(event.request).then(response => {
if (response.ok || response.status === 0) {
let copy = response.clone();
//if it was not in the cache it must be added to the dynamic cache
caches.open(dynamicCache).then(cache => {
cache.put(event.request, copy);
});
}
return response;
});
}));
});
There are many caching patterns that you can employ in your application. I like to say that it depends on your application and data's personality. Personally, I pull from over 20 different patterns, but they generally use the cache first, falling back to the network strategy as their base.
You can also cache when the device is offline and provide a fallback. For example, you can add service worker templating to your caching strategy and create a custom response on the fly. You could also provide generic fallbacks for different content types.
You May Be Wondering About Traditional Browser Cache?
Traditional browser cache doesn't allow you to have any control over how and if network assets are persisted. Cache control headers give you some knowledge over how the assets may be stored, but browsers can purge as they feel necessary.
This is a been extremely problematic on mobile devices as browser cache is purged very aggressively, meaning your website or application can have much more network chatter than you ever anticipated. This is one of the many reasons why the average webpage today has a more then 19-second load cycle.
More Than Just A Cache Machine
Caching isn't the only thing that service workers provide. Because service workers function in an external process and are event driven, they can respond to external stimulus.
The first exciting feature to take advantage of this is native push notifications. This has been the last big native application feature that the Web has been lacking. It's one of the few native features potential customers have asked me to implement on the Web and I simply couldn't offer.
Service Workers Provide a Native Push Notification Experience
We've had the ability to do notifications through Web workers for a few years. The drawback is that those notifications require the browser to be open. Now, push notifications can work even if the user isn't actively using their device. This gives businesses the ability to engage with and interact with customers anytime. For example, if I have a new blog post, I could send a push notification to all my push notification subscribers that a new post has been made. Statistically, this means that my readership will go up.
If I'm a retailer, I can promote specials and new products that the customer may find interesting, as shown in Figure 11.
There are additional service worker features in the pipeline. Browsers are already shipping background sync, for instance. Other capabilities are being debated and we should start seeing implementations soon. Service workers are designed to be extensible and serve as a new platform for the Web.
Service Workers: The Serverless Browser Platform
I think the best description I've heard of service workers is that it's a proxy server in the browser. Although it's not technically in the browser, it is commonly associated with the browser.
I think a good analogy for service workers is that of a cloud-based function like AWS Lambda or Azure functions. If you're not familiar with them, they're tiny task-specific features that you can implement in any cloud platform that spin up for a very short period of time do the one task they need to do and then quietly go away. They don't consume expensive computer resources and in the case of our mobile phones, service workers won't drain the battery.
Service workers briefly perform a small task and then go quietly away.
Performance as a Requirement
So far, I've discussed technical changes to the Web to enable PWAs. But there's more to it than just some of these new platform features. PWAs are about delivering the best user experience possible through the Web. And that means that you have to implement Web development and architectural best practices.
Unfortunately, this doesn't always happen. The 19-second average page load time that the Web currently suffers from is largely due to poor development practices. The Chrome team has been promoting best practices for the last four years, known as RAIL and PRPL. (See the sidebar for a brief definition of RAIL and PRPL.)
These two patterns are all about designing your code to work as efficiently as possible in the browser. When doing so, you eliminate almost all the scenarios they create that are what we might call “janky.” This is where the page re-renders itself or jumps around as the user's trying to read it. And that's because the JavaScript is typically out of control and manipulating the DOM and in an uncontrollable manner.
Being Progressive
PWAs are simply existing websites that implement these new features as they become available. This is where the progressive part comes into play. When you have a progressively enhanced website, modern features and functionalities only kick in when the platform (browser) supports them.
For example, you don't want to try to register a service worker in a browser that doesn't support service workers because it throws an exception. But if you do feature detection, you can determine if service workers are supported. This means that you can dynamically load the script to execute those new features based on platform support.
If a feature has a polyfill, a JavaScript fallback, you can also feature-detect to determine if the polyfill should be loaded. I always check to see if Promises and the Fetch API are supported. If one of them isn't, I load polyfills to add the missing feature.
You can do this for many modern features, and dynamically load polyfills as needed. This means that your page and site's loading profile will be as lean as possible based on the user's browser.
Which Browsers Support PWAs?
As of the end of 2017, we have a very good browser support story, as far as PWAs are concerned. Microsoft Edge and Apple Safari are the last two browsers to implement full PWA functionality. All browsers now support HTTPS and HTTP/2.
Apple Safari is the only one that hasn't shipped a Web manifest implementation. Apple began work on it this past autumn and recently shipped support in a technology preview.
With Microsoft and Apple on the PWA bandwagon, we have a full set of platforms engaged. This means that you can expect anyone anywhere to experience your PWA.
The Future of Frameworks and Single Page Apps
A common question many have is how PWAs relate to single page apps (SPAs). A SPA can be a PWA, but a PWA doesn't need to be a SPA. In fact, I'd argue that PWAs will bring SPAs as we know them to an end.
By removing that overhead from the UI thread and moving it to the server to pre-render you can sit back and watch load times improve. This also means that you don't need to deliver as much code to the client.
This has a big impact because your site's payload is much smaller than without PWAs and that you're moving work previously done on the UI thread to the background or the server. Now you can leave the UI thread to do what it's really good at doing: rendering the DOM.
I think the days of heavy SPAs are numbered because we no longer need to worry about page transition delays. In the past, page load latencies created an unwanted delay as a user navigated from page to page. Because your site assets should be cached using the service worker, pages can load instantly, with no network latency.
This doesn't mean that your entire site will be pre-cached, but you should use caching strategies that provide a rich experience. You may still employ some SPA techniques, like an app shell you “fill in” once markup is received from the server. This is where advanced caching and rendering strategies can be employed.
Service workers can be very simple or very complex. It's up to you to determine how much you're going to learn in order to implement service workers. If you want to be a programmer on the Web, it's my advice to learn how to use service workers properly and understand the power and functionality that they offer. I'd also advise you to look at how the server is used in your PWA. Just like modern JavaScript dependencies are changing, so are Web server requirements and functionality.
The good news is that you can upgrade existing websites to a PWA without changing existing code. If your application is a SPA, it'll still work just fine. You can also make your site work off-line and load instantly.
You don't have to change the way you're laying out your application because a mobile-first responsive website is exactly what a PWA should be. If you're rendering everything on-demand on the server, you can continue to do so and your PWA will work just fine.
As you add PWA functionality, I guarantee that you'll start seeing some of the benefits I've discussed. You can start doing much of the work that you been putting onto the client-side and the server into the service worker. You'll see that it can be done much more efficiently there.
PWAs offer a brand-new experience for the Web that should engage customers and end-users more than ever before. They are a force that's going to make the Web equal to or better than native applications. With the decline of native applications, it's time for the Web to step up and take its place as the preferred client platform again. Just like we did on the desktop some 10 years ago.
