Browser Storage for HTML5 Apps

For years one of the main advantages of writing desktop applications has been the easy access to local storage on the client machine. Web developers have put up with the inability to store and retrieve data from the client’s machine for a long time, but it looks like that may change soon. You might even think it already has after reading this article. Yes, I’m going to discuss the origins of persisting data on clients machine and then introduce you to the Web Storage standard.

Most web developers know that the only kind of local storage we could expect from a web browser comes in the form of cookies. Well, not entirely. The desire to store data on the client machine is not a new concept and it wasn’t conceived while creating HTML5 specifications. What is even more surprising is that a working implementation was developed by Microsoft as part of the IE6 feature set. They called it userData and it essentially guaranteed at least 640KB of local space per domain depending on the IE security policies set by the user. That may seem like very little space by today’s standard, but when we compare it to the maximum 4KB space available to us by Cookies, the improvement is appreciable.

What’s Wrong With Cookies?

A number of things. The foremost problem with Cookies is that they are sent back and forth between browser and server with every HTTP request. This isn’t a desirable behaviour because more often than not developers do not wish to transmit local data to the server more than once, if once at all. Cookies give developer no choice.

As we said previously, Cookies can only store up to 4KB of data. It’s not a lot of data, nevertheless 4KB is enough data to noticeably slow down page requests.

Also, Cookies are passed back and forth between the client and server in clear text. Therefore, the only way to protect them is by encrypting the data while communicating with the backend server using SSL (Secure Socket Layer). However, most websites on the internet don’t use SSL, which leaves storage open to eavesdropping.

There are other issues that make Cookies less useful. Ideally, developers wish to have the ability to persist large amounts of data on the client machine and not have to transmit that data to the server over and over again.

What Are The Alternative Solutions?

So far we have not discussed non-standard workarounds for persisting data on the client machine. When Adobe (then known as Macromedia) developers were releasing Flash Player 6, they too had to address the same problem. In 2002, Flash Player 6 introduced a new feature called Local Shared Object or more often known as Flash Cookies to effectively introduce the same capabilities as standard HTTP Cookies to Flash movies and sites. Local Shared Object allowed developers to persist up 100KB of data onto the client machine by default.

The second solution is Googleís implementation of Local Storage as part of the Gears plugin for Web Browsers. Gears was (and I tell you why I use was in a moment) the combination of multiple missing and useful features needed for developing Rich Internet Applications (RIA). Gears’ Local Storage was based on the less popular Web SQL specification that took advantage of SQLite. You guessed it right, Gears gave developers a full blown SQL database for persisting an unlimited amount of data on the client machine.

The developers of Ajax Massive Storage System (AMASS) took this opportunity and developed a third party JavaScript library that made it possible for standard HTML sites to take advantage of the Local Shared Object feature in Flash or the Gears plugin in order to persist data on the client machine. Also, Dojo JavaScript Library is capable of detecting the availability of Local Storage mechanics (e.g. Google Gears, Local Shared Object. etc.) and provides a unified interface for persisting data across different Web Browsers.

Back to why I said Gears ‘was’ instead of ‘still is’: that’s because Google recently announced that they will be dropping further development of the Gears plugin in favor of HTML5 and the Web Storage specification presented in this tutorial.

HTML5 and Web Storage Specifications

Now that the history lesson is over, we are going to learn about Web Storage and dive into some code to better understand it. The easiest way to describe Web Storage is the ability to persist data on the client machine in the form of one key for one value. This is very similar to how associative arrays are used:

    { "The Key" : "The Value" }

Local Storage is designed to be natively supported by Web Browsers. This means no more third party libraries and messing with Flash. Surprisingly, Web Storage has been one of the more successful specifications in terms of adoption by modern browsers. In fact, almost all modern browsers support Web Storage, including:

• Internet Explorer 8+
• Firefox 3.5+
• Safari 4+
• Opera 10.5+
• iPhone Safari
• Android Web Browser

While Web Storage aims to provide functionality similar to Cookies, it has been further refined to not carry any of their negative attributes. For instance, Web Storage allows for persisting up to 5MB of data, a significant increase in space compared to how much data can be stored in a Cookie. Also, persisting data using Web Storage will not result in sending that data to the server backend with every page request. This significantly increases performance. According to the Web Storage specification, Web Browsers only expire the persisted data from the local machine when requested to do so by the user and will always avoid deleting data while a script that could access that data is running.

Web Browsers expose Web Storage through the localStorage object in JavaScript. One easy way to determine whether a Web Browser can support Web Storage is to execute this JavaScript code:

    var webStorageSupported = ('localStorage' in window) && window['localStorage'] !== null;

According to the W3C’s specification for Web Storage, the localStorage object implements the following set of methods and properties from the Storage interface. Let’s examine each of these methods and properties to find out what they do and how they can be used:

interface Storage {
    readonly long length;
    void setItem(String key, Object data);
    Object getItem(String key);
    void removeItem(String key);
    void clear();
    String key(long index);
};

The length property is very useful. It will return the number of key/value pairs currently saved to Local Storage under the currently accessed domain:

    alert(localStorage.length);

If no key/value pairs have previously been saved in the local storage, then the above script will display an alert window with “0″ as the message, otherwise the message will be the number of persisted key/value pairs.

The setItem(key, value) method simply saves a new entry on the local machine. To save the key name with the value arman we could execute this script:

    localStorage.setItem('name', 'arman');

To ensure that key name was truly saved to the local storage with the value arman we need to use the getItem(key) method. The getItem method simply accepts a key and searches the local storage to find a matching key and then returns its value.

    localStorage.setItem('name', 'arman');
    var value = localStorage.getItem('name');
    alert(value);

If you run the above script you should see an alert window containing the word arman appear on the screen, confirming that we’ve successfully saved a new key/value pair to the local storage. Since the localStorage object behaves similar to associative arrays, we could simplify the above script to look like this and it will still function just the same:

    localStorage['name'] = 'arman';
    var value = localStorage['name'];
    alert(value);

Let’s look at the removeItem(key) method. This method is designed to remove a previously saved key/value pair from the local storage. If the key does not exist, this method simply does nothing. The following code sample demonstrates the use of the removeItem method:

    localStorage.setItem('name', 'arman');
    localStorage.removeItem('name');
    var value = localStorage.getItem('name');
    alert(value);

When the above script is executed, you should see an alert window with the value null in the alert box. Using the name key, the above script simply creates a new key/value pair and immediately removes it from the local storage. Subsequently, a null value is returned when accessing the local storage with the same name key.

There will come occasions where there will be a need to completely clear the local storage and start with a clean slate. The clear() method is designed exactly for that purpose. This method automatically empties all the previously saved key/value pairs from the local storage. If there are no entries then nothing will change.

    localStorage.setItem('name', 'arman');
    localStorage.setItem('name', 'smith');
    localStorage.setItem('name', 'frank');
    alert(localStorage.length);
    localStorage.clear();
    alert(localStorage.length);

Although the above script creates three new key/value pairs (as evidenced by the first alert), the call to the clear() method removes all the entries. Subsequently, the second alert window will display a “0″ message.

The final method we need to look at is the key(index) method. This method will retrieve the name of a key based on the index parameter. localStorage maintains a 0 based list of all entries within itself. Therefore to access the first key from the local storage, we need to use 0 as the index as illustrated in this script:

    localStorage.clear();
    localStorage.setItem('age', 5);
    alert(localStorage.key(0));

When the above script is executed it should display an alert window with the message “age”. Note how in the above example the first line of code clears the local storage. This is to ensure we begin with a clean slate. Another useful application of the key() method is in conjunction with the length property. For instance to get all the key/value pairs from the local storage without knowing the keys in advance, we could write a script like the following:

    localStorage.clear();
    localStorage.setItem("title", "Mr.");
    localStorage.setItem("fullname", "Aaron Darwin");
    localStorage.setItem("age", 17);
    localStorage.setItem("height", 182.5);
    for(var i = 0; i < localStorage.length; i++)
    {
        var keyName = localStorage.key(i);
        var value = localStorage.getItem(keyName);
        alert(keyName + " = " + value);
    }

In the above script, our code first clears and then adds four new key/value pairs to the local storage. Then it uses the length property in a For loop to work out the key for each key/value pair. On every iteration of the loop the key is assigned to the keyName variable which is then passed to the getItem() method to retrieve its value.

The Subtleties

When accessing data using a key that doesn't exist in the local storage, instead of an exception, a null value is always returned. This makes it difficult to know if the value of the key is null or the key simply doesn't exist in the local storage.

The second one to talk about is the setItem(key, value) method. We know that we can pass any type of value to the setItem() method for the value parameter, but that isn't entirely true. In JavaScript, we are able to create Image objects. However, the current implementation of Web Storage only allows persisting primitive types such as String, Boolean, Integer and Float types. Therefore, we can expect the following script to throw an exception and crash:

    var imageObject = new Image("http://www.google.com/logo.gif");
    localStorage.setItem("image", imageObject);

Additionally, even though we can save Boolean, Integer and Float types, the underlying representation of these types falls back onto the String type. This means regardless of the type of value passed to the setItem() method, the getItem() method will always return a value that is of type String. Let's look at an example script:

    var integerVariable = 34;
    localStorage.setItem("age", integerVariable);
    var newVariable = localStorage.getItem("age");
    alert(typeof(newVariable) == "number");

In the above script, ultimately the value of the integerVariable is saved as String. Therefore when we inspect the type of newVariable after getting its value from the local storage, it's no longer an Integer type and therefore the comparison statement will evaluate to false. In such situations, we must use the handy parseInt(String) and parseFloat(String) functions for conversion. There is NO function to parse Boolean data type so we must simply compare the retrieved value with "true" or "false" strings.

Finally, to get the comparison statement in the above script to evaluate to true, we must modify our code to use the parseInt() function as shown in the following script:

    var integerVariable = 34;
    localStorage.setItem("age", integerVariable);
    var newVariable = parseInt(localStorage.getItem("age"));
    alert(typeof(newVariable) == "number");

Now, when the above script is executed, the type of the value stored in the newVariable will be Integer. Subsequently, the comparison statement will evaluate to true.

When the persisting data reaches the 5MB quota, Web Browsers do not provide the ability to ask for more space in the local storage. They instead throw the QUOTA_EXCEEDED_ERR exception to notify the script that there is no more space. The simplest way to handle this exception is to catch the exception and notify the user of the problem.

Alternatively when catching the exception, the script can delete some key/value entries from the local storage to clear up space for new pairs.

Lastly, when the document is requested directly from the disk and not from a web server, the local storage will be cleared every time you navigate away from the page.

Web Storage Events

Web Storage Standard allows scripts to be notified when other parts of the code add, update or delete entries from the local storage. Thus, whenever something changes in the local storage, a Storage Event is going to fire. However, in the case of deleting an entry using a non existing key, no event will be fired. This is because nothing will change in the local storage. Unfortunately, my tests proved that currently Webkit browsers (such as Safari and Chrome) do not fire the Storage Events as described by the Web Storage specification. Internet Explorer, Firefox and Opera browsers do behave as expected.

A local storage event cannot be cancelled. Its sole purpose is to notify the user code of a change that has happened inside the local storage.

The following script shows you how to listen for changes in the local storage, by registering an event handler which will be called every time a Storage Event is fired:

    if (window.addEventListener) {
        window.addEventListener("storage", handleStorageChange, false);
    }
    else
    {
        window.attachEvent("onstorage", handleStorageChange);
    }
    function handleStorageChange(event)
    {
        alert("Something was changed in the local storage");
    }

Since no version of Internet Explorer (except for version 9 public preview) supports the DOM Level 2 event handling system, the attacheEvent() method has to be used to register event handlers.

The event argument is useful because it carries information about changes in the local storage. It adheres to the StorageEvent interface:

interface StorageEvent : Event {
    readonly String key;
    readonly Object oldValue;
    readonly object newValue;
    readonly String url;
};

The key property identifies which key/value pair was changed. The oldValue and the newValue properties behave as their names suggest. That is, they respectively contain the previous and the new value for the key.

Finally, the url property contains the address of the document for which the local storage was just modified. However, due to multiple implementation iterations of the Web Storage specification, in some Web Browsers the url property may be implemented as uri. Therefore it's better to first check the url property and then uri in case the url property is undefined.

Demo

After reading this much, I usually like to see an example before I believe it all. Just like myself, I'm sure many of you would also like to see a working example. That's no problem, because I've prepared a small script that simply demonstrates the use of the web storage standard in a working application.

It's made such that it can be also be tested using iPhone Safari and the Android Web Browser. The idea is to have the browser remember which boxes have been opened before using the local storage. I'll leave you with this code to play with:

<!DOCTYPE html>
<html>
    <head>
        <meta name="viewport" content="width=240,user-scalable=no" />
        <style>
            * { padding: 0px; margin: 0px; font-family: "Lucida Sans", "Tahoma", san-serif, arial; }
            body { background: #333; }
            h1 {  font-size: 20px;  }
            p { padding: 5px; margin-bottom: 10px; font-size: 12px; }
            button { padding: 4px; margin-top: 10px; }
            #wrap { width: 240px; background: #FFF; margin: 0px auto; padding: 10px; }
            #blocks .unchecked, a { height: 40px; width: 40px; margin: 4px; display: inline-block; cursor: pointer; text-align: center; line-height: 40px; }
            #blocks .unchecked { background: #000; }
            #blocks .unchecked:hover { background: #333; }
        </style>
        <script>
            var colorTable = {
                "0" : "blue",
                "1" : "red",
                "2" : "green",
                "3" : "orange",
                "4" : "purple",
                "5" : "brown",
                "6" : "gold",
                "7" : "lime",
                "8" : "lightblue",
                "9" : "yellow"
            };
            function initializeGame()
            {
                if (browserSupportsLocalStorage() == false)
                {
                    alert("This browser doesn't support Web Storage standard");
                    return;
                }
                var containerDiv = document.getElementById("blocks");
                for (var i = 0; i < 10; i++)
                {
                    var id           = i.toString();
                    var anchor       = document.createElement("a");
                    anchor.id        = id;
                    anchor.innerHTML = i + 1;
                    if (localStorage.getItem(id) != null)
                    {
                        anchor.style.backgroundColor = colorTable[id];
                    }
                    else
                    {
                        anchor.className = "unchecked";
                        if (anchor.addEventListener)
                        {
                           anchor.addEventListener("click", handleBoxClick, false);
                        }
                        else
                        {
                           anchor.attachEvent("onclick", handleBoxClick);
                        }
                    }
                    containerDiv.appendChild(anchor);
                }
            }
            function handleBoxClick(e)
            {
                var target = (e.target) ? e.target : e.srcElement;
                if (target.className == "")
                    return;
                var id = target.id;
                var color = colorTable[id]
                target.className = "";
                target.style.backgroundColor = colorTable[id];
                localStorage.setItem(id, color);
            }
            function resetGame()
            {
                var containerDiv = document.getElementById("blocks");
                containerDiv.innerHTML = "";
                localStorage.clear();
                initializeGame();
            }
            function browserSupportsLocalStorage()
            {
                return ('localStorage' in window) && (window['localStorage'] != null);
            }
        </script>
    </head>
    <body onload="initializeGame();">
        <div id="wrap">
            <h1>Web Storage Demo</h1>
            <p>
                This page was design to simply demonstrate the use of Web Storage in modern browsers. In this example boxes
                that haven't yet been clicked remain black. Once you click a black box, it's real color will be revealed.
                The browser however will remember which boxes are clicked, even when you navigate away from this page.
                Go on, give it a try.
            </p>
            <div id="blocks">
            </div>
            <button onclick="resetGame()">Reset Game</button>
            <button onclick="document.location.reload(true);">Refresh Page</button>
        </div>
    </body>
</html>