Chapter 15. User Authentication, Integrating Third-Party Plugins, and Mocking with JavaScript

Our beautiful lists site has been live for a few days, and our users are starting to come back to us with feedback. “We love the site”, they say, “but we keep losing our lists. Manually remembering URLs is hard. It’d be great if it could remember what lists we’d started”.

Remember Henry Ford and faster horses. Whenever you hear a user requirement, it’s important to dig a little deeper and think—what is the real requirement here? And how can I make it involve a cool new technology I’ve been wanting to try out?

Clearly the requirement here is that people want to have some kind of user account on the site. So, without further ado, let’s dive into authentication.

Naturally we’re not going to mess about with remembering passwords ourselves—besides being so ’90s, secure storage of user passwords is a security nightmare we’d rather leave to someone else. We’ll use a federated authentication system instead.

(If you insist on storing your own passwords, Django’s default auth module is ready and waiting for you. It’s nice and straightforward, and I’ll leave it to you to discover on your own.)

In this chapter, we’re going to get pretty deep into a testing technique called “mocking”. Personally, I know it took me a few weeks to really get my head around mocking, so don’t worry if it’s confusing at first. In this chapter we do a lot of mocking in JavaScript. In the next chapter we’ll do some mocking with Python, which you might find a little easier to grasp. I would recommend reading both of them through together, and just letting the whole concept wash over you; then come back and do them again, and see if you understand all of the steps a little better on the second round.

Note

Do let me know via if you feel there’s any particular sections where I don’t explain things well, or where I’m going too fast.

Mozilla Persona (BrowserID)

But which federated authentication system to use? Oauth? Openid? “Login with Facebook”? Ugh. In my book those all have unacceptable creepy overtones; why should Google or Facebook know what sites you’re logging into and when? Thankfully there are still some techno-hippy-idealists out there, and the lovely people at Mozilla have cooked up a privacy-friendly auth mechanism they call “Persona”, or sometimes “BrowserID”.

The theory goes that your web browser acts as a third party between the website that wants to check your ID, and the website that you will use as a guarantor of your ID. The latter may be Google or Facebook or whomever, but a clever protocol means that they never need know which website you were logging into or when.

Ultimately, Persona may never take off as an authentication platform, but the main lessons from the next couple of chapters should be relevant no matter what third-party auth system you want to integrate:

  • Don’t test other people’s code or APIs.
  • But, test that you’ve integrated them correctly into your own code.
  • Check that everything works from the point of view of the user.
  • Test that your system degrades gracefully if the third party is down.

Exploratory Coding, aka “Spiking”

Before I wrote this chapter all I’d seen of Persona was a talk at PyCon by Dan Callahan, in which he promised it could be implemented in 30 lines of code, and magic’d his way through a demo—in other words, I knew it not at all.

In Chapter 10 and Chapter 11 we saw that you can use a unit test as a way of exploring a new API, but sometimes you just want to hack something together without any tests at all, just to see if it works, to learn it or get a feel for it. That’s absolutely fine. When learning a new tool or exploring a new possible solution, it’s often appropriate to leave the rigorous TDD process to one side, and build a little prototype without tests, or perhaps with very few tests. The goat doesn’t mind looking the other way for a bit.

This kind of prototyping activity is often called a “spike”, for reasons best known.

The first thing I did was take a look at an existing Django-Persona integration called Django-BrowserID, but unfortunately it didn’t really support Python 3. I’m sure it will by the time you read this, but I was quietly relieved since I was rather looking forward to writing my own code for this!

It took me about three hours of hacking about, using a combination of code stolen from Dan’s talk and the example code on the Persona site, but by the end I had something which just about works. I’ll take you on a tour, and then we’ll go through and “de-spike” the implementation.

You should go ahead and add this code to your own site too, and then you can have a play with it, try logging in with your own email address, and convince yourself that it really does work.

Starting a Branch for the Spike

Before embarking on a spike, it’s a good idea to start a new branch, so you can still use your VCS without worrying about your spike commits getting mixed up with your production code:

$ git checkout -b persona-spike

Frontend and JavaScript Code

Let’s start with the frontend. I was able to cut and paste code from the Persona site and Dan’s slides with minimal modification:

lists/templates/base.html (ch15l001)

<script src="http://code.jquery.com/jquery.min.js"></script>
<script src="/static/list.js"></script>
<script src="https://login.persona.org/include.js"></script>
<script>
$(document).ready(function() {

var loginLink = document.getElementById('login');
if (loginLink) {
  loginLink.onclick = function() { navigator.id.request(); };
}

var logoutLink = document.getElementById('logout');
if (logoutLink) {
  logoutLink.onclick = function() { navigator.id.logout(); };
}

var currentUser = '{{ user.email }}' || null;
var csrf_token = '{{ csrf_token }}';
console.log(currentUser);

navigator.id.watch({
  loggedInUser: currentUser,
  onlogin: function(assertion) {
    $.post('/accounts/login', {assertion: assertion, csrfmiddlewaretoken: csrf_token})
    .done(function() { window.location.reload(); })
    .fail(function() { navigator.id.logout();});
  },
  onlogout: function() {
    $.post('/accounts/logout')
    .always(function() { window.location.reload(); });
  }
});

});
</script>

The Persona JavaScript library gives us a special navigator.id object. We bind its request method to our link called “login” (which I’ve put in any old where at the top of the page), and similarly a “logout” link gets bound to a logout function:

lists/templates/base.html (ch15l002)

<body>
<div class="container">

    <div class="navbar">
        {% if user.email %}
            <p>Logged in as {{ user.email}}</p>
            <p><a id="logout" href="{% url 'logout' %}">Sign out</a></p>
        {% else %}
            <a href="#" id="login">Sign in</a>
        {% endif %}
        <p>User: {{user}}</p>
    </div>

    <div class="row">
    [...]

The Browser-ID Protocol

Persona will now pop up its authentication dialog box if users click the log in link. What happens next is the clever part of the Persona protocol: the user enters an email address, and the browser takes care of validating that email address, by taking the user to the email provider (Google, Yahoo, or whoever), and validating it with them.

Let’s say it’s Google: Google asks the user to confirm their username and password, and maybe even does some two-factor auth wizardry, and is then prepared to confirm to your browser that you are who you say you are. Google then passes a certificate back to the browser, which is cryptographically signed to prove it’s from Google, and which contains the user’s email address.

At this point the browser can trust that you do own that email address, and it can incidentally reuse that certificate for any other websites that use Persona.

Now it combines the certificate with the domain name of the website you want to log into in to a blob called an “assertion”, and sends them on to our site for validation.

This is the point between the navigator.id.request and the navigator.id.watch callback for onlogin—we send the assertion via POST to the login URL on our site, which I’ve put at accounts/login.

On the server, we now have the job of verifying the assertion: is it really proof that the user owns that email address? Our server can check, because Google has signed part of the assertion with its public key. We can either write code to do the crypto for this step ourselves, or we can use a public service from Mozilla to do it for us.

Note

Yes, letting Mozilla do it for us totally defeats the whole privacy point, but it’s the principle. We could do it ourselves if we wanted to. It’s left as an exercise for the reader! There are more details on the Mozilla site, including all the clever public key crypto that keeps Google from knowing what site you want to log in to, but also stops replay attacks and so on. Smart.

The Server Side: Custom Authentication

Next we prep an app for our accounts stuff:

$ python3 manage.py startapp accounts

Here’s the view that handles the POST to accounts/login:

accounts/views.py

import sys
from django.contrib.auth import authenticate
from django.contrib.auth import login as auth_login
from django.shortcuts import redirect

def login(request):
    print('login view', file=sys.stderr)
    # user = PersonaAuthenticationBackend().authenticate(request.POST['assertion'])
    user = authenticate(assertion=request.POST['assertion'])
    if user is not None:
        auth_login(request, user)
    return redirect('/')

You can see that it’s clearly “spike” code from things like the commented-out line, evidence of an early experiment that failed.

Here’s the authenticate function, which is implemented as a custom Django “authentication backend”. (We could have done it inline in the view, but using a backend is the Django recommended way. It would let us reuse the authentication system in the admin site, for example.)

accounts/authentication.py

import requests
import sys
from accounts.models import ListUser

class PersonaAuthenticationBackend(object):

    def authenticate(self, assertion):
        # Send the assertion to Mozilla's verifier service.
        data = {'assertion': assertion, 'audience': 'localhost'}
        print('sending to mozilla', data, file=sys.stderr)
        resp = requests.post('https://verifier.login.persona.org/verify', data=data)
        print('got', resp.content, file=sys.stderr)

        # Did the verifier respond?
        if resp.ok:
            # Parse the response
            verification_data = resp.json()

            # Check if the assertion was valid
            if verification_data['status'] == 'okay':
                email = verification_data['email']
                try:
                    return self.get_user(email)
                except ListUser.DoesNotExist:
                    return ListUser.objects.create(email=email)


    def get_user(self, email):
        return ListUser.objects.get(email=email)

This code is copy-pasted directly from the Mozilla site, as you can see from the explanatory comments.

You’ll need to pip install requests into your virtualenv. If you’ve never used it before, Requests is a great alternative to the Python standard library tools for HTTP requests.

To finish off the job of customising authentication in Django, we just need a custom user model:

accounts/models.py

from django.contrib.auth.models import AbstractBaseUser, PermissionsMixin
from django.db import models

class ListUser(AbstractBaseUser, PermissionsMixin):
    email = models.EmailField(primary_key=True)
    USERNAME_FIELD = 'email'
    #REQUIRED_FIELDS = ['email', 'height']

    objects = ListUserManager()

    @property
    def is_staff(self):
        return self.email == 'harry.percival@example.com'

    @property
    def is_active(self):
        return True

That’s what I call a minimal user model! One field, none of this firstname/lastname/username nonsense, and, pointedly, no password! Somebody else’s problem! But, again, you can see that this code isn’t ready for production, from the commented-out lines to the hardcoded harry email address.

Note

At this point I’d recommend a little browse through the Django auth documentation.

Aside from that, you need a model manager for the user:

accounts/models.py (ch15l006)

from django.contrib.auth.models import AbstractBaseUser, BaseUserManager, PermissionsMixin

class ListUserManager(BaseUserManager):

    def create_user(self, email):
        ListUser.objects.create(email=email)

    def create_superuser(self, email, password):
        self.create_user(email)

A logout view:

accounts/views.py (ch15l007)

from django.contrib.auth import login as auth_login, logout as auth_logout
[...]

def logout(request):
    auth_logout(request)
    return redirect('/')

Some URLs for our two views:

superlists/urls.py (ch15l008)

from accounts import urls as account_urls

urlpatterns = [
    url(r'^$', list_views.home_page, name='home'),
    url(r'^lists/', include(list_urls)),
    url(r'^accounts/', include(account_urls)),
    # url(r'^admin/', include(admin.site.urls)),
]

and

accounts/urls.py

from django.conf.urls import patterns, url

urlpatterns = [
    url(r'^login$', 'accounts.views.login', name='login'),
    url(r'^logout$', 'accounts.views.logout', name='logout'),
]

Almost there. We switch on the auth backend and our new accounts app in settings.py:

superlists/settings.py

INSTALLED_APPS = (
    #'django.contrib.admin',
    'django.contrib.auth',
    'django.contrib.contenttypes',
    'django.contrib.sessions',
    'django.contrib.messages',
    'django.contrib.staticfiles',
    'lists',
    'accounts',
)

AUTH_USER_MODEL = 'accounts.ListUser'
AUTHENTICATION_BACKENDS = (
    'accounts.authentication.PersonaAuthenticationBackend',
)

MIDDLEWARE_CLASSES = (
[...]

And a quick makemigrations to make the new user model real:

$ python3 manage.py makemigrations
Migrations for 'accounts':
  0001_initial.py:
    - Create model ListUser

And a migrate to build the database:

$ python3 manage.py migrate
[...]
Running migrations:
  Applying accounts.0001_initial... OK

And we should be all done! Why not spin up a dev server with runserver and see how it all looks (Figure 15-1)?

The Persona login screen
Figure 15-1. It works! It works! Mwahahahaha.

That’s pretty much it! Along the way, I had to fight pretty hard, including debugging Ajax requests by hand in the Firefox console (see Figure 15-2), catching infinite page-refresh loops, stumbling over several missing attributes on my custom user model (because I didn’t read the docs properly), and even one point switching to the dev version of Django to overcome a bug, which thankfully turned out to be irrelevant.

Shows the Firefox debug console open on the network tab
Figure 15-2. Debugging Ajax requests in the Firefox network console

Tip

If it’s not working when you try it manually, and you see “audience mismatch” errors in the console, make sure you’re visiting the site via http://localhost:8000, and not 127.0.0.1.

But we now have a working solution! Let’s commit it on our spike branch:

$ git status
$ git add accounts
$ git commit -am "spiked in custom auth backend with persona"

Time to de-spike!

De-spiking

De-spiking means rewriting your prototype code using TDD. We now have enough information to “do it properly”. So what’s the first step? An FT of course!

We’ll stay on the spike branch for now, to see our FT pass against our spiked code. Then we’ll go back to master, and commit just the FT.

Here’s the basic outline:

functional_tests/test_login.py

from .base import FunctionalTest

class LoginTest(FunctionalTest):

    def test_login_with_persona(self):
        # Edith goes to the awesome superlists site
        # and notices a "Sign in" link for the first time.
        self.browser.get(self.server_url)
        self.browser.find_element_by_id('login').click()

        # A Persona login box appears
        self.switch_to_new_window('Mozilla Persona')  #1

        # Edith logs in with her email address
        ## Use mockmyid.com for test email
        self.browser.find_element_by_id(
            'authentication_email'  #2
        ).send_keys('edith@mockmyid.com') #3
        self.browser.find_element_by_tag_name('button').click()

        # The Persona window closes
        self.switch_to_new_window('To-Do')

        # She can see that she is logged in
        self.wait_for_element_with_id('logout')  #4
        navbar = self.browser.find_element_by_css_selector('.navbar')
        self.assertIn('edith@mockmyid.com', navbar.text)

1 4

The FT needs a couple of helper functions, both of which do something that’s very common in Selenium testing: they wait for something to happen. Listings for them follow.

2

I found the ID of the Persona login box by opening the site manually, and using the Firefox debug toolbar (Ctrl+Shift+I). See Figure 15-3.

3

Rather than using a “real” email address and having to click through their authentication screens, we use a “fake” provider. MockMyID is one; you can also check out Persona Test User.

The Firefox debug toolbar open on the Persona screen
Figure 15-3. Using the Debug toolbar to find locators

A Common Selenium Technique: Explicit Waits

Here’s the first of the two “wait” helper functions:

functional_tests/test_login.py (ch15l014)

import time
[...]

    def switch_to_new_window(self, text_in_title):
        retries = 60
        while retries > 0:
            for handle in self.browser.window_handles:
                self.browser.switch_to_window(handle)
                if text_in_title in self.browser.title:
                    return
            retries -= 1
            time.sleep(0.5)
        self.fail('could not find window')

In this one we’ve “rolled our own” wait—we iterate through all the current browser windows, looking for one with a particular title. If we can’t find it, we do a short wait, and try again, decrementing a retry counter.

This is such a common pattern in Selenium tests that the team created an API for waiting—it doesn’t quite handle all use cases though, so that’s why we had to roll our own the first time around. When doing something simpler like waiting for an element with a given ID to appear on the page, we can use the WebDriverWait class:

functional_tests/test_login.py (ch15l015)

from selenium.webdriver.support.ui import WebDriverWait
[...]

    def wait_for_element_with_id(self, element_id):
        WebDriverWait(self.browser, timeout=30).until(
            lambda b: b.find_element_by_id(element_id)
        )

This is what Selenium calls an “explicit wait”. If you remember, we already defined an “implicit wait” in FunctionalTest.setUp. We set that to just three seconds, which is fine in most cases, but when we’re waiting for an external service like Persona, we sometimes need to bump that default timeout.

There are more examples in the Selenium docs, but I actually found reading the source code more instructive—there are good docstrings!

Tip

implicitly_wait is unreliable, especially once JavaScript is involved. Prefer the “wait-for” pattern in your FT whenever you need to check for asynchronous interactions on your pages. We’ll see this again in Chapter 20.

And if we run the FT, it works!

$ python3 manage.py test functional_tests.test_login
Creating test database for alias 'default'...
Not Found: /favicon.ico
login view
sending to mozilla {'assertion': [...]
[...]

got b'{"audience":"localhost","expires":[...]
[...]

.
 ---------------------------------------------------------------------
Ran 1 test in 32.222s

OK
Destroying test database for alias 'default'...

You can even see some of the debug output I left in my spiked view implementations. Now it’s time to revert all of our temporary changes, and reintroduce them one by one in a test-driven way.

Reverting Our Spiked Code

$ git checkout master # switch back to master branch
$ rm -rf accounts # remove any trace of spiked code
$ git add functional_tests/test_login.py
$ git commit -m "FT for login with Persona"

Now we rerun the FT and let it drive our development:

$ python3 manage.py test functional_tests.test_login
selenium.common.exceptions.NoSuchElementException: Message: Unable to locate
element: {"method":"id","selector":"login"}
[...]

The first thing it wants us to do is add a login link. Incidentally, I prefer prefixing HTML IDs with id_; it’s a convention to make it easy to tell the difference between classes and IDs in HTML and CSS. So let’s tweak the FT first:

functional_tests/test_login.py (ch15l017)

    self.browser.find_element_by_id('id_login').click()
    [...]
    self.wait_for_element_with_id('id_logout')

Next a “do-nothing” login link. Bootstrap has some built-in classes for navigation bars, so we’ll use them:

lists/templates/base.html

<div class="container">

    <nav class="navbar navbar-default" role="navigation">
        <a class="navbar-brand" href="/">Superlists</a>
        <a class="btn navbar-btn navbar-right" id="id_login" href="#">Sign in</a>
    </nav>

    <div class="row">
    [...]

After 30 seconds, that gives:

AssertionError: could not find window

License to move on! Next thing: more JavaScript.

JavaScript Unit Tests Involving External Components: Our First Mocks!

To get our FT further, we’re going to need to get the Persona window to pop up. For that, we’ll need to de-spike our client-side JavaScript code that uses the Persona libraries. We’ll test-drive that using JavaScript unit tests and mocking.

Housekeeping: A Site-Wide Static Files Folder

A bit of housekeeping first: create a site-wide static files directory inside superlists/superlists, and move all the Bootsrap CSS, QUnit code, and base.css into it, so it looks like this:

$ tree superlists -L 3 -I __pycache__
superlists
├── __init__.py
├── settings.py
├── static
│   ├── base.css
│   ├── bootstrap
│   │   ├── css
│   │   ├── fonts
│   │   └── js
│   └── tests
│       ├── qunit.css
│       └── qunit.js
├── urls.py
└── wsgi.py

6 directories, 7 files

Tip

Always do a commit before and after a bit of housekeeping like this.

That means adjusting our existing JavaScript unit tests:

lists/static/tests/tests.html (ch15l020)

    <link rel="stylesheet" href="../../../superlists/static/tests/qunit.css">

    [...]

    <script src="http://code.jquery.com/jquery.min.js"></script>
    <script src="../../../superlists/static/tests/qunit.js"></script>
    <script src="../list.js"></script>

And we check they still work, by opening them up in a browser:

2 assertions of 2 passed, 0 failed.

Here’s how we tell our settings file about the new static folder:

superlists/settings.py

[...]
STATIC_ROOT = os.path.abspath(os.path.join(BASE_DIR, '../static'))
STATICFILES_DIRS = (
    os.path.join(BASE_DIR, 'superlists', 'static'),
)

Note

I recommend reintroducing the LOGGING setting from earlier at this point. There’s no need for an explicit test for it; our current test suite will let us know in the unlikely event that it breaks anything. As we’ll find out in Chapter 17, it’ll be useful for debugging later.

And we can quickly run the layout + styling FT to check the CSS all still works:

$ python3 manage.py test functional_tests.test_layout_and_styling
[...]
OK

Next, create an app called accounts to hold all the code related to login. That will include our Persona JavaScript stuff:

$ python3 manage.py startapp accounts
$ mkdir -p accounts/static/tests

That’s the housekeeping done. Now’s a good time for a commit. Then, let’s take another look at our spiked-in javascript:

var loginLink = document.getElementById('login');
if (loginLink) {
  loginLink.onclick = function() { navigator.id.request(); };
}

Mocking: Who, Why, What?

We want our login link’s on-click to be bound to a function provided by the Persona library, navigator.id.request.

Now we don’t want to call the actual third-party function in our unit tests, because we don’t want our unit tests popping up Persona windows all over the shop. So instead, we are going to do what’s called “mocking it out”: creating a “fake” or “mock” implementation of the third-party API for our tests to run against.

What we’re going to do is replace the real navigator object with a fake one that we’ve built ourselves, one that will be able to tell us what happens to it.

Note

I had hoped that our first Mock example was going to be in Python, but it looks like it’s going to be JavaScript instead. Needs must. You may find it’s worth rereading the rest of the chapter a couple of times after you get to the end of it, to let it all sink in.

Namespacing

In the context of base.html, navigator is just an object in the global scope, as provided by the include.js <script> tag that we get from Mozilla. Testing global variables is a pain though, so we can turn it into a local variable by passing it into an “initialize”[22] function. The code we’ll end up with in base.html will look like this:

lists/templates/base.html

<script src="/static/accounts/accounts.js"></script>
<script>
    $(document).ready(function() {

        Superlists.Accounts.initialize(navigator)

    });
</script>

I’ve specified that our initialize function will be namespaced inside some nested objects, Superlists.Accounts. JavaScript suffers from a programming model that’s tied into a global scope, and this sort of namespacing/naming convention helps to keep things under control. Lots of JavaScript libraries might want to call a function initialize, but very few will call it Superlists.Accounts.initialize![23]

This call to initialize is simple enough that I’m happy it doesn’t need any unit tests of its own.

A Simple Mock to Unit Tests Our initialize Function

The initialize function itself we will test. Copy the lists tests across to get the boilerplate HTML, and then adjust the following:

accounts/static/tests/tests.html

    <div id="qunit-fixture">
        <a id="id_login">Sign in</a>
    </div>

    <script src="http://code.jquery.com/jquery.min.js"></script>
    <script src="../../../superlists/static/tests/qunit.js"></script>
    <script src="../accounts.js"></script>
    <script>
/*global $, test, equal, sinon, Superlists */

test("initialize binds sign in button to navigator.id.request", function () {
    var requestWasCalled = false; //1
    var mockRequestFunction = function () { requestWasCalled = true; }; //2
    var mockNavigator = { //3
        id: {
            request: mockRequestFunction
        }
    };

    Superlists.Accounts.initialize(mockNavigator); //4

    $('#id_login').trigger('click'); //5

    equal(requestWasCalled, true); //6
});

    </script>

One of the best ways to understand this test, or indeed any test, is to work backwards. The first thing we see is the assertion:

6

We are asserting that a variable called requestWasCalled is true. We’re checking that, one way or another, the request function, as in navigator.id.request, was called.

5

Called when? When a click event happens to the id_login element.

4

Before we trigger that click event, we call our Superlists.Accounts.initialize function, just like we will on the real page. The only difference is, instead of passing it the real global navigator object from Persona, we pass in a fake one called mockNavigator.[24]

3

That’s defined as a generic JavaScript object, with an attribute called id which in turn has an attribute called request, which we’re assigning to a variable called mockRequestFunction.

2

mockRequestFunction we define as a very simple function, which if called will simply set the value of the requestWasCalled variable to true.

1

And finally (firstly?) we make sure that requestWasCalled starts out as false.

The upshot of all this is: the only way this test will pass is if our initialize function binds the click event on id_login to the method .id.request of the object we pass it. If we get the tests passing when we use the mock object, we are reassured that our initialize function will also do the right thing when we give it a real object on our real page.

Does that make sense? Let’s play around with the test and see if we can get the hang of it.

Tip

When testing events on DOM elements, you need an actual element to trigger events against, and to register listeners on. If you forget, it’s a particularly fiendish test bug, because .trigger will just silently no-op, and you’ll be left scratching your head about why it’s not working. So don’t forget to add the <a id="id_login"> inside the qunit-fixture div!

Our first error is this:

1. Died on test #1
@file:///workspace/superlists/accounts/static/tests/tests.html:35:
Superlists is not defined

That’s the equivalent of an ImportError in Python. Let’s start work on accounts/static/accounts.js:

accounts/static/accounts.js

window.Superlists = null;

Just as in Python we might do Superlists = None, here we do window.Superlists = null. Using window. makes sure we get the global object:

1. Died on test #1
@file:///workspace/superlists/accounts/static/tests/tests.html:35:
Superlists is null

OK, next baby step or two:

accounts/static/accounts.js

window.Superlists = {
    Accounts: {}
};

gives:[25]

Superlists.Accounts.initialize is not a function

So let’s make it a function:

accounts/static/accounts.js

window.Superlists = {
    Accounts: {
        initialize: function () {}
    }
};

And now we get a real test failure instead of just errors:

1. initialize binds sign in button to navigator.id.request (1, 0, 1)

    1. failed
        Expected: true
        Result: false

Next—let’s separate defining our initialize function from the part where we export it into the Superlists namespace. We’ll also do a console.log, which is the JavaScript equivalent of a debug-print, to take a look at what the initialize function is being called with:

accounts/static/accounts.js (ch15l028)

var initialize = function (navigator) {
    console.log(navigator);
};

window.Superlists = {
    Accounts: {
        initialize: initialize
    }
};

In Firefox and I believe Chrome also, you can use the shortcut Ctrl-Shift-I to bring up the JavaScript console, and see the [object Object] that was logged (see Figure 15-4). If you click on it, you can see it has the properties we defined in our test: an id, and inside that, a function called request.

The JavaScript console in our qunit run, showing the console.log
Figure 15-4. Debugging in the JavaScript console

So let’s now just pile in and get the test to pass:

accounts/static/accounts.js (ch15l029)

var initialize = function (navigator) {
    navigator.id.request();
};

That gets the tests to pass, but it’s not quite the implementation we want. We’re calling navigator.id.request always, instead of only on click. We’ll need to adjust our tests.

1 assertions of 1 passed, 0 failed.
1. initialize binds sign in button to navigator.id.request (0, 1, 1)

Before we do, let’s just have a play around to see if we really understand what’s going on. What happens if we do this?

accounts/static/accounts.js (ch15l029-1)

var initialize = function (navigator) {
    navigator.id.request();
    navigator.id.doSomethingElse();
};

We get:

1. Died on test #1
@file:///workspace/superlists/accounts/static/tests/tests.html:35:
navigator.id.doSomethingElse is not a function

You see, the mock navigator object that we pass in is entirely under our control. It has only the attributes and methods we give it. You can play around with it now if you like:

accounts/static/tests/tests.html

    var mockNavigator = {
        id: {
            request: mockRequestFunction,
            doSomethingElse: function () { console.log("called me!");}
        }
    };

That will give you a pass, and if you open up the debug window, you’ll see:

[01:22:27.456] "called me!"

Does that help to see what’s going on? Let’s revert those last two changes, and tweak our unit test so that it checks the request function is only called after we fire off the click event. We also add some error messages to help see which of the two equal assertions is failing:

accounts/static/tests/tests.html (ch15l032)

    var mockNavigator = {
        id: {
            request: mockRequestFunction
        }
    };
    Superlists.Accounts.initialize(mockNavigator);
    equal(requestWasCalled, false, 'check request not called before click');
    $('#id_login').trigger('click');
    equal(requestWasCalled, true, 'check request called after click');

Note

Assertion messages (the third argument to equal), in QUnit, are actually “success” messages. Rather than only being displayed if the test fails, they are also displayed when the test passes. That’s why they have the positive phrasing.

Now we get a neater failure:

1 assertions of 2 passed, 1 failed.
1. initialize binds sign in button to navigator.id.request (1, 1, 2)
    1. check request not called before click
        Expected: false
        Result: true

So let’s make it so that the call to navigator.id.request only happens if our id_login is clicked:

accounts/static/accounts.js (ch15l033)

/*global $ */

var initialize = function (navigator) {
    $('#id_login').on('click', function () {
        navigator.id.request();
    });
};
[...]

That passes. A good start! Let’s try pulling it into our template:

lists/templates/base.html

<script src="http://code.jquery.com/jquery.min.js"></script>
<script src="https://login.persona.org/include.js"></script>
<script src="/static/accounts.js"></script>
<script src="/static/list.js"></script>
<script>
    /*global $, Superlists, navigator */
    $(document).ready(function () {
        Superlists.Accounts.initialize(navigator);
    });
</script>
</body>

We also need to add the accounts app to settings.py, otherwise it won’t be serving the static file at accounts/static/accounts.js:

superlists/settings.py

+++ b/superlists/settings.py
@@ -37,4 +37,5 @@ INSTALLED_APPS = (
     'lists',
+    'accounts',
 )

A quick check on the FT…doesn’t get any further unfortunately. To see why, we can open up the site manually, and check the JavaScript debug console:

[01:36:54.572] Error: navigator.id.watch must be called before
navigator.id.request @ https://login.persona.org/include.js:8

More Advanced Mocking

We now need to call Mozilla’s navigator.id.watch correctly. Taking another look at our spike, it should be something like this:

var currentUser = '{{ user.email }}' || null;
var csrf_token = '{{ csrf_token }}';
console.log(currentUser);

navigator.id.watch({
  loggedInUser: currentUser, //1
  onlogin: function(assertion) {
    $.post('/accounts/login', {assertion: assertion, csrfmiddlewaretoken: csrf_token}) //2
    .done(function() { window.location.reload(); })
    .fail(function() { navigator.id.logout();});
  },
  onlogout: function() {
    $.post('/accounts/logout')
    .always(function() { window.location.reload(); });
  }
});

Decoding that, the watch function needs to know a couple of things from the global scope:

1

The current user’s email, to be passed in as the loggedInUser parameter to watch.

2

The current CSRF token, to pass in the Ajax POST request to the login view.[26]

We’ve also got two hardcoded URLs in there, which would be better to get from Django, something like this:

var urls = {
    login: "{% url 'login' %}",
    logout: "{% url 'logout' %}",
};

So that would be a third parameter to pass in from the global scope. We’ve already got an initialize function, so let’s imagine using it like this:

Superlists.Accounts.initialize(navigator, user, token, urls);

Using a sinon.js mock to check we call the API correctly

“Rolling your own” mocks is possible as we’ve seen, and JavaScript actually makes it relatively easy, but using a mocking library can save us a lot of heavy lifting. The most popular one in the JavaScript world is called sinon.js. Let’s download it (from http://sinonjs.org) and put it in our site-wide static tests folder:

$ tree superlists/static/tests/
superlists/static/tests/
├── qunit.css
├── qunit.js
└── sinon.js

Next we include it in our accounts tests:

accounts/static/tests/tests.html

    <script src="http://code.jquery.com/jquery.min.js"></script>
    <script src="../../../superlists/static/tests/qunit.js"></script>
    <script src="../../../superlists/static/tests/sinon.js"></script>
    <script src="../accounts.js"></script>

And now we can write a test that uses Sinon’s mock object:[27]

accounts/static/tests/tests.html (ch15l038)

test("initialize calls navigator.id.watch", function () {
    var user = 'current user';
    var token = 'csrf token';
    var urls = {login: 'login url', logout: 'logout url'};
    var mockNavigator = {
        id: {
            watch: sinon.mock() //1
        }
    };

    Superlists.Accounts.initialize(mockNavigator, user, token, urls);

    equal(
        mockNavigator.id.watch.calledOnce, //2
        true,
        'check watch function called'
    );
});

1

We create a mock navigator object as before, but now instead of hand-crafting a function to mock out the function we’re interested in, we use a sinon.mock() object.

2

This object then records what happens to it inside special properties like calledOnce, which we can make assertions against.

There’s more info in the Sinon docs—the front page actually has quite a good overview.

Here’s our expected test failure:

2 assertions of 3 passed, 1 failed.

1. initialize binds sign in button to navigator.id.request (0, 2, 2)
2. initialize calls navigator.id.watch (1, 0, 1)
    1. check watch function called
        Expected: true
        Result: false

We add in the call to watch

accounts/static/accounts.js

var initialize = function (navigator) {
    $('#id_login').on('click', function () {
        navigator.id.request();
    });

    navigator.id.watch();
};

But that breaks the other test!

1 assertions of 2 passed, 1 failed.

1. initialize binds sign in button to navigator.id.request (1, 0, 1)
    1. Died on test #1
@file:///workspace/superlists/accounts/static/tests/tests.html:36:
missing argument 1 when calling function navigator.id.watch

2. initialize calls navigator.id.watch (0, 1, 1)

That was a puzzler—that “missing argument 1 when calling function navigator.id.watch” took me a while to figure out. Turns out that, in Firefox, .watch is a function on every object. We’ll need to mock it out in the previous test too:

accounts/static/tests/tests.html

test("initialize binds sign in button to navigator.id.request", function () {
    var requestWasCalled = false;
    var mockRequestFunction = function () { requestWasCalled = true; };
    var mockNavigator = {
        id: {
            request: mockRequestFunction,
            watch: function () {}
        }
    };
    [...]

And we’re back to passing tests:

3 assertions of 3 passed, 0 failed.

1. initialize binds sign in button to navigator.id.request (0, 2, 2)
2. initialize calls navigator.id.watch (0, 1, 1)

Checking Call Arguments

We’re not calling the watch function correctly yet—it needs to know the current user, and we have to set up a couple of callbacks for login and logout. Let’s start with the user:

accounts/static/tests/tests.html (ch15l042)

test("watch sees current user", function () {
    var user = 'current user';
    var token = 'csrf token';
    var urls = {login: 'login url', logout: 'logout url'};
    var mockNavigator = {
        id: {
            watch: sinon.mock()
        }
    };

    Superlists.Accounts.initialize(mockNavigator, user, token, urls);
    var watchCallArgs = mockNavigator.id.watch.firstCall.args[0];
    equal(watchCallArgs.loggedInUser, user, 'check user');
});

We have a very similar setup (which is a code smell incidentally—on the next test, we’re going to want to do some de-duplication of test code). Then we use the .firstCall.args[0] property on the mock to check on the parameter we passed to the watch function (args being a list of positional arguments). That gives us:

3. watch sees current user (1, 0, 1)
    1. Died on test #1
@file:///workspace/superlists/accounts/static/tests/tests.html:72:
watchCallArgs is undefined

because we’re not currently passing any arguments to watch. Step by step, we can do:

accounts/static/accounts.js (ch15l043)

    navigator.id.watch({});

and get a clearer error message:

3. watch sees current user (1, 0, 1)
    1. check user
        Expected: "current user"
        Result: undefined

and fix it thusly:

accounts/static/accounts.js (ch15l044)

var initialize = function (navigator, user, token, urls) {
    [...]

    navigator.id.watch({
        loggedInUser: user
    });

Good.

4 assertions of 4 passed, 0 failed.

QUnit setup and teardown, Testing Ajax

Next we need to check the onlogin callback, which is called when Persona has some user authentication information, and we need to send it up to our server for validation. That involves an Ajax call ($.post), and they’re normally quite hard to test, but sinon.js has a helper called fake XMLHttpRequest.

This patches out the native JavaScript XMLHttpRequest class, so it’s good practice to make sure we restore it afterwards. This gives us a good excuse to learn about QUnit’s setup and teardown methods—they are used in a function called module, which acts a bit like a unittest.TestCase class, and groups all the tests that follow it together.

Let’s add this “module” after the first test, before the test for "initialize calls navigator.id.watch":

accounts/static/tests/tests.html (ch15l045)

var user, token, urls, mockNavigator, requests, xhr; //1
module("navigator.id.watch tests", {
    setup: function () {
        user = 'current user'; //2
        token = 'csrf token';
        urls = { login: 'login url', logout: 'logout url' };
        mockNavigator = {
            id: {
                watch: sinon.mock()
            }
        };
        xhr = sinon.useFakeXMLHttpRequest(); //3
        requests = []; //4
        xhr.onCreate = function (request) { requests.push(request); }; //5
    },
    teardown: function () {
        mockNavigator.id.watch.reset(); //6
        xhr.restore(); //7
    }
});

test("initialize calls navigator.id.watch", function () {
    [...]

1

We pull out the variables user, token, urls, etc. up to a higher scope, so that they’ll be available to all of the tests in the file.

2

We initialise said variables inside the setup function, which, just like a unittest setUp function, will run before each test. That includes our mockNavigator.

3

We also invoke Sinon’s useFakeXMLHttpRequest, which patches out the browser’s Ajax capabilities.

4 5

There’s one more bit of boilerplate: we tell Sinon to take any Ajax requests and put them into the requests array, so that we can inspect them in our tests.

6

Finally we have the cleanup—we “reset” the mock for the watch function in between each test (otherwise calls from one test would show up in others).

7

And we put the JavaScript XMLHttpRequest back to the way we found it.

That lets us rewrite our two tests to be much shorter:

accounts/static/tests/tests.html (ch15l046)

test("initialize calls navigator.id.watch", function () {
    Superlists.Accounts.initialize(mockNavigator, user, token, urls);
    equal(mockNavigator.id.watch.calledOnce, true, 'check watch function called');
});


test("watch sees current user", function () {
    Superlists.Accounts.initialize(mockNavigator, user, token, urls);
    var watchCallArgs = mockNavigator.id.watch.firstCall.args[0];
    equal(watchCallArgs.loggedInUser, user, 'check user');
});

And they still pass, but their name is neatly prefixed with our module name:

4 assertions of 4 passed, 0 failed.

1. initialize binds sign in button to navigator.id.request (0, 2, 2)
2. navigator.id.watch tests: initialize calls navigator.id.watch (0, 1, 1)
3. navigator.id.watch tests: watch sees current user (0, 1, 1)

And here’s how we test the onlogin callback:

accounts/static/tests/tests.html (ch15l047)

test("onlogin does ajax post to login url", function () {
    Superlists.Accounts.initialize(mockNavigator, user, token, urls);
    var onloginCallback = mockNavigator.id.watch.firstCall.args[0].onlogin; //1
    onloginCallback(); //2
    equal(requests.length, 1, 'check ajax request'); //3
    equal(requests[0].method, 'POST');
    equal(requests[0].url, urls.login, 'check url');
});

test("onlogin sends assertion with csrf token", function () {
    Superlists.Accounts.initialize(mockNavigator, user, token, urls);
    var onloginCallback = mockNavigator.id.watch.firstCall.args[0].onlogin;
    var assertion = 'browser-id assertion';
    onloginCallback(assertion);
    equal(
        requests[0].requestBody,
        $.param({ assertion: assertion, csrfmiddlewaretoken: token }), //4
        'check POST data'
    );
});

1

The mock we set on the mock navigator’s watch function lets us extract the callback function we set as “onlogin.”

2

We can then actually call that function in order to test it.

3

Sinon’s fakeXMLHttpRequest server will catch any Ajax requests we make, and put them into the requests array. We can then check on things like whether it was a POST and what URL it was sent to.

4

The actual POST parameters are held in .requestBody, but they are URL-encoded (using the &key=val syntax). jQuery’s $.param function does URL-encoding, so we use that to do our comparison.

And the two tests fail as expected:

4. navigator.id.watch tests: onlogin does ajax post to login url (1, 0, 1)
    1. Died on test #1
@file:///workspace/superlists/accounts/static/tests/tests.html:78:
onloginCallback is not a function

5. navigator.id.watch tests: onlogin sends assertion with csrf token (1, 0, 1)
    1. Died on test #1
@file:///workspace/superlists/accounts/static/tests/tests.html:90:
onloginCallback is not a function

Another unit-test/code cycle. Here’s the failure messages I went through:

1. check ajax request
Expected: 1

3. check url
Expected: "login url"

7 assertions of 8 passed, 1 failed.
1. check POST data
Expected:
"assertion=browser-id+assertion&csrfmiddlewaretoken=csrf+token"
Result: null

1. check POST data
Expected:
"assertion=browser-id+assertion&csrfmiddlewaretoken=csrf+token"
Result: "assertion=browser-id+assertion"

8 assertions of 8 passed, 0 failed.

And I ended up with this code:

accounts/static/accounts.js

    navigator.id.watch({
        loggedInUser: user,
        onlogin: function (assertion) {
            $.post(
                urls.login,
                { assertion: assertion, csrfmiddlewaretoken: token }
            );
        }
    });

Logout

At the time of writing, the “onlogout” part of the watch API’s status was uncertain. It works, but it’s not necessary for our purposes. We’ll just make it a do-nothing function, as a placeholder. Here’s a minimal test for that:

accounts/static/tests/tests.html (ch15l053)

test("onlogout is just a placeholder", function () {
    Superlists.Accounts.initialize(mockNavigator, user, token, urls);
    var onlogoutCallback = mockNavigator.id.watch.firstCall.args[0].onlogout;
    equal(typeof onlogoutCallback, "function", "onlogout should be a function");
});

And we get quite a simple logout function:

accounts/static/accounts.js (ch15l054)

    },
    onlogout: function () {}
});

More Nested Callbacks! Testing Asynchronous Code

This is what JavaScript’s all about folks! Thankfully, sinon.js really does help. We still need to test that our login post methods also set some callbacks for things to do after the POST request comes back:

    .done(function() { window.location.reload(); })
    .fail(function() { navigator.id.logout();});

I’m going to skip testing the window.location.reload, because it’s a bit unnecessarily complicated,[28] and I think we can allow that this will be tested by our Selenium test. We will do a test for the on-fail callback though, just to demonstrate that it is possible:

accounts/static/tests/tests.html (ch15l055)

test("onlogin post failure should do navigator.id.logout ", function () {
    mockNavigator.id.logout = sinon.mock(); //1
    Superlists.Accounts.initialize(mockNavigator, user, token, urls);
    var onloginCallback = mockNavigator.id.watch.firstCall.args[0].onlogin;
    var server = sinon.fakeServer.create(); //2
    server.respondWith([403, {}, "permission denied"]); //3

    onloginCallback();
    equal(mockNavigator.id.logout.called, false, 'should not logout yet');

    server.respond(); //4
    equal(mockNavigator.id.logout.called, true, 'should call logout');
});

1

We put a mock on the navigator.id.logout function which we’re interested in.

2

We use Sinon’s fakeServer, which is an abstraction on top of the fakeXMLHttpRequest to simulate Ajax server responses.

3

We set up our fake server to respond with a 403 “permission denied” response, to simulate what will happen for unauthorized users.

4

We then explicitly tell the fake server to send that response. Only then should we see the logout call.

That gets us to this—a slight change to our spiked code:

accounts/static/accounts.js (ch15l056)

    onlogin: function (assertion) {
        $.post(
            urls.login,
            { assertion: assertion, csrfmiddlewaretoken: token }
        ).fail(function () { navigator.id.logout(); });
    },
    onlogout: function () {}

Finally we add our window.location.reload, just to check it doesn’t break any unit tests:

accounts/static/accounts.js (ch15l057)

    navigator.id.watch({
        loggedInUser: user,
        onlogin: function (assertion) {
            $.post(
                urls.login,
                { assertion: assertion, csrfmiddlewaretoken: token }
            )
                .done(function () { window.location.reload(); })
                .fail(function () { navigator.id.logout(); });
        },
        onlogout: function () {}
    });

Everything’s still OK:

11 assertions of 11 passed, 0 failed.

If those chained .done and .fail calls are bugging you—they bug me a little—you can rewrite that as, eg:

    var deferred = $.post(
        urls.login,
        { assertion: assertion, csrfmiddlewaretoken: token }
    );
    deferred.done(function () { window.location.reload(); })
    deferred.fail(function () { navigator.id.logout(); });

But async code is always a bit mind-bending. I find it just about readable as it is: “do a post to urls.login with the assertion and csrf token, when it’s done, do a window reload, or if it fails, do a navigator.id.logout”. You can read up on JavaScript deferreds, aka “promises”, here.

We’re approaching the moment of truth: will our FTs get any further? First, we adjust our initialize call:

lists/templates/base.html

<script>
    /*global $, Superlists, navigator */
    $(document).ready(function () {
        var user = "{{ user.email }}" || null;
        var token = "{{ csrf_token }}";
        var urls = {
            login: "TODO",
            logout: "TODO",
        };
        Superlists.Accounts.initialize(navigator, user, token, urls);
    });
</script>

And we run the FT…

$ python3 manage.py test functional_tests.test_login
Creating test database for alias 'default'...
Not Found: /favicon.ico
Not Found: /TODO
E
======================================================================
ERROR: test_login_with_persona (functional_tests.test_login.LoginTest)
 ---------------------------------------------------------------------
Traceback (most recent call last):
  File "/workspace/superlists/functional_tests/test_login.py", line 47, in
test_login_with_persona
    self.wait_for_element_with_id('id_logout')
  File "/workspace/superlists/functional_tests/test_login.py", line 23, in
wait_for_element_with_id
    lambda b: b.find_element_by_id(element_id)
[...]
selenium.common.exceptions.TimeoutException: Message:

 ---------------------------------------------------------------------
Ran 1 test in 28.779s

FAILED (errors=1)
Destroying test database for alias 'default'...

Hooray! I mean, I know it failed, but we saw it popping up the Persona dialog and getting through it and everything! Next chapter: the server side.



[22] UK-English speakers may bristle at that incorrect spelling of the word “initialise”. I know, it grates with me too. But it’s an increasingly accepted convention to use American spelling in code. It makes it easier to search, for example, and just to work together more generally, if we all agree on how words are spelt. We have to accept that we’re in the minority here, and this is one battle we’ve probably lost.

[23] The new shiny in the JavaScript world for avoiding namespacing problems is called require.js. It was one thing too many to squeeze into this book, but you should check it out.

[24] I’ve called this object a “mock”, but it’s probably more correctly called a “spy”. We don’t have to concern ourselves with the differences in this book, but for more on the general class of tools called “Test Doubles”, including the difference between stubs, mocks, fakes, and spies, see Mocks, Fakes and Stubs by Emily Bache.

[25] In the real world, when setting up a namespace like this, you’d want to follow a sort of “add-or-create” pattern, so that, if there’s already a window.Superlists in the scope, we extend it rather than replacing it. window.Superlists = window.Superlists || {} is one formulation, and jQuery’s $.extend is another possibilty. But, there’s already a lot of content in this chapter, and I thought this was probably one too many things to talk about!

[26] Incidentally, notice we use {{ csrf_token }} which gives you the raw string token, rather than {% csrf_token%} which would give us a full HTML tag, <input type="hidden" name="etc etc.

[27] Sinon also has more specialised objects for “spies” and “stubs”. Mocks can do everything that spies and stubs can do though, so I figured, one less piece of terminology would keep things simple.

[28] You can’t mock out window.location.reload, so instead you have to define an (untested) function called Superlists.Accounts.refreshPage, and then put a mock on that to check that it gets set as the Ajax .done callback.

Get Test-Driven Development with Python now with the O’Reilly learning platform.

O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.