Test-Driven Development with Python

Chapter 10. Input Validation and Test Organisation

Over the next few chapters we’ll talk about testing and implementing validation of user inputs. We’ll also take the opportunity to do a little tidying up—both in our application code, and also in our tests.

Validation FT: Preventing Blank Items

As our first few users start using the site, we’ve noticed they sometimes make mistakes that mess up their lists, like accidentally submitting blank list items, or accidentally inputting two identical items to a list. Computers are meant to help stop us from making silly mistakes, so let’s see if we can get our site to help.

Here’s the outline of an FT:

functional_tests/tests.py (ch10l001).

def test_cannot_add_empty_list_items(self):
    # Edith goes to the home page and accidentally tries to submit
    # an empty list item. She hits Enter on the empty input box

    # The home page refreshes, and there is an error message saying
    # that list items cannot be blank

    # She tries again with some text for the item, which now works

    # Perversely, she now decides to submit a second blank list item

    # She receives a similar warning on the list page

    # And she can correct it by filling some text in
    self.fail('write me!')

That’s all very well, but before we go any further—our functional tests file is beginning to get a little crowded. Let’s split it out into several files, in which each has a single test method.

Remember that functional tests are closely linked to “user stories”. If you were using some sort of project management tool like an issue tracker, you might make it so that each file matched one issue or ticket, and its filename contained the ticket ID. Or, if you prefer to think about things in terms of “features”, where one feature may have several user stories, then you might have one file and class for the feature, and several methods for each of its user stories.

We’ll also have one base test class which they can all inherit from. Here’s how to get there step by step.

Skipping a Test

It’s always nice, when doing refactoring, to have a fully passing test suite. We’ve just written a test with a deliberate failure. Let’s temporarily switch it off, using a decorator called “skip” from unittest:

functional_tests/tests.py (ch10l001-1).

from unittest import skip
[...]

    @skip
    def test_cannot_add_empty_list_items(self):

This tells the test runner to ignore this test. You can see it works—if we rerun the tests, it’ll say it passes:

$ python3 manage.py test functional_tests
[...]
Ran 3 tests in 11.577s
OK

Warning

Skips are dangerous—you need to remember to remove them before you commit your changes back to the repo. This is why line-by-line reviews of each of your diffs are a good idea!

A criticism that’s sometimes levelled at TDD is that it leads to badly architected code, as the developer just focuses on getting tests to pass rather than stopping to think about how the whole system should be designed. I think it’s slightly unfair.

TDD is no silver bullet. You still have to spend time thinking about good design. But what often happens is that people forget the “Refactor” in “Red, Green, Refactor”. The methodology allows you to throw together any old code to get your tests to pass, but it also asks you to then spend some time refactoring it to improve its design.

Often, however, the best ideas for how to refactor code don’t occur to you straight away. They may occur to you days, weeks, even months after you wrote a piece of code, when you’re working on something totally unrelated and you happen to see some old code again with fresh eyes. But if you’re halfway through something else, should you stop to refactor the old code?

The answer is that it depends. In the case at the beginning of the chapter, we haven’t even started writing our new code. We know we are in a working state, so we can justify putting a skip on our new FT (to get back to fully passing tests) and do a bit of refactoring straight away.

Later in the chapter we’ll spot other bits of code we want to alter. In those cases, rather than taking the risk of refactoring an application that’s not in a working state, we’ll make a note of the thing we want to change on our scratchpad and wait until we’re back to a fully passing test suite before refactoring.

Splitting Functional Tests out into Many Files

We start putting each test into its own class, still in the same file:

functional_tests/tests.py (ch10l002).

class FunctionalTest(StaticLiveServerTestCase):

    @classmethod
    def setUpClass(cls):
        [...]
    @classmethod
    def tearDownClass(cls):
        [...]
    def setUp(self):
        [...]
    def tearDown(self):
        [...]
    def check_for_row_in_list_table(self, row_text):
        [...]


class NewVisitorTest(FunctionalTest):

    def test_can_start_a_list_and_retrieve_it_later(self):
        [...]



class LayoutAndStylingTest(FunctionalTest):

    def test_layout_and_styling(self):
        [...]



class ItemValidationTest(FunctionalTest):

    @skip
    def test_cannot_add_empty_list_items(self):
        [...]

At this point we can rerun the FTs and see they all still work:

Ran 3 tests in 11.577s

OK

That’s labouring it a little bit, and we could probably get away doing this stuff in fewer steps, but, as I keep saying, practising the step-by-step method on the easy cases makes it that much easier when we have a complex case.

Now we switch from a single tests file to using one for each class, and one “base” file to contain the base class all the tests will inherit from. We’ll make four copies of tests.py, naming them appropriately, and then delete the parts we don’t need from each:

$ git mv functional_tests/tests.py functional_tests/base.py
$ cp functional_tests/base.py functional_tests/test_simple_list_creation.py
$ cp functional_tests/base.py functional_tests/test_layout_and_styling.py
$ cp functional_tests/base.py functional_tests/test_list_item_validation.py

base.py can be cut down to just the FunctionalTest class. We leave the helper method on the base class, because we suspect we’re about to reuse it in our new FT:

functional_tests/base.py (ch10l003).

from django.contrib.staticfiles.testing import StaticLiveServerTestCase
from selenium import webdriver
import sys

class FunctionalTest(StaticLiveServerTestCase):

    @classmethod
    def setUpClass(cls):
        [...]
    def tearDownClass(cls):
        [...]
    def setUp(self):
        [...]
    def tearDown(self):
        [...]
    def check_for_row_in_list_table(self, row_text):
        [...]

Note

Keeping helper methods in a base FunctionalTest class is one useful way of preventing duplication in FTs. Later in the book (in Chapter 21) we’ll use the “Page pattern”, which is related, but prefers composition over inheritance.

Our first FT is now in its own file, and should be just one class and one test method:

functional_tests/test_simple_list_creation.py (ch10l004).

from .base import FunctionalTest
from selenium import webdriver
from selenium.webdriver.common.keys import Keys

class NewVisitorTest(FunctionalTest):

    def test_can_start_a_list_and_retrieve_it_later(self):
        [...]

I used a relative import (from .base). Some people like to use them a lot in Django code (e.g., your views might import models using from .models import List, instead of from list.models). Ultimately this is a matter of personal preference. I prefer to use relative imports only when I’m super-super sure that the relative position of the thing I’m importing won’t change. That applies in this case because I know for sure all the tests will sit next to base.py, which they inherit from.

The layout and styling FT should now be one file and one class:

functional_tests/test_layout_and_styling.py (ch10l005).

from .base import FunctionalTest

class LayoutAndStylingTest(FunctionalTest):
        [...]

Lastly our new validation test is in a file of its own too:

functional_tests/test_list_item_validation.py (ch10l006).

from unittest import skip
from .base import FunctionalTest

class ItemValidationTest(FunctionalTest):

    @skip
    def test_cannot_add_empty_list_items(self):
        [...]

And we can test everything worked by rerunning manage.py test functional_tests, and checking once again that all three tests are run:

Ran 3 tests in 11.577s

OK

Now we can remove our skip:

functional_tests/test_list_item_validation.py (ch10l007).

class ItemValidationTest(FunctionalTest):

    def test_cannot_add_empty_list_items(self):
        [...]

Running a Single Test File

As a side bonus, we’re now able to run an individual test file, like this:

$ python3 manage.py test functional_tests.test_list_item_validation
[...]
AssertionError: write me!

Brilliant, no need to sit around waiting for all the FTs when we’re only interested in a single one. Although we need to remember to run all of them now and again, to check for regressions. Later in the book we’ll see how to give that task over to an automated Continuous Integration loop. For now let’s commit!

$ git status
$ git add functional_tests
$ git commit -m "Moved Fts into their own individual files"

Fleshing Out the FT

Now let’s start implementing the test, or at least the beginning of it:

functional_tests/test_list_item_validation.py (ch10l008).

def test_cannot_add_empty_list_items(self):
    # Edith goes to the home page and accidentally tries to submit
    # an empty list item. She hits Enter on the empty input box
    self.browser.get(self.server_url)
    self.browser.find_element_by_id('id_new_item').send_keys('\n')

    # The home page refreshes, and there is an error message saying
    # that list items cannot be blank
    error = self.browser.find_element_by_css_selector('.has-error') #
    self.assertEqual(error.text, "You can't have an empty list item")

    # She tries again with some text for the item, which now works
    self.browser.find_element_by_id('id_new_item').send_keys('Buy milk\n')
    self.check_for_row_in_list_table('1: Buy milk') #

    # Perversely, she now decides to submit a second blank list item
    self.browser.find_element_by_id('id_new_item').send_keys('\n')

    # She receives a similar warning on the list page
    self.check_for_row_in_list_table('1: Buy milk')
    error = self.browser.find_element_by_css_selector('.has-error')
    self.assertEqual(error.text, "You can't have an empty list item")

    # And she can correct it by filling some text in
    self.browser.find_element_by_id('id_new_item').send_keys('Make tea\n')
    self.check_for_row_in_list_table('1: Buy milk')
    self.check_for_row_in_list_table('2: Make tea')

A couple of things to note about this test:

: We specify we’re going to use a CSS class from Bootstrap called .has-error to mark our error text. We’ll see that Bootstrap has some useful styling for those
: As predicted, we are reusing the check_for_row_in_list_table helper function when we want to confirm that list item submission does work.

The technique of keeping helper methods in a parent class is meant to prevent duplication across your functional test code. The day we decide to change the implementation of how our list table works, we want to make sure we only have to change our FT code in one place, not in dozens of places across loads of FTs…

And we’re off!

selenium.common.exceptions.NoSuchElementException: Message: Unable to locate
element: {"method":"css selector","selector":".has-error"}

I’ll let you do your own “first-cut FT” commit.

Using Model-Layer Validation

There are two levels at which you can do validation in Django. One is at the model level, and the other is higher up at the forms level. I like to use the lower level whenever possible, partially because I’m a bit too fond of databases and database integrity rules, and partially because it’s safer—you can sometimes forget which form you use to validate input, but you’re always going to use the same database.

Refactoring Unit Tests into Several Files

We’re going to want to add another test for our model, but before we do so, it’s time to tidy up our unit tests in a similar way to the functional tests. A difference will be that, because the lists app contains real application code as well as tests, we’ll separate out the tests into their own folder:

$ mkdir lists/tests
$ touch lists/tests/__init__.py
$ git mv lists/tests.py lists/tests/test_all.py
$ git status
$ git add lists/tests
$ python3 manage.py test lists
[...]
Ran 10 tests in 0.034s

OK
$ git commit -m "Move unit tests into a folder with single file"

If you get a message saying “Ran 0 tests”, you probably forgot to add the dunderinit—it needs to be there or else the tests folder isn’t a valid Python package…^[16]

Now we turn test_all.py into two files, one called test_views.py, which only contains view tests, and one called test_models.py:

$ git mv lists/tests/test_all.py lists/tests/test_views.py
$ cp lists/tests/test_views.py lists/tests/test_models.py

We strip test_models.py down to being just the one test—it means it needs far fewer imports:

lists/tests/test_models.py (ch10l009).

from django.test import TestCase
from lists.models import Item, List


class ListAndItemModelsTest(TestCase):
        [...]

Whereas test_views.py just loses one class:

lists/tests/test_views.py (ch10l010).

--- a/lists/tests/test_views.py
+++ b/lists/tests/test_views.py
@@ -103,34 +104,3 @@ class ListViewTest(TestCase):
         self.assertNotContains(response, 'other list item 1')
         self.assertNotContains(response, 'other list item 2')

-
-
-class ListAndItemModelsTest(TestCase):
-
-    def test_saving_and_retrieving_items(self):
[...]

We rerun the tests to check everything is still there:

$ python3 manage.py test lists
[...]
Ran 10 tests in 0.040s

OK

Great!

$ git add lists/tests
$ git commit -m "Split out unit tests into two files"

Note

Some people like to make their unit tests into a tests folder straight away, as soon as they start a project, with the addition of another file, test_forms.py. That’s a perfectly good idea; I just thought I’d wait until it became necessary, to avoid doing too much housekeeping all in the first chapter!

Unit Testing Model Validation and the self.assertRaises Context Manager

Let’s add a new test method to ListAndItemModelsTest, which tries to create a blank list item:

lists/tests/test_models.py (ch10l012-1).

from django.core.exceptions import ValidationError
[...]

class ListAndItemModelsTest(TestCase):
    [...]

    def test_cannot_save_empty_list_items(self):
        list_ = List.objects.create()
        item = Item(list=list_, text='')
        with self.assertRaises(ValidationError):
            item.save()

Tip

If you’re new to Python, you may never have seen the with statement. It’s used with what are called “context managers”, which wrap a block of code, usually with some kind of setup, cleanup, or error-handling code. There’s a good write-up in the Python 2.5 release notes.

This is a new unit testing technique: when we want to check that doing something will raise an error, we can use the self.assertRaises context manager. We could have used something like this instead:

try:
    item.save()
    self.fail('The save should have raised an exception')
except ValidationError:
    pass

But the with formulation is neater. Now, we can try running the test, and see it fail:

    item.save()
AssertionError: ValidationError not raised

A Django Quirk: Model Save Doesn’t Run Validation

And now we discover one of Django’s little quirks. This test should already pass. If you take a look at the docs for the Django model fields, you’ll see that TextField actually defaults to blank=False, which means that it should disallow empty values.

So why is the test not failing? Well, for slightly counterintuitive historical reasons, Django models don’t run full validation on save. As we’ll see later, any constraints that are actually implemented in the database will raise errors on save, but SQLite doesn’t support enforcing emptiness constraints on text columns, and so our save method is letting this invalid value through silently.

There’s a way of checking whether the constraint will happen at the database level or not: if it was at the database level, we would need a migration to apply the constraint. But, Django knows that SQLite doesn’t support this type of constraint, so if we try and run makemigrations, it will report there’s nothing to do:

$ python3 manage.py makemigrations
No changes detected

Django does have a method to manually run full validation however, called full_clean. Let’s hack it in to see it work:

lists/tests/test_models.py.

    with self.assertRaises(ValidationError):
        item.save()
        item.full_clean()

That gets the test to pass:

OK

That taught us a little about Django validation, and the test is there to warn us if we ever forget our requirement and set blank=True on the text field (try it!).

Surfacing Model Validation Errors in the View

Let’s try and enforce our model validation in the views layer and bring it up through into our templates, so the user can see them. Here’s how we can optionally display an error in our HTML—we check whether the template has been passed an error variable, and if so, we display it next to the form:

lists/templates/base.html (ch10l013).

<form method="POST" action="{% block form_action %}{% endblock %}">
    <input name="item_text" id="id_new_item"
           class="form-control input-lg"
           placeholder="Enter a to-do item"
    />
    {% csrf_token %}
    {% if error %}
        <div class="form-group has-error">
            <span class="help-block">{{ error }}</span>
        </div>
    {% endif %}
</form>

Take a look at the Bootstrap docs for more info on form controls.

Passing this error to the template is the job of the view function. Let’s take a look at the unit tests in the NewListTest class. I’m going to use two slightly different error-handling patterns here.

In the first case, our URL and view for new lists will optionally render the same template as the home page, but with the addition of an error message. Here’s a unit test for that:

lists/tests/test_views.py (ch10l014).

class NewListTest(TestCase):
    [...]

    def test_validation_errors_are_sent_back_to_home_page_template(self):
        response = self.client.post('/lists/new', data={'item_text': ''})
        self.assertEqual(response.status_code, 200)
        self.assertTemplateUsed(response, 'home.html')
        expected_error = "You can't have an empty list item"
        self.assertContains(response, expected_error)

As we’re writing this test, we might get slightly offended by the /lists/new URL, which we’re manually entering as a string. We’ve got a lot of URLs hardcoded in our tests, in our views, and in our templates, which violates the DRY principle. I don’t mind a bit of duplication in tests, but we should definitely be on the lookout for hardcoded URLs in our views and templates, and make a note to refactor them out. But we won’t do them straight away, because right now our application is in a broken state. We want to get back to a working state first.

Back to our test, which is failing because the view is currently returning a 302 redirect, rather than a “normal” 200 response:

AssertionError: 302 != 200

Let’s try calling full_clean() in the view:

lists/views.py.

def new_list(request):
    list_ = List.objects.create()
    item = Item.objects.create(text=request.POST['item_text'], list=list_)
    item.full_clean()
    return redirect('/lists/%d/' % (list_.id,))

As we’re looking at the view code, we find a good candidate for a hardcoded URL to get rid of. Let’s add that to our scratchpad:

Now the model validation raises an exception, which comes up through our view:

[...]
  File "/workspace/superlists/lists/views.py", line 11, in new_list
    item.full_clean()
[...]
django.core.exceptions.ValidationError: {'text': ['This field cannot be
blank.']}

So we try our first approach: using a try/except to detect errors. Obeying the Testing Goat, we start with just the try/except and nothing else. The tests should tell us what to code next…

lists/views.py (ch10l015).

from django.core.exceptions import ValidationError
[...]

def new_list(request):
    list_ = List.objects.create()
    item = Item.objects.create(text=request.POST['item_text'], list=list_)
    try:
        item.full_clean()
    except ValidationError:
        pass
    return redirect('/lists/%d/' % (list_.id,))

That gets us back to the 302 != 200:

AssertionError: 302 != 200

Let’s return a rendered template then, which should take care of the template check as well:

lists/views.py (ch10l016).

    except ValidationError:
        return render(request, 'home.html')

And the tests now tell us to put the error message into the template:

AssertionError: False is not true : Couldn't find 'You can't have an empty list
item' in response

We do that by passing a new template variable in:

lists/views.py (ch10l017).

    except ValidationError:
        error = "You can't have an empty list item"
        return render(request, 'home.html', {"error": error})

Hmm, it looks like that didn’t quite work:

AssertionError: False is not true : Couldn't find 'You can't have an empty list
item' in response

A little print-based debug…

lists/tests/test_views.py.

expected_error = "You can't have an empty list item"
print(response.content.decode())
self.assertContains(response, expected_error)

…will show us the cause: Django has HTML-escaped the apostrophe:

[...]
<span class="help-block">You can&#39;t have an
empty list item</span>

We could hack something like this into our test:

    expected_error = "You can&#39;t have an empty list item"

But using Django’s helper function is probably a better idea:

lists/tests/test_views.py (ch10l019).

from django.utils.html import escape
[...]

        expected_error = escape("You can't have an empty list item")
        self.assertContains(response, expected_error)

That passes!

Ran 12 tests in 0.047s

OK

Checking Invalid Input Isn’t Saved to the Database

Before we go further though, did you notice a little logic error we’ve allowed to creep into our implementation? We’re currently creating an object, even if validation fails:

lists/views.py.

    item = Item.objects.create(text=request.POST['item_text'], list=list_)
    try:
        item.full_clean()
    except ValidationError:
        [...]

Let’s add a new unit test to make sure that empty list items don’t get saved:

lists/tests/test_views.py (ch10l020-1).

class NewListTest(TestCase):
    [...]

    def test_validation_errors_are_sent_back_to_home_page_template(self):
        [...]

    def test_invalid_list_items_arent_saved(self):
        self.client.post('/lists/new', data={'item_text': ''})
        self.assertEqual(List.objects.count(), 0)
        self.assertEqual(Item.objects.count(), 0)

That gives:

[...]
Traceback (most recent call last):
  File "/workspace/superlists/lists/tests/test_views.py", line 57, in
test_invalid_list_items_arent_saved
    self.assertEqual(List.objects.count(), 0)
AssertionError: 1 != 0

We fix it like this:

lists/views.py (ch10l020-2).

def new_list(request):
    list_ = List.objects.create()
    item = Item(text=request.POST['item_text'], list=list_)
    try:
        item.full_clean()
        item.save()
    except ValidationError:
        list_.delete()
        error = "You can't have an empty list item"
        return render(request, 'home.html', {"error": error})
    return redirect('/lists/%d/' % (list_.id,))

Do the FTs pass?

$ python3 manage.py test functional_tests.test_list_item_validation
[...]
  File "/workspace/superlists/functional_tests/test_list_item_validation.py",
line 26, in test_cannot_add_empty_list_items
[...]
selenium.common.exceptions.NoSuchElementException: Message: Unable to locate
element: {"method":"css selector","selector":".has-error"}

Not quite, but they did get a little further. Checking the line 26, we can see that we’ve got past the first part of the test, and are now onto the second check—that submitting a second empty item also shows an error.

We’ve got some working code though, so let’s have a commit:

$ git commit -am "Adjust new list view to do model validation"

Django Pattern: Processing POST Requests in the Same View as Renders the Form

This time we’ll use a slightly different approach, one that’s actually a very common pattern in Django, which is to use the same view to process POST requests as to render the form that they come from. Whilst this doesn’t fit the REST-ful URL model quite as well, it has the important advantage that the same URL can display a form, and display any errors encountered in processing the user’s input.

The current situation is that we have one view and URL for displaying a list, and one view and URL for processing additions to that list. We’re going to combine them into one. So, in list.html, our form will have a different target:

lists/templates/list.html (ch10l020).

{% block form_action %}/lists/{{ list.id }}/{% endblock %}

Incidentally, that’s another hardcoded URL. Let’s add it to our to-do list, and while we’re thinking about it, there’s one in home.html too:

This will immediately break our original functional test, because the view_list page doesn’t know how to process POST requests yet:

$ python3 manage.py test functional_tests
[...]
selenium.common.exceptions.NoSuchElementException: Message: Unable to locate
element: {"method":"css selector","selector":".has-error"}
[...]
AssertionError: '2: Use peacock feathers to make a fly' not found in ['1: Buy
peacock feathers']

Note

In this section we’re performing a refactor at the application level. We execute our application-level refactor by changing or adding unit tests, and then adjusting our code. We use the functional tests to tell us when our refactor is complete and things are back to working as before. Have another look at the diagram from the end of Chapter 4 if you need to get your bearings.

Refactor: Transferring the new_item Functionality into view_list

Let’s take all the old tests from NewItemTest, the ones that are about saving POST requests to existing lists, and move them into ListViewTest. As we do so, we also make them point at the base list URL, instead of …/add_item:

lists/tests/test_views.py (ch10l021).

class ListViewTest(TestCase):

    def test_uses_list_template(self):
        [...]

    def test_passes_correct_list_to_template(self):
        [...]

    def test_displays_only_items_for_that_list(self):
        [...]

    def test_can_save_a_POST_request_to_an_existing_list(self):
        other_list = List.objects.create()
        correct_list = List.objects.create()

        self.client.post(
            '/lists/%d/' % (correct_list.id,),
            data={'item_text': 'A new item for an existing list'}
        )

        self.assertEqual(Item.objects.count(), 1)
        new_item = Item.objects.first()
        self.assertEqual(new_item.text, 'A new item for an existing list')
        self.assertEqual(new_item.list, correct_list)


    def test_POST_redirects_to_list_view(self):
        other_list = List.objects.create()
        correct_list = List.objects.create()

        response = self.client.post(
            '/lists/%d/' % (correct_list.id,),
            data={'item_text': 'A new item for an existing list'}
        )
        self.assertRedirects(response, '/lists/%d/' % (correct_list.id,))

Note that the NewItemTest class disappears completely. I’ve also changed the name of the redirect test to make it explicit that it only applies to POST requests.

That gives:

FAIL: test_POST_redirects_to_list_view (lists.tests.test_views.ListViewTest)
AssertionError: 200 != 302 : Response didn't redirect as expected: Response
code was 200 (expected 302)
[...]
FAIL: test_can_save_a_POST_request_to_an_existing_list
(lists.tests.test_views.ListViewTest)
AssertionError: 0 != 1

We change the view_list function to handle two types of request:

lists/views.py (ch10l022-1).

def view_list(request, list_id):
    list_ = List.objects.get(id=list_id)
    if request.method == 'POST':
        Item.objects.create(text=request.POST['item_text'], list=list_)
        return redirect('/lists/%d/' % (list_.id,))
    return render(request, 'list.html', {'list': list_})

That gets us passing tests:

Ran 13 tests in 0.047s

OK

Now we can delete the add_item view, since it’s no longer needed…oops, a couple of unexpected failures:

[...]
AttributeError: 'module' object has no attribute 'add_item'
[...]
FAILED (errors=10)

It’s because we’ve deleted the view, but it’s still being referred to in urls.py. We remove it from there:

lists/urls.py (ch10l023).

urlpatterns = [
    url(r'^new$', views.new_list, name='new_list'),
    url(r'^(\d+)/$', views.view_list, name='view_list'),
]

And that gets us to the OK. Let’s try a full FT run:

$ python3 manage.py test
[...]

Ran 16 tests in 15.276s

FAILED (errors=1)

We’re back to the one failure in our new functional test. Our refactor of the add_item functionality is complete. We should commit there:

$ git commit -am "Refactor list view to handle new item POSTs"

Note

So did I break the rule about never refactoring against failing tests? In this case, it’s allowed, because the refactor is required to get our new functionality to work. You should definitely never refactor against failing unit tests. But in my book it’s OK for the FT for the current story you’re working on to be failing. If you prefer a clean test run, you could add a skip to the current FT.

Enforcing Model Validation in view_list

We still want the addition of items to existing lists to be subject to our model validation rules. Let’s write a new unit test for that; it’s very similar to the one for the home page, with just a couple of tweaks:

lists/tests/test_views.py (ch10l024).

class ListViewTest(TestCase):
    [...]

    def test_validation_errors_end_up_on_lists_page(self):
        list_ = List.objects.create()
        response = self.client.post(
            '/lists/%d/' % (list_.id,),
            data={'item_text': ''}
        )
        self.assertEqual(response.status_code, 200)
        self.assertTemplateUsed(response, 'list.html')
        expected_error = escape("You can't have an empty list item")
        self.assertContains(response, expected_error)

That should fail, because our view currently does not do any validation, and just redirects for all POSTs:

    self.assertEqual(response.status_code, 200)
AssertionError: 302 != 200

Here’s an implementation:

lists/views.py (ch10l025).

def view_list(request, list_id):
    list_ = List.objects.get(id=list_id)
    error = None

    if request.method == 'POST':
        try:
            item = Item(text=request.POST['item_text'], list=list_)
            item.full_clean()
            item.save()
            return redirect('/lists/%d/' % (list_.id,))
        except ValidationError:
            error = "You can't have an empty list item"

    return render(request, 'list.html', {'list': list_, 'error': error})

It’s not deeply satisfying is it? There’s definitely some duplication of code here, that try/except occurs twice in views.py, and in general things are feeling clunky.

Ran 14 tests in 0.047s

OK

Let’s wait a bit before we do more refactoring though, because we know we’re about to do some slightly different validation coding for duplicate items. We’ll just add it to our scratchpad for now:

Note

One of the reasons that the “three strikes and refactor” rule exists is that, if you wait until you have three use cases, each might be slightly different, and it gives you a better view for what the common functionality is. If you refactor too early, you may find that the third use case doesn’t quite fit with your refactored code…

At least our functional tests are back to passing:

$ python3 manage.py test functional_tests
[...]
OK

We’re back to a working state, so we can take a look at some of the items on our scratchpad. This would be a good time for a commit. And possibly a tea break.

$ git commit -am "enforce model validation in list view"

Refactor: Removing Hardcoded URLs

Do you remember those name= parameters in urls.py? We just copied them across from the default example Django gave us, and I’ve been giving them some reasonably descriptive names. Now we find out what they’re for.

lists/urls.py.

    url(r'^new$', views.new_list, name='new_list'),
    url(r'^(\d+)/$', views.view_list, name='view_list'),

The {% url %} Template Tag

We can replace the hardcoded URL in home.html with a Django template tag which refers to the URL’s “name”:

lists/templates/home.html (ch10l026-1).

{% block form_action %}{% url 'new_list' %}{% endblock %}

We check that doesn’t break the unit tests:

$ python3 manage.py test lists
OK

Let’s do the other template. This one is more interesting, because we pass it a parameter:

lists/templates/list.html (ch10l026-2).

{% block form_action %}{% url 'view_list' list.id %}{% endblock %}

Check out the Django docs on reverse URL resolution for more info.

We run the tests again, and check they all pass:

$ python3 manage.py test lists
OK
$ python3 manage.py test functional_tests
OK

Excellent:

$ git commit -am "Refactor hard-coded URLs out of templates"

Using get_absolute_url for Redirects

Now let’s tackle views.py. One way of doing it is just like in the template, passing in the name of the URL and a positional argument:

lists/views.py (ch10l026-3).

def new_list(request):
    [...]
    return redirect('view_list', list_.id)

That would get the unit and functional tests passing, but the redirect function can do even better magic than that! In Django, because model objects are often associated with a particular URL, you can define a special function called get_absolute_url which says what page displays the item. It’s useful in this case, but it’s also useful in the Django admin (which I don’t cover in the book, but you’ll soon discover for yourself): it will let you jump from looking at an object in the admin view to looking at the object on the live site. I’d always recommend defining a get_absolute_url for a model whenever there is one that makes sense; it takes no time at all.

All it takes is a super-simple unit test in test_models.py:

lists/tests/test_models.py (ch10l026-4).

    def test_get_absolute_url(self):
        list_ = List.objects.create()
        self.assertEqual(list_.get_absolute_url(), '/lists/%d/' % (list_.id,))

Which gives:

AttributeError: 'List' object has no attribute 'get_absolute_url'

And the implementation is to use Django’s reverse function, which essentially does the reverse of what Django normally does with urls.py (see docs):

lists/models.py (ch10l026-5).

from django.core.urlresolvers import reverse


class List(models.Model):

    def get_absolute_url(self):
        return reverse('view_list', args=[self.id])

And now we can use it in the view—the redirect function just takes the object we want to redirect to, and it uses get_absolute_url under the hood automagically!

lists/views.py (ch10l026-6).

def new_list(request):
    [...]
    return redirect(list_)

There’s more info in the Django docs. Quick check that the unit tests still pass:

OK

Then we do the same to view_list:

lists/views.py (ch10l026-7).

def view_list(request, list_id):
    [...]

            item.save()
            return redirect(list_)
        except ValidationError:
            error = "You can't have an empty list item"

And a full unit test and functional test run to assure ourselves that everything still works:

$ python3 manage.py test lists
OK
$ python3 manage.py test functional_tests
OK

Cross off our to-dos:

Let’s do a commit:

$ git commit -am "Use get_absolute_url on List model to DRY urls in views"

That final to-do item will be the subject of the next chapter…

Use a tests folder

Just as you use multiple files to hold your application code, you should split your tests out into multiple files.

Use a folder called tests, with a __init__.py.
For functional tests, group them into tests for a particular feature or user story.
For unit tests, you want a separate test file for each tested source code file. For Django, that’s typically test_models.py, test_views.py, and test_forms.py.
Have at least a placeholder test for every function and class.

Don’t forget the “Refactor” in “Red, Green, Refactor”

The whole point of having tests is to allow you to refactor your code! Use them, and make your code as clean as you can.

Don’t refactor against failing tests

In general!
But the FT you’re currently working on doesn’t count.
You can occasionally put a skip on a test which is testing something you haven’t written yet.
More commonly, make a note of the refactor you want to do, finish what you’re working on, and do the refactor a little later, when you’re back to a working state.
Don’t forget to remove any skips before you commit your code! You should always review your diffs line by line to catch things like this.

^[16]“Dunder” is shorthand for double-underscore, so “dunderinit” means __init__.py.

Get Test-Driven Development with Python now with the O’Reilly learning platform.

O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.

Start your free trial

Test-Driven Development with Python by Harry J.W. Percival

Chapter 10. Input Validation and Test Organisation

Validation FT: Preventing Blank Items

Skipping a Test

Warning

Splitting Functional Tests out into Many Files

Note

Running a Single Test File

Fleshing Out the FT

Using Model-Layer Validation

Refactoring Unit Tests into Several Files

Note

Unit Testing Model Validation and the self.assertRaises Context Manager

Tip

A Django Quirk: Model Save Doesn’t Run Validation

Surfacing Model Validation Errors in the View

Checking Invalid Input Isn’t Saved to the Database

Django Pattern: Processing POST Requests in the Same View as Renders the Form

Note

Refactor: Transferring the new_item Functionality into view_list

Note

Enforcing Model Validation in view_list

Note

Refactor: Removing Hardcoded URLs

The {% url %} Template Tag

Using get_absolute_url for Redirects

Don’t leave empty-handed

It’s yours, free.

Check it out now on O’Reilly