Django Expense Manager 3 - Fixing Our Factories and Adding REST API Authentication
Changing (read Fixing) Our Tests
Today we’ll add an API for authentication. How will we do that? We will use the dj-rest-auth package. The documentation is available here and the source code here. But before we do that, let’s have a look at our User model, the django-allauth package we are using and our factoryboy UserFactory. Why? Well, we had the following in our UserFactory:
class UserFactory(factory.django.DjangoModelFactory):
class Meta:
model = User
username = factory.Sequence(lambda n: f"user_{n}")
email = factory.Faker("email")
password = factory.PostGenerationMethodCall("set_password", f"{email}_password")
is_active = True
is_staff = False
which yes, are the fields that the default User model has. But what is the problem with using this in our tests for now? Well, we’ve only been testing manually creating instances of our various models. When it comes to calling a view function (in our case some API endpoint), we will have to be logged in and our account email would have to be confirmed. Nowhere in our UserFactory do we do that. In fact, when we register for an account on our website now using django-allauth and the settings we have configured in this post, which were:
ACCOUNT_AUTHENTICATION_METHOD = "email"
ACCOUNT_EMAIL_REQUIRED = True
ACCOUNT_USERNAME_REQUIRED = False
ACCOUNT_EMAIL_VERIFICATION = "mandatory"
EMAIL_BACKEND = 'django.core.mail.backends.console.EmailBackend'
then a few more things happen when the User registers. The easiest way to see it is to have a look at one fo the the django-allauth tests from here. Please note I’ve extracted bits and pieces from a few different tests and put them together.
...
user = get_user_model().objects.create(
username="john", email="john@example.org", is_active=True
)
user.set_password("doe")
user.save()
email = EmailAddress.objects.create(
user=user, email="a@b.com", verified=False, primary=True
)
confirmation = EmailConfirmationHMAC(email)
confirmation.send()
self.assertEqual(len(mail.outbox), 1)
self.client.post(reverse("account_confirm_email", args=[confirmation.key]))
email = EmailAddress.objects.get(pk=email.pk)
self.assertTrue(email.verified)
...
The test gives as a pretty good indication of the flow. A User instance is created and saved. Then, an EmailAddress object is created with a default verified value of False. A confirmation email is then sent and once the email is confirmed, the verified attribute of the associated EmailAddress object is set to True. Why am I talking about this? Well, when I was testing the authentication endpoint of the app (we’ll get there soon), I was getting a error 400 because the email was not yet confirmed. Once it got confirmed, it would login correctly. Ok, so should we change our UserFactory? Well, let’s create all of the User instances with verified EmailAddress objects. First of all, let’s change our fixture.
In the conftest.py file, let’s add a few things:
...
from allauth.account.models import EmailAddress
...
@pytest.fixture(scope="session")
def django_db_setup(django_db_setup, django_db_blocker):
with django_db_blocker.unblock():
user_1: User = User.objects.create(
username="hi", email="hi@there.com", password="themostsecurepassword", is_active=True
)
EmailAddress.objects.create(email=user_1.email, verified=True, primary=True, user=user_1)
retailer_1 = Retailer.objects.create(name="AmazIn", online=True, user=user_1)
category_1 = Category.objects.create(
name="Groceries", product_type="P", user=user_1
)
transaction_1 = Transaction.objects.create(
amount=Money(420.00, "GBP"),
name="Yoga Mat for Meditation",
retailer=retailer_1,
category=category_1,
date="2021-04-20",
transaction_type="E",
recurring=False,
user=user_1,
)
After the user_1 User instance is created, we also create a confirmed EmailAddress for it. We do something similar in our factories.py file for the UserFactory:
...
from allauth.account.models import EmailAddress
...
class UserFactory(factory.django.DjangoModelFactory):
class Meta:
model = User
username = factory.Sequence(lambda n: f"user_{n}")
email = factory.Faker("email")
password = factory.PostGenerationMethodCall("set_password", f"{email}_password")
is_active = True
is_staff = False
@factory.post_generation
def verify(obj, create, extracted, **kwargs):
if not create:
return
EmailAddress.objects.create(email=obj.email, verified=True, primary=True, user=obj)
I used the @factory.post_generation decorator which calls the verify function after the User instance is created. The if not create: return is there in case the model is created using the ‘build’ strategy. We are using the ‘create’ strategy by default so we don’t have to call .save() on our objects. All our tests should still pass, they don’t rely on anything we have changed now.
What was this post about? Authentication
I wouldn’t blame you if you strongly disliked me for that detour but it important. Ok so back to dj-rest-auth. First install it:
pip install dj-rest-auth
Then add it to our project settings in expenses_backend/settings.py file:
...
INSTALLED_APPS = [
...
'rest_framework',
'rest_framework.authtoken',
'dj_rest_auth',
'dj_rest_auth.registration',
...
]
Then I actuall created a new app called users. Why? I decided I was gonna add the login/register API endpoints in there. So let’s quickly create:
python manage.py startapp users
Add it to our settings INSTALLED_APPS:
...
INSTALLED_APPS = [
...
'users',
...
]
...
Are we done installing stuff? Almost. I am actually going to use JWT token instead of Django’s Token-based authentication. So we need to install djangorestframework-simplejwt (rolls right off your tongue doesn’t it?). Then we need to change some settings…again…:
...
REST_FRAMEWORK = {
'DEFAULT_AUTHENTICATION_CLASSES': [
'dj_rest_auth.jwt_auth.JWTCookieAuthentication',
],
}
REST_USE_JWT = True
JWT_AUTH_COOKIE = 'expenses-app-auth
...
Oh and since I am here, I actually upgraded the Django installation to 3.2. You might get a warning/error message for models created in previous versions (like 3.1), so I added
DEFAULT_AUTO_FIELD='django.db.models.AutoField'
to the configuration file as well. You can read more about it here and here but the idea is that before, all models that were created (unless we specified a primary_key=True on one of the fields) had an id field that held that primary_key. Now we either do it ourselves or specify that setting.
DJ-REST-auth eEndpoints
Ok back to dj-rest-auth. So we installed it. Now what? Let’s add the API endpoints to our users/urls.py file. (which you have to create yourself). Yes some people prefer to have API endpoints in a different location. I don’t mind too much for this project so I am keeping it simple. In the users/urls.py file we now have:
from django.urls import path, include
urlpatterns = [
path('', include('dj_rest_auth.urls')),
path('register/', include('dj_rest_auth.registration.urls')),
]
And in our expenses_backend/urls.py file we have:
from django.contrib import admin
from django.urls import path, include
urlpatterns = [
path('mysupersecretnotadmin/', admin.site.urls),
# path('accounts/', include('allauth.urls')), <- can delete this
path('users/', include('users.urls')),
]
Actually, delete the accounts one. We will only use the API endpoints.
What what are we include()ing in there in the users/urls.py file? Well, there are two of them. The first one contains the ones relating to login, logout, password change and the second one the ones related to registration. You can see the full list for ‘dj_rest_auth.urls’ and ‘dj_rest_auth.registration.urls’.
Testing the Register/Login Endpoints
And just like that, we have authentication/registration via API endpoints. What is that you’re saying? YOU WANT TESTS? ok let’s write two. First of all, dj-rest-auth depends on Django REST Framework so some of the information that follows is related to that. We are going to use the APIClient in order to run those tests. We will have that accessible as a fixture so in our tests/conftest.py file:
...
from rest_framework.test import APIClient, APIRequestFactory
...
@pytest.fixture
def api_client():
return APIClient()
@pytest.fixture
def api_reqfactory():
return APIRequestFactory()
Ok I know I said APIClient, but I put APIRequestFactoy in there too. We won’t be using that today though. Then, let’s create an api folder in the tests directory and in there, a file called test_auth.py.
What’s inside
from typing import Dict
import pytest
from django.urls import reverse
from django.core import mail
from django.contrib.auth import get_user_model
from rest_framework import status
from rest_framework.response import Response
User = get_user_model()
@pytest.mark.django_db
def test_registration_verification_and_authentication(api_client):
current_user_count: int = User.objects.all().count()
current_mail_count: int = len(mail.outbox)
email: str = 'suchgoodemail@aws.moon'
password: str = 'amazingpassword'
registration_data: Dict[str, str] = {
'email': email,
'password1': password,
'password2': password
}
register_url: str = reverse('rest_register')
response: Response = api_client.post(
register_url,
data=registration_data
)
# check that the user was created successfully
assert response.status_code == status.HTTP_201_CREATED
# check that the user count increased by one
assert User.objects.all().count() == (current_user_count + 1)
assert len(mail.outbox) == (current_mail_count + 1)
assert 'key' not in response.data
# if we try to log in without verifying should get HTTP 400
login_url: str = reverse('rest_login')
login_response: Response = api_client.post(
login_url,
data={
'email': email,
'password': password,
}
)
assert login_response.status_code == status.HTTP_400_BAD_REQUEST
# let's verify the user
# TODO: rewrite the part below. is there another way to get the key?
verify_url: str = reverse('rest_verify_email')
key: str = mail.outbox[-1].body.split('/')[-2]
verify_response: Response = api_client.post(
verify_url,
data={"key": key}
)
assert verify_response.status_code == status.HTTP_200_OK
# now let's try logging in again
login_response: Response = api_client.post(
login_url,
data={
'email': email,
'password': password,
}
)
assert login_response.status_code == status.HTTP_200_OK
# check if we got an access token
assert 'access_token' in login_response.data
@pytest.mark.django_db
def test_authentication_with_verified_email(api_client, user_factory):
email = "bad@email.com"
password = "amazingsecurity"
_ = user_factory(email=email, password=password)
login_url: str = reverse('rest_login')
login_response: Response = api_client.post(
login_url,
data={
'email': email,
'password': password,
}
)
assert login_response.status_code == status.HTTP_200_OK
Woah, that’s quite a bit. Let’s actually start with the second test. Note the api_client fixture in the function’s arguments. Initially the test was called test_authentication. I created an user user_facory with an email and password. Got the login_url that I will POST the data to. The data includes the email and password and we catch the response. Then we assert that the status_code is HTTP_200_OK. All good. So why did I start with this email? Well because initially I was sending in user information of user instances whose emails were not verifid. Consequently, I kept getting 400 errors. So I took it step by step until I figured out how to verify the emails when creating the user instances (that’s what we started with today, remember?)
Ok so I did the whole process manually in the first test. Let’s go a few lines at a time.
current_user_count: int = User.objects.all().count()
current_mail_count: int = len(mail.outbox)
Since we are gonna be creating a new User, let’s check how many we have now. After we create one, the count should increase by 1. What am I doing with that email?? Well, when the tests run, the outgoing emails are saved in django.core.mail.outbox. I can then access the emails from there.
email: str = 'suchgoodemail@aws.moon'
password: str = 'amazingpassword'
registration_data: Dict[str, str] = {
'email': email,
'password1': password,
'password2': password
}
We then create the data that we will POST to our register API endpoint.
register_url: str = reverse('rest_register')
response: Response = api_client.post(
register_url,
data=registration_data
)
We then get the register_url and use the api_client fixture to POST our data to it. Now, let’s make some assertions.
# check that the user was created successfully
assert response.status_code == status.HTTP_201_CREATED
# check that the user count increased by one
assert User.objects.all().count() == (current_user_count + 1)
assert len(mail.outbox) == (current_mail_count + 1)
assert 'key' not in response.data
We first check the status_code of our response. HTTP_201_CREATED means the User object was created. Then we check that the User.objects.count did in fact increase by 1 and same for the mail.outbox. We also check that ‘key’ is not in response.data. What is this key? Well, it’s the ‘key’ that we need to verify our email. It’s not returned in the registration process, instead it’s sent by email.
Now let’s try to log in and fail.
# if we try to log in without verifying should get HTTP 400
login_url: str = reverse('rest_login')
login_response: Response = api_client.post(
login_url,
data={
'email': email,
'password': password,
}
)
assert login_response.status_code == status.HTTP_400_BAD_REQUEST
Our account is not verified. This will fail. Let’s verify the user.
# let's verify the user
# TODO: rewrite the part below. is there another way to get the key?
verify_url: str = reverse('rest_verify_email')
key: str = mail.outbox[-1].body.split('/')[-2]
verify_response: Response = api_client.post(
verify_url,
data={"key": key}
)
assert verify_response.status_code == status.HTTP_200_OK
I extract the key from the latest email in the mail outbox and send that to the rest_verify_email endpoint. If all goes well, we get HTTP_200_OK and our email is now verified. Let’s try logging in again.
# now let's try logging in again
login_response: Response = api_client.post(
login_url,
data={
'email': email,
'password': password,
}
)
assert login_response.status_code == status.HTTP_200_OK
# check if we got an access token
assert 'access_token' in login_response.data
We do the same thing we did before. But now it works. We also get an authentication access_token that we then use for protected endpoints.
So why did we do all of this? Surely this process is tested by dj-rest-auth package maintainers. You’re absolutely right it is. But because I kept forgetting about the email confirmation, I rewrote it to try and figure out what I was forgetting about. Even though it’s an unnecessary test, I think it’s a good learning exercis and I will leave it in there for a future end-to-end test.
And that’s it for today! In the next post (I PROMISE), I will first use GitHub Actions to run these tests. I am currently rewriting some of the workflow files for both this project, the React project, the Vue project and the FastAPI project. They’re all pretty similar with such subtle differences.
The commit for this post is here.