CNLearn FastAPI - Adding Mypy
Today will be a shorter post. We are going to add mypy to the project and meanwhile refactor certain things. We won’t look at all the types we added, not essential at the moment, just the things that were refactored/changed as well as what was added to the CI pipeline. We are also moving a bit away from the cookiecutter template here, specifically having imports in the init.py files. I’ve never been a fan of that. I prefer more explicit imports.
Installing Mypy and addtional types packages
So, first of all, we need to install mypy, but also a few different packages that help with types:
pip install mypy
pip install mypy-extensions
pip install sqlalchemy2-stubs
pip install types-passlib
pip install types-python-jose
For now, these will allow us to fully type the project including the tests. I also added a mypy config section to the pyproject.toml file. Have I mentioned we are upgrading to 3.10? :)
[tool.mypy]
python_version = "3.10"
files = "app/**/*.py, tests/**/*.py"
plugins = "sqlalchemy.ext.mypy.plugin"
exclude = ['.venv/']
strict = true
We then run mypy in the project directory, and……so many errors. Ok so I fixed them one by one. Here are some of the most important changes:
Adding an ID column to the Declarative Base Class
Our CRUDBase had a method such as:
async def get(self, db: AsyncSession, id: Any) -> Optional[ModelType]:
results = await db.execute(select(self.model).where(self.model.id == id))
return results.scalars().first()
What’s the problem with that? Well, so far we only had one SQLAlchemy model, the User model which did have an ID. Since the upper bound of the ModelType is Base, that does not have it. As such, mypy complains. So how about we move the ID column to a Base class that all other models will inherit from?
from sqlalchemy import Column, Integer
from sqlalchemy.ext.declarative import declared_attr
from sqlalchemy.orm import declarative_base
class BaseWithID:
@declared_attr
def __tablename__(cls):
return cls.__name__.lower()
id: int = Column(Integer, primary_key=True, index=True)
# declarative base class
Base = declarative_base(cls=BaseWithID)
I have also removed that id column from the User model. Now, any model that inherits from Base, will have this id column on it. Mypy is now happier.
Removing imports from init files
This was actually one of the things I never liked, I just didn’t get to change it yet. I am not a fan of:
from app import schemas
where in the schemas folder you have an __init__.py file with imports in it
from .token import Token, TokenPayload # noqa: F401
from .user import User, UserCreate, UserUpdate # noqa: F401
I don’t like it, Flake8 doesn’t either. I have changed it now. Now, rather than using schemas.UserCreate, we have to import from app.schemas.user import UserCreate and use it as UserCreate. I did the same thing for crud files and did some renaming when importing to avoid any clashes.
Changing the testing environment slightly
Ok so I realised the environment variable wasn’t read in time and so by default, the testing suite ran the migrations on the main db…oops. I have changed it now, but you need to make sure that if you run the tests with pytest, you must have TESTING=1 in your environment.
TESTING=1 pytest
is enough to make it work.
Aaaand that’s it for this short post. There will be one more CI one, in which we will add coverage to our test suite, and we’ll go back to what I said I’d do: adding the models for Word/Character.
The commit for this post is here