Enforcing Style in a Python Project

GitPythonlintpre-commit / 5 min read

A linter and a styler can help you to write cleaner and more consistent code. In this post we’ll look at how to set up both for a Python project.

What are Pre-Commit Hooks?

Git has the ability to execute specific actions when certain events occur. The connections between the actions and the events are known as hooks. These hooks are configured via files in the .git/hooks folder.

Git hooks provide the perfect mechanism for insuring that the code committed to a repository is clean and consistent. We’ll be setting up pre-commit hooks that will run actions immediately before each commit to the Git repository. A commit will only succeed if all of the associated actions are successful.

The Pre-Commit Framework

Despite the relative simplicity of their implementation, Git hooks can be somewhat fiddly. The pre-commit framework makes it easier to manage and maintain pre-commit hooks. It eliminates a lot of the fiddliness.

The way that the pre-commit framework replaces a collection of distinct hooks with a single hook (the pre-commit hook) and a configuration file. At commit time the pre-commit hook is triggered and it runs all of the actions specified in the configuration file.

Install

Installing the pre-commit framework is simple. You’ll probably want to do this in a virtual environment.

pip install pre-commit

At this point you should also add pre-commit to your project requirements.txt.

Now add pre-commit as a hook.

pre-commit install

This will create a hook file at .git/hooks/pre-commit.

💡 Add pre-commit to your requirements.txt.

Configure

The actions run by pre-commit are configured via the .pre-commit-config.yaml file. Run the following to generate a simple default configuration.

pre-commit sample-config >.pre-commit-config.yaml

The contents of the configuration file should look something like this:

repos:
-   repo: https://github.com/pre-commit/pre-commit-hooks
    rev: v3.2.0
    hooks:
    -   id: trailing-whitespace
    -   id: end-of-file-fixer
    -   id: check-yaml
    -   id: check-added-large-files

This configuration will run four distinct actions (trailing-whitespace, end-of-file-fixer, check-yaml and check-added-large-files) against each file in the repository.

💡 The .pre-commit-config.yaml file should be staged and committed to the repository so that everybody who works on the code applies the same rules.

Test

You can test the configuration by manually running the hooks against all files.

pre-commit run --all-files

If you then run git status you’ll likely find that one or more of the files in your repository has been modified. In my case this generally involves adding empty lines at the end of various files. Take a look at the changes and if you are happy, stage and commit the changes.

Lint: Flake 8

The Flake8 linter is used to check for syntactic problems in Python code. To enable Flake8 add the following to the .pre-commit-config.yaml file.

-   repo: https://github.com/pycqa/flake8
    rev: 5.0.4
    hooks:
      - id: flake8

You might want to check the Flake8 repository to see if there are more recent releases and update the rev field accordingly.

Configuration

You can tweak some Flake8 options by creating a .flake8 file. Its contents might look something like this:

[flake8]
max-line-length = 120
ignore = F401, F403, F405, W503
exclude =
    database/__init__.py

A complete list of available options can be found here.

Ignoring Code

You can tell Flake8 to ignore a specific line of code by adding a noqa hint as a comment at the end of the line.

from .database import * # noqa

You can be more specific by telling Flake8 which errors it should ignore.

from .database import * # noqa: F403
from .database import * # noqa: F401, F403

Lint: Ruff

Another option for linting is Ruff. Add the following to the .pre-commit-config.yaml file.

-   repo: https://github.com/astral-sh/ruff-pre-commit
    rev: v0.4.1
    hooks:
      - id: ruff
        args: ["--fix", "--verbose"]
      - id: ruff-format

Configuration

You can tweak some Ruff options by creating a .ruff.toml file. Its contents might look something like this:

lint.select = ["E", "F"]
lint.ignore = ["F405", "F403"]
exclude = [
    "build",
    "dist",
    ".venv",
]
line-length = 120

[lint.per-file-ignores]
"tools/ignore.py" = ["F401"]

For more information on configurations, see the documentation. You can find a full list of rules here.

Style

The Black code formatter will enforce a consistent formatting style on Python code. Add the following to the .pre-commit-config.yaml file.

-   repo: https://github.com/psf/black
    rev: 22.10.0
    hooks:
      - id: black
        args: ["--line-length", "120"]

You might want to check the Black repository to see if there are more recent releases and update the rev field accordingly.

Sorting Imports

The isort can be used to tidy up your imports. It also does a bunch of other useful things. There’s extensive and detailed documentation.

pip3 install isort

Create a configuration file, .isort.cfg. Find out more about available options here.

[settings]
profile = black
skip = .gitignore
line_length = 120
# Sections
sections = FUTURE,STDLIB,THIRDPARTY,FIRSTPARTY,LOCALFOLDER
# Custom section title
import_heading_stdlib = Standard Library

Make sure that the specified line length is consistent with what you have configured for Flake8! There’s also a guide for ensuring that it plays nicely with Black.

Add the following to the .pre-commit-config.yaml file.

-   repo: https://github.com/pycqa/isort
    rev: 5.12.0
    hooks:
      - id: isort
        name: isort (python)

Not Running Pre-Commit Hooks

If for some reason you want to commit changes without running the pre-commit hooks then you can simply provide the --no-verify option to git commit.

CI/CD

GitHub Actions

Add the following to your workflow:

jobs:
  lint:
    runs-on: ubuntu-latest
    container: python:3.10.11

    steps:
      - name: Install pipx
        run: pip install pipx
      - uses: actions/checkout@v3
      - name: flake8
        uses: py-actions/flake8@v2
      - name: ruff
        uses: chartboost/ruff-action@v1
      - name: blake
        uses: psf/black@stable
        with:
          options: "--check --verbose --line-length 120"
          version: "22.10.0"

Prosper

With this setup in place your code will be checked every time you commit. Many issues will be automatically fixed. Others will be highlighted and you’ll have to manually intervene.

This works particularly well if you are part of a team because it means that everybody on the team will be committing and pushing code without any syntactic issues and with consistent formatting.

⚠️ You might not want to include all of these options on your project. There’s redundancy between what’s done by Flake 8, Ruff and Black. If you include all of them then you will likely find that they make conflicting changes to the code.

If you are interested in applying security checks to your Python project using pre-commit hooks then take a look at this.