Python, mkdocs and mkdocstrings

When creating a library with some rather uncommon features, it is vital to ensure its users will be able to correctly use it by providing a useful documentation. Turns out this is possible even for Python projects.

One of the recent projects we are working on right now is a stream processing library for Crypt4GH encrypted containers. There is a lot of work to be done before it is actually useful for some real-world usage, however even during these early stages of development we found out we need both prose-like documentation and some generated API reference as well.

For creating both types of documentation, the MkDocs documentation tool can be used. Although it primarily deals with converting Markdown project documentation to a static HTML website, it can also be extended with plugins. One such plugin is mkdocstring with its mkdocstrings-python handler for Python Docstrings which can generate documentation from annotated Python source files.

Configuring this basic infrastructure in pyproject.toml file requires only listing required packages as optional development dependencies. In this following configuration snippet the mkdocs-material package is listed too as the material theme is commonly used and the users will find the resulting documentation site familiar.

[project.optional-dependencies]
dev = [
    "mkdocstrings[python]>=0.18",
    "mkdocs-material",
]

The mkdocs package requires its own configuration which configures at least the site name, but for our use-case it more importantly configures the mkdocstrings plugin. The skeleton of our configuration in mkdocs.yml file becomes:

site_name: Crypt4GH Processing
plugins:
  - mkdocstrings:
      handlers:
        python:
          paths:
            - oarepo_c4gh
          options:
            allow_inspection: false
            show_source: true
            filters:
              - "!^__"
              - "__init__"
              - "__bytes__"
            show_signature_annotations: true
            show_root_heading: true
            ignore_init_summary: false

As we can see, mkdocstrings plugin allows for quite some flexibility with respect to filtering collected identifiers, showing type hints (as signature annotations) and even directly showing appropriate source file section which the generated documentation is refering to.

Now that the generated content is configured, the visual style should be configured as well. However with the mkdocs-material package installed, it is just a matter of setting the theme name to a particular value (again, in mkdocs.yml):

theme:
  name: "material"

The only thing missing is the actual list of documentation pages. In the same configuration file a simple nav key with a list of "title" : file.md pairs has to be written:

nav:
  - "Public API": index.md
  - "Exceptions": exceptions.md

In index.md we have decided to write a prose-like guidelines on how to use the library without delving into implementation details. Then in the exceptions.md a generated list of exceptions with more information should be produced. Using mkdocstring syntax allows the author to combine prose with extracted documentation starting at given module and/or specific identifier. The exceptions.md file may look like this:

Exceptions
==========

All the exceptions that might be raised by the libary come from the
`exceptions` module.

::: oarepo_c4gh.exceptions

If you are interested in the documentation of a development version of this library, you can see the result in the official published documentation at Crypt4GH Processing.

To our pleasant surprise, it is apparently possible to semi-automatically maintain decent documentation and API reference even in Python. See ya next time for more surprises!