Making use of pygments' filters with Pelican
Fri 01 September 2023 — download

I've been using Pelican more or less since the beginning of this blog and I'm still pretty happy about it. Mostly because of how boring it is, and its complete absence of fundamental changes thorough the years.

Anyway, I was looking at how to reduce the size of the pages of my blog and looked at how code is syntactically highlighted: Pelican is using Pygments to do this, and looking at its documentation, the TokenMergeFilter should help a bit, by merging token of the same type together, instead of highlighting them separately.

Pelican's documentation says that options can be passed to python-markdown like this: MARKDOWN = { 'extension_configs': { 'markdown.extensions.codehilite': {'css_class': 'highlight'} } }.

Looking at python-markdown's one, one can pass various things as parameters, but it doesn't mention filters. Pygments documentation on this topic implies that the only way to add filters is to use the add_filter method on a lexer.

But looking at the code as suggested here, filters can be passed like any other options, meaning that one only needs to add the following code into the pelicanconf.py file to used the TokenMergeFilter:

from pelican import TokenMergeFilter

MARKDOWN = {
    'extension_configs': {
        'markdown.extensions.codehilite': {
            'filters': [TokenMergeFilter()]
        }
    }
}`.

Totally worth the effort for a marginal page size reduction!