At Nos oignons, we're running Tor exit nodes, but also a handful of services, like our website, some git repositories, some lightweight monitoring, … but also internal and external mailing lists. For the latter, we're using mailman3/hyperkitty with schleuder on top of it, making it an unholy pile of python, ruby, gpg and email shenanigans that the sysadmin team has to prevent from falling over.
Unfortunately, teaching sand to think was a mistake, and anything running
on a computer will invariably fail at some point, mailing lists are no
exceptions. Today I got the following email on the schleuder-admins@ list:
Net::ReadTimeout with #<TCPSocket:(closed)>
/usr/lib/ruby/3.3.0/net/protocol.rb:229:in `rbuf_fill'
/usr/lib/ruby/3.3.0/net/protocol.rb:199:in `readuntil'
/usr/lib/ruby/3.3.0/net/protocol.rb:209:in `readline'
/usr/lib/ruby/gems/3.3.0/gems/net-smtp-0.5.1/lib/net/smtp.rb:1017:in `recv_response'
/usr/lib/ruby/gems/3.3.0/gems/net-smtp-0.5.1/lib/net/smtp.rb:979:in `block in data'
/usr/lib/ruby/gems/3.3.0/gems/net-smtp-0.5.1/lib/net/smtp.rb:1027:in `critical'
/usr/lib/ruby/gems/3.3.0/gems/net-smtp-0.5.1/lib/net/smtp.rb:965:in `data'
/usr/lib/ruby/gems/3.3.0/gems/net-smtp-0.5.1/lib/net/smtp.rb:799:in `block in send_message'
/usr/lib/ruby/gems/3.3.0/gems/net-smtp-0.5.1/lib/net/smtp.rb:926:in `rcptto_list'
/usr/lib/ruby/gems/3.3.0/gems/net-smtp-0.5.1/lib/net/smtp.rb:799:in `send_message'
/usr/lib/ruby/vendor_ruby/mail/network/delivery_methods/smtp_connection.rb:53:in `deliver!'
/usr/lib/ruby/vendor_ruby/mail/network/delivery_methods/smtp.rb:101:in `block in deliver!'
/usr/lib/ruby/gems/3.3.0/gems/net-smtp-0.5.1/lib/net/smtp.rb:643:in `start'
/usr/lib/ruby/vendor_ruby/mail/network/delivery_methods/smtp.rb:109:in `start_smtp_session'
/usr/lib/ruby/vendor_ruby/mail/network/delivery_methods/smtp.rb:100:in `deliver!'
/usr/lib/ruby/vendor_ruby/mail/message.rb:2145:in `do_delivery'
/usr/lib/ruby/vendor_ruby/mail/message.rb:255:in `deliver'
/usr/lib/ruby/vendor_ruby/schleuder/mail/gpg/delivery_handler.rb:27:in `deliver_mail'
/usr/lib/ruby/vendor_ruby/mail/message.rb:253:in `deliver'
/usr/lib/ruby/vendor_ruby/schleuder/logger_notifications.rb:48:in `block in notify_admin'
/usr/lib/ruby/vendor_ruby/schleuder/logger_notifications.rb:29:in `each'
/usr/lib/ruby/vendor_ruby/schleuder/logger_notifications.rb:29:in `notify_admin'
/usr/lib/ruby/vendor_ruby/schleuder/logger_notifications.rb:13:in `error'
/usr/lib/ruby/vendor_ruby/schleuder/list.rb:379:in `rescue in block in send_to_subscriptions'
/usr/lib/ruby/vendor_ruby/schleuder/list.rb:362:in `block in send_to_subscriptions'
/usr/share/rubygems-integration/all/gems/activerecord-7.2.2.1/lib/active_record/relation/delegation.rb:98:in `each'
/usr/share/rubygems-integration/all/gems/activerecord-7.2.2.1/lib/active_record/relation/delegation.rb:98:in `each'
/usr/lib/ruby/vendor_ruby/schleuder/list.rb:361:in `send_to_subscriptions'
/usr/lib/ruby/vendor_ruby/schleuder/runner.rb:90:in `run'
/usr/lib/ruby/vendor_ruby/schleuder/cli.rb:38:in `work'
/usr/share/rubygems-integration/all/gems/thor-1.3.2/lib/thor/command.rb:28:in `run'
/usr/share/rubygems-integration/all/gems/thor-1.3.2/lib/thor/invocation.rb:127:in `invoke_command'
/usr/share/rubygems-integration/all/gems/thor-1.3.2/lib/thor.rb:538:in `dispatch'
/usr/share/rubygems-integration/all/gems/thor-1.3.2/lib/thor/base.rb:584:in `start'
/usr/bin/schleuder:13:in `<main>'
[Django] ERROR (EXTERNAL IP): Internal Server Error:.eml
Subject: [Django] ERROR (EXTERNAL IP): Internal Server Error: /postorius/lists/<REDACTED>@nos-oignons.net/held_messages
From: <REDACTED>@lists.nos-oignons.net
Date: Thu, 23 Oct 2025 05:06:17 -0000
To: root@localhost, <REDACTED>@nos-oignons.net
Internal Server Error: /postorius/lists/<REDACTED>@nos-oignons.net/held_messages
TemplateSyntaxError at /postorius/lists/<REDACTED>@nos-oignons.net/held_messages
'humanize' is not a registered tag library. Must be one of:
account
admin_list
admin_modify
admin_urls
allauth
bootstrap_tags
cache
compress
d_gravatar
date_helpers
debugger_tags
decorate
gravatar
highlight
highlighting
hk_generic
hk_haystack
i18n
indent_text
l10n
log
markdown
membership_helpers
more_like_this
nav_helpers
p_gravatar
pagination
postorius_helpers
rest_framework
socialaccount
static
syntax_color
tz
widont
Request Method: GET
Request URL: https://lists.nos-oignons.net/postorius/lists/<REDACTED>@nos-oignons.net/held_messages
Django Version: 4.2.23
Python Executable: /usr/bin/uwsgi-core
Python Version: 3.13.5
Python Path: ['.', '', '/usr/lib/python313.zip', '/usr/lib/python3.13', '/usr/lib/python3.13/lib-dynload', '/usr/local/lib/python3.13/dist-packages', '/usr/lib/python3/dist-packages']
Server time: Thu, 23 Oct 2025 05:06:17 +0000
Installed Applications:
('hyperkitty',
'postorius',
'django_mailman3',
'django.contrib.admin',
'django.contrib.auth',
'django.contrib.contenttypes',
'django.contrib.sessions',
'django.contrib.sites',
'django.contrib.messages',
'django.contrib.staticfiles',
'rest_framework',
'django_gravatar',
'compressor',
'haystack',
'django_extensions',
'django_q',
'allauth',
'allauth.account',
'allauth.socialaccount')
Installed Middleware:
('allauth.account.middleware.AccountMiddleware',
'django.contrib.sessions.middleware.SessionMiddleware',
'django.middleware.common.CommonMiddleware',
'django.middleware.csrf.CsrfViewMiddleware',
'django.middleware.locale.LocaleMiddleware',
'django.contrib.auth.middleware.AuthenticationMiddleware',
'django.contrib.messages.middleware.MessageMiddleware',
'django.middleware.clickjacking.XFrameOptionsMiddleware',
'django.middleware.security.SecurityMiddleware',
'django_mailman3.middleware.TimezoneMiddleware',
'postorius.middleware.PostoriusMiddleware')
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/django/template/defaulttags.py", line 1026, in find_library
return parser.libraries[name]
^^^^^^^^^^^^^^^^^^^^^^
During handling of the above exception ('humanize'), another exception occurred:
File "/usr/lib/python3/dist-packages/django/core/handlers/exception.py", line 55, in inner
response = get_response(request)
^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/django/core/handlers/base.py", line 197, in _get_response
response = wrapped_callback(request, *callback_args, **callback_kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/django/contrib/auth/decorators.py", line 23, in _wrapper_view
return view_func(request, *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/postorius/auth/decorators.py", line 66, in wrapper
return fn(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/postorius/views/list.py", line 889, in list_moderation
return render(request, 'postorius/lists/held_messages.html', context)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/django/shortcuts.py", line 24, in render
content = loader.render_to_string(template_name, context, request, using=using)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/django/template/loader.py", line 61, in render_to_string
template = get_template(template_name, using=using)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/django/template/loader.py", line 15, in get_template
return engine.get_template(template_name)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/django/template/backends/django.py", line 33, in get_template
return Template(self.engine.get_template(template_name), self)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/django/template/engine.py", line 175, in get_template
template, origin = self.find_template(template_name)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/django/template/engine.py", line 157, in find_template
template = loader.get_template(name, skip=skip)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/django/template/loaders/cached.py", line 57, in get_template
template = super().get_template(template_name, skip)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/django/template/loaders/base.py", line 28, in get_template
return Template(
File "/usr/lib/python3/dist-packages/django/template/base.py", line 154, in __init__
self.nodelist = self.compile_nodelist()
^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/django/template/base.py", line 200, in compile_nodelist
return parser.parse()
^^^^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/django/template/base.py", line 513, in parse
raise self.error(token, e)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/django/template/base.py", line 511, in parse
compiled_result = compile_func(self, token)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/django/template/loader_tags.py", line 293, in do_extends
nodelist = parser.parse()
^^^^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/django/template/base.py", line 513, in parse
raise self.error(token, e)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/django/template/base.py", line 511, in parse
compiled_result = compile_func(self, token)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/django/template/defaulttags.py", line 1088, in load
lib = find_library(parser, name)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/django/template/defaulttags.py", line 1028, in find_library
raise TemplateSyntaxError(
^
Exception Type: TemplateSyntaxError at /postorius/lists/ag@nos-oignons.net/held_messages
Exception Value: 'humanize' is not a registered tag library. Must be one of:
account
admin_list
admin_modify
admin_urls
allauth
bootstrap_tags
cache
compress
d_gravatar
date_helpers
debugger_tags
decorate
gravatar
highlight
highlighting
hk_generic
hk_haystack
i18n
indent_text
l10n
log
markdown
membership_helpers
more_like_this
nav_helpers
p_gravatar
pagination
postorius_helpers
rest_framework
socialaccount
static
syntax_color
tz
widont
Raised during: postorius.views.list.list_moderation
Since this is a django stacktrace complaining
about a missing application, so it should simply be a matter of
adding to /usr/share/mailman3-web/settings.py. Unfortunately:
root@bulbe:~# grep humanize /usr/share/mailman3-web/settings.py
'django.contrib.humanize',
root@bulbe:~#
After a fair amount of cursing, spelunking in mailman-users
and
various strace+grep monstrosities, the
solution was found: Instead of adding 'django.contrib.humanize' to the INSTALLED_APPS tuple
in /usr/share/mailman3-web/settings.py as one would expect, it should be
added in /etc/mailman3/mailman-web.py instead.
In the hope that this blogpost will help you waste less time on this than I did.