There are two types of workflows associated with mailing lists:
1) that in which you want any list subscriber to be able to send an email to the list’s email address, and that email to be automatically sent to all the other list subscribers;
2) that in which you want to send the same email message directly to thousands of subscribers and receive their responses (if any) on a ‘Reply-To’ email address.
For the first situation, the free and open source application of choice is Mailman, for the second, it’s phpList.
Mailman, officially ‘GNU Mailman’, is a web-based mailing lists manager that provides a web administration interface, while relying on a SMTP server such as Postfix to send and receive emails.
You can install Mailman from the Debian repositories by running:
apt-get install mailman
During the installation, you will be asked to select languages for Mailman. Use the arrow keys to move up and down. Use the space bar to select your language. A star character indicates that the language is selected. Press Enter to use the selected language. Immediately after that, a new window will inform you that you will have to create a new list called mailman. Just press Enter to close the window. You don’t need to create the mailman list at this moment.
Edit the /usr/lib/mailman/Mailman/mm_cfg.py file:
nano /usr/lib/mailman/Mailman/mm_cfg.py
Change/add the following settings, to make them look like this:
DEFAULT_URL_PATTERN = ‘https://%s/mailman/’
DEFAULT_EMAIL_HOST = ‘mailman.example.com’
DEFAULT_URL_HOST = ‘mailman.example.com’
add_virtualhost(DEFAULT_URL_HOST, DEFAULT_EMAIL_HOST)
POSTFIX_STYLE_VIRTUAL_DOMAINS = [‘mailman.example.com’]
MTA=’Postfix’
OWNERS_CAN_DELETE_THEIR_OWN_LISTS = yes
OWNERS_CAN_ENABLE_PERSONALIZATION = 1
Replace example.com with the main domain hosted on your server.
22.1. Configure Postfix
Open the /etc/postfix/main.cf file:
nano /etc/postfix/main.cf
Enable the following lines, by uncommenting them:
relay_domains = mailman.example.com
relay_recipient_maps = hash:/var/lib/mailman/data/virtual-mailman
transport_maps = hash:/etc/postfix/transport
mailman_destination_recipient_limit = 1
virtual_alias_maps = proxy:mysql:/etc/postfix/mysql-virtual-alias-maps.cf,
hash:/var/lib/mailman/data/virtual-mailman
Please note that you have to remove this line:
virtual_alias_maps = proxy:mysql:/etc/postfix/mysql-virtual-alias-maps.cf
since you already added the virtual_alias_maps parameter.
Also open the /etc/postfix/master.cf file:
nano /etc/postfix/master.cf
Uncomment to last two lines of the file, to make them look like this:
mailman unix – n n – – pipe
flags=FR user=list:list argv=/var/lib/mailman/bin/postfix-to-mailman.py ${nexthop} ${recipient}
Create the /etc/postfix/transport file:
nano /etc/postfix/transport
Add the following line inside this file:
mail.example.com mailman:
Replace example.com with the main domain hosted on your server. Please note that the domain is not mailman.example.com, but mail.example.com .
Turn the /etc/postfix/transport file into a Postfix lookup table:
postmap /etc/postfix/transport
Add user postfix to the list group:
adduser postfix list
Restart Postfix:
systemctl restart postfix
22.2. Install FCGIWrap
To properly display the Mailman web interface, we need to install the fcgiwrap package which allows us to run CGI applications with FastCGI and Nginx.
apt-get install fcgiwrap
Once installed, fcgiwrap will start automatically, as can be seen by running:
systemctl status fcgiwrap
Output:
● fcgiwrap.service – Simple CGI Server
Loaded: loaded (/lib/systemd/system/fcgiwrap.service; indirect; vendor preset: enabled)
Active: active (running) since Mon 2020-10-12 01:25:20 EDT; 4min 27s ago
Main PID: 27780 (fcgiwrap)
CGroup: /system.slice/fcgiwrap.service
└─27780 /usr/sbin/fcgiwrap -f
The fcgiwrap service listens on the /var/run/fcigwrap.socket by default.
22.3. Configure Nginx for Mailman
To access Mailman from a sub-directory of the mailman.example.com domain, first create the mailman.example.com directory:
cd /var/www
mkdir mailman.example.com
To obtain a Let’s Encrypt SSL Certificate for mailman.example.com, first edit the Nginx server blocks configuration file:
nano /etc/nginx/sites-enabled/0-conf
Add a temporary server block for mailman.example.com, at the bottom of the file:
server {
listen 80;
listen [::]:80;
server_name mailman.example.com;
location /.well-known/acme-challenge {
root /var/www;
}
}
Restart Nginx:
systemctl restart nginx
Next edit your DNS settings. Add an A and an AAAA entry for mailman.example.com. These entries will be identical to those for mail.example.com. It’s just that instead of mail, you’ll enter mailman . Wait a few minutes until the DNS changes propagate.
You will need to add an MX entry for mailman.example.com, like this:
mailman IN MX 10 mail.example.com.
You will also need to add a SPF record for mailman.example.com like this:
mailman IN TXT “v=spf1 a mx ip4:123.123.123.123 ip6:2b03:8df0:a24b:6eb::1 -all”
Where 123.123.123.123 is the IPv4 of your server and 2b03:8df0:a24b:6eb::1 is its IPv6. Please remember that 2b03:8df0:a24b:6eb::1 should be an address from the /64 subnet allocated by your hosting provider to your machine. If your provider didn’t allocate a /64 subnet to your server and they refuse to do so after you open a support ticket, it’s recommended to remove the ip6:2b03:8df0:a24b:6eb::1 part from the record presented above and to disable IPv6 connectivity for your mail server, in order to avoid having your emails rejected just because other neighboring IPs in the same /64 range sent spam.
To obtain the Let’s Encrypt certificate for mailman.example.com, run:
certbot certonly –agree-tos –webroot -w /var/www/ -d mailman.example.com
Then open the /etc/nginx/sites-enabled/0-conf file:
nano /etc/nginx/sites-enabled/0-conf
Replace the temporary server block for mailman.example.com with the following server blocks:
server {
listen 80;
listen [::]:80;
server_name mailman.example.com;
return 301 https://mailman.example.com$request_uri;
}
server {
listen 443 ssl http2;
listen [::]:443 ssl http2;
server_name mailman.example.com;
root /var/www/mailman.example.com;
index index.html;
ssl_certificate /etc/letsencrypt/live/mailman.example.com/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/mailman.example.com/privkey.pem;
ssl_trusted_certificate /etc/letsencrypt/live/mailman.example.com/chain.pem;
ssl_dhparam /etc/nginx/ssl/dhparam.pem;
ssl_session_timeout 4h;
ssl_session_cache shared:SSL:40m;
ssl_protocols TLSv1.2 TLSv1.3;
ssl_prefer_server_ciphers on;
ssl_ciphers ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384;
ssl_stapling on;
ssl_stapling_verify on;
add_header Strict-Transport-Security “max-age=63072000” always;
add_header X-Frame-Options SAMEORIGIN;
add_header X-Content-Type-Options nosniff;
location = /robots.txt {
allow all;
}
location / {
try_files $uri $uri/ /index.php?$args;
}
location /.well-known/acme-challenge {
root /var/www;
}
location /mailman {
root /usr/lib/cgi-bin;
fastcgi_pass unix:/var/run/fcgiwrap.socket;
fastcgi_param HTTPS on;
fastcgi_split_path_info (^/mailman/[^/]*)(.*)$;
fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
fastcgi_param PATH_INFO $fastcgi_path_info;
include /etc/nginx/fastcgi_params;
}
location /mailman/admin {
auth_basic ‘Restricted’;
auth_basic_user_file /etc/nginx/htpass/mailman.example.com;
root /usr/lib/cgi-bin;
fastcgi_pass unix:/var/run/fcgiwrap.socket;
fastcgi_param HTTPS on;
fastcgi_split_path_info (^/mailman/[^/]*)(.*)$;
fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
fastcgi_param PATH_INFO $fastcgi_path_info;
include /etc/nginx/fastcgi_params;
}
location /images/mailman {
alias /usr/share/images/mailman;
}
location /pipermail {
alias /var/lib/mailman/archives/public;
autoindex on;
}
access_log /var/log/sites/mailman.example.com/access.log;
error_log /var/log/nginx/mailman.example.com.error.log notice;
}
Replace example.com with the main domain hosted on your server.
Create the access log directory:
mkdir /var/log/sites/mailman.example.com
Create the robots.txt file.
cd /var/www/mailman.example.com
nano robots.txt
Add the following lines inside this file:
User-agent: *
Disallow: /mailman
Disallow: /images
In this way all the search engine bots will not crowl and index the mailman.example.com/mailman directory, the mailman.example.com/images directory and their respective subdirectories, but they will crowl and index the mailman.example.com/pipermail directory, containing all the public mailing list archives and their messages.
22.4. Configure logrotate to rotate the Mailman access logs
nano /etc/logrotate.d/nginx
Add the following section at the bottom of this file:
/var/log/sites/mailman.example.com/access.log {
missingok
rotate 10
compress
delaycompress
notifempty
create 0640 www-data adm
size 2M
sharedscripts
prerotate
if [ -d /etc/logrotate.d/httpd-prerotate ]; then \
run-parts /etc/logrotate.d/httpd-prerotate; \
fi; \
endscript
postrotate
[ ! -f /var/run/nginx.pid ] || kill -USR1 `cat /var/run/nginx.pid`
endscript
}
Replace example.com with the main domain hosted on your server.
22.5. Initial configuration
Please note that you have set up HTTP authentication for the ‘mailman/admin’ directory so as to prevent unauthorized access to the Mailman administration page. You’ll have to create the /etc/nginx/htpass/mailman.example.com password file, to allow specific users to access the Mailman administration page. If you want to give access to the user gerald, run:
htpasswd -c /etc/nginx/htpass/mailman.example.com gerald
To add the user verner to the same password file run:
htpasswd /etc/nginx/htpass/mailman.example.com verner
Change ownership and permissions for the password file:
chown www-data:root /etc/nginx/htpass/mailman.example.com
chmod 400 /etc/nginx/htpass/mailman.example.com
Restart Nginx:
systemctl restart nginx
Next, run the following command to create a site list creator password and avoid a future error when creating new lists:
mmsitepass
New site password: strongpassword
Please note that you cannot start Mailman until you create the mailman list. To create it run:
newlist mailman
The output of this command and your answers should be:
Enter the email of the person running the list: admin@example.com
Initial mailman password: strongpassword
Hit enter to notify mailman owner… (Press Enter)
Replace admin@example.com with an email address that you want to use for this task. Restart Mailman:
systemctl restart mailman
Now you can access the administration web page of Mailman at:
On the administration page you will see a list with the names of all the mailing lists available. Since you have created only one mailing list, ‘mailman’, you will see its name on the list:
If you click on ‘Mailman’ under List, you will be asked to enter the password that you configured earlier when you created the ‘mailman’ list, then you will see the administration page for this list:
Here you can click on an entry in the ‘Configuration Categories’ or ‘Other Administrative Activities’ sections and you can change any setting available for that category or activity.
22.6. Create a new mailing list
To create a new mailing list go to https://mailman.example.com/mailman/admin and click on the ‘create a new mailing list’ link. The new screen will look like this:
In the ‘Name of list’ field enter the name of the new list, let’s say test1. In the ‘Initial list owner address’ field enter the email address of the list owner, let’s say admin@example.com (it can be any email address that you like), in the ‘Initial list password’ field enter a password for this list, confirm the password, then, check a language from the list of languages, in the ‘List creator’s (authentication) password’ field at the bottom of the form, enter the password that you configured earlier, when you created the ‘mailman’ list in command line, then click ‘Create List’. In the next window you will be informed that the new list has been created.
You can change the settings of a list by going to the administration page: https://mailman.example.com/mailman/admin and clicking on the name of the list that you want to configure. Then you can click on an entry in the ‘Configuration Categories’ or ‘Other Administrative Activities’ sections and you can change any setting available for that category or activity.
22.7. Add subscribers to a mailing list
To manually add multiple subscribers to the test1 mailing list that you have just created, first go to the list’s administration page: https://mailman.example.com /mailman/admin/test1 . Here click on ‘Membership Management’ > ‘Mass Subscription’ , then enter one email address per line in the first text box, then click ‘Submit Your Changes’ at the bottom of the page.
However, the typical way in which subscribers are added to a mailing list is by directing them to the list’s subscription page: https://mailman.example.com/mailman/listinfo/test1 :
On this page they can enter their email address and a password. However, it’s better to leave the password field empty, because Mailman will generate a random password and send it in the ‘Welcome’ message. The subscriber can then change the generated password with the password that they desire.
After subscribers click ‘Subscribe’, they receive an email in which they have to click a link to confirm their subscription. Once they confirm their subscription, they receive the ‘Welcome’ message containing the URL of their subscription page:
https://mailman.example.com/mailman/options/test1/username@domain.com
where username@domain.com is the email address that they used to subscribe. That message also contains the email address of the mailing list (name-of-list@mailman.example.com), the URL with general information about the mailing list (https://mailman.example.com/mailman/listinfo/name-of-list) and the subscriber’s password.
On their subscription page, the subscribers can change their email address, they can change their password, they can unsubscribe and they can make other configurations related to the way they want to interact with the mailing list.
The next step for new subscribers should be to go to their subscription page and change their password, in the ‘Change Your Password’ section. Also, under ‘Get password reminder email for this list?‘ they should select ‘No’, then click ‘Submit My Changes’, so that they can avoid receiving an email with a password reminder every month.
22.8. Send an email to a mailing list
Once a person subscribes to a mailing list, they can send messages to all the subscribers of that list by sending the email to the list’s email address: name-of-list@mailman.example.com .
When trying to send an email to a list address, if you ever encounter Postfix errors such as: ‘User not found in local recipients table’ you can simply delete the aliases and virtual-mailman files:
cd /var/lib/mailman/data
rm aliases aliases.db virtual-mailman virtual-mailman.db
then regenerate these files by running:
/usr/lib/mailman/bin/genaliases
Then restart Postfix and Mailman:
systemctl restart postfix
systemctl restart mailman
The administration page is:
The page where a user can subscribe to a list is:
If the archive page of a list is public, the URL of the archive page will be:
If you want the archive pages to be publicly accessible for a particular list, so that they can be viewed by anyone and indexed by search engines, on the list’s administration page go to ‘Archiving Options’ and verify that ‘Is archive file source for public or private archival?’ is set to ‘public’. Also, as mentioned before, to let search engines index the archive pages, you should make sure that the content of the /var/www/mailman.example.com/robots.txt file looks like this:
User-agent: *
Disallow: /mailman
Disallow: /images
In this way, all the search engine bots will be instructed not to crowl and index the mailman.example.com/mailman directory, the mailman.example.com/images directory and their respective subdirectories, but to crowl and index the mailman.example.com/pipermail directory, containing all the public mailing list archives, with their respective messages.
If on the contrary, you don’t want the archive pages to be publicly accessible, on the list’s administration page, under ‘Archiving Options’, you should set ‘Is archive file source for public or private archival?’ to ‘private’ and click ‘Submit Your Changes’; from that moment on, the archive pages for that respective list will be accessible only after entering the email address and password of a list’s subscriber. For a private archive, the URL will be:
https://mailman.example.com/mailman/private/name-of-list/
If all the mailing lists on your server are private lists, you should also include the following line in the server block for mailman.example.com, in the /etc/nginx/sites-enabled/0-conf file, right above the location = /robots.txt { line:
add_header X-Robots-Tag “noindex, nofollow, nosnippet, noarchive”;
Please note that you will have to send emails in plain textor in both plain text and HTML format to mailing lists, if you want to be sure that all the receivers will be able to view them properly. If you send messages to mailing lists in HTML format, by default, the archived messages will be displayed as ‘scrubbed’ attachments and will contain all the escaped HTML tags, thus becoming impossible to read. The best method is to send emails to mailing lists in both plain text and HTML format. In this way, the receivers that can view them as HTML will view them as such, those who can view plain text only emails will view them as plain text, and the archived messages will contain the proper plain text version, with the HTML version attached as scrubbed content.
If you use Thunderbird to send emails to mailing lists, to make sure that you send them in plain text, you should go to Edit > Preferences > Composition > General tab > click on the ‘Send Options …’ button; uncheck the ‘Send messages as plain text if possible’ box and from the drop-down list under ‘When sending messages in HTML format and one or more recipients are not listed as being able to receive HTML’, choose ‘Send the message in both plain text and HTML’ > click OK.
When replying to messages coming from a mailing list, the subscribers have to click on the ‘Reply List’ button in their email client. If they click on ‘Reply’, they will send their reply just to the subscriber who sent the initial message and not to the other subscribers.
If a subscriber has a badly configured auto-responder that sends “I’m on vacation” replies to all incoming messages, this can trigger a mailing list loop: each automatic reply will be sent back to the mailing list, Mailman will then send it to all the subscribers, the auto-responder will receive another email and will reply again, etc. This can clutter the mailing list very rapidly. Well configured auto-responders will identify that the email comes from a mailing list and will not trigger this loop. However, if it ever happens, the first thing to do is to go to the list’s administration page (https://mailman.example.com/mailman/admin/name-of-list) > ‘Membership Management’ > ‘Membership List’ and there check the ‘mod’ and ‘nomail’ checkboxes next to the email address of the user causing the loop, in order to remove her/his ability to send emails to the list and receive emails from the list respectively, then click on the ‘Submit Your Changes’ button. If you cannot identify the user causing the loop quickly, you can go to the ‘General Options’ and at ‘Emergency moderation of all list traffic’ select ‘Yes’, then click on ‘Submit Your Changes’. This will suspend all email traffic to/from the mailing list. The next step is to inform the user that caused the loop that (s)he has a badly configured auto-responder that needs to be fixed.
22.9. Configure Fail2ban to protect Mailman against brute-force attacks
Navigate to the /etc/fail2ban/filter.d directory:
cd /etc/fail2ban/filter.d
Create a file called mailman.conf:
nano mailman.conf
Add the following content in this file:
[Definition]
failregex = ^<HOST> .* \”POST /mailman/admin/.* HTTP/2.0\” 401 2.*$
^<HOST> .* \”POST /mailman/private/.* HTTP/2.0\” 401 2.*$
^<HOST> .* \”GET /mailman/admin.* HTTP/2.0\” 401 195 .*$
^<HOST> .* \”GET /mailman/private.* HTTP/2.0\” 401 195 .*$
ignoreregex =
Please note that the third and the fourth line in the ‘failregex’ section are used to identify failed log in attempts against the HTTP authentication login window, which can be also subject to brute-force attacks.
Next, open the /etc/fail2ban/jail.local file:
nano /etc/fail2ban/jail.local
Add the following section below the [lighttpd-auth] section:
[mailman]
enabled = true
filter = mailman
logpath = /var/log/sites/mailman.example.com/access.log
port = 80,443
maxretry = 4
bantime = 259200
Replace example.com with the main domain hosted on your server.
Reload Fail2ban:
systemctl reload fail2ban
22.10. Upgrading Mailman
Before upgrading Mailman to a new version, it’s recommended to verify if the new version has been tested and confirmed to function well within the suite of applications described in this guide. Once we test an application and confirm that it works well, we include it on this page.
Since the mailman package has been installed from the Debian repository and it can be included automatically among many other upgrades when executing apt-get dist-upgrade or apt-get upgrade, it is recommended to exclude it from automatic upgrades so as to prevent an upgrade before making a backup of the most important files. Therefore, first exclude mailman from automatic upgrades by running:
apt-mark hold mailman
Then, you can upgrade Mailman manually, after making all the necessary backups.
It’s recommended to run apt-get dist-upgrade periodically, at least once every four weeks. When you run this command, if you see:
The following packages have been kept back:
mailman
The following packages will be upgraded:
…
this means that there is an upgrade to Mailman that has been held back. So, after upgrading all the other packages, you can upgrade Mailman manually, by following the next steps:
Stop Mailman:
systemctl stop mailman
Go to the backups directory, /var/bm_archives, and make a backup copy of the /var/lib/mailman directory:
cd /var/bm_archives
tar czf var-lib-mailman-2020-10-20.tgz /var/lib/mailman
Replace 2020-10-20 with the date of the backup. Also make a backup copy of the /etc/mailman directory:
tar czf etc-mailman-2020-10-20.tgz /etc/mailman
Upgrade Mailman to the latest version by running:
apt-get install mailman
Start Mailman:
systemctl start mailman
If you want to see all the packages that are excluded from upgrading at a certain moment, run:
dpkg –get-selections | grep hold
If you want to unhold a package run:
apt-mark unhold packagename
You can also join the Mailman-announce mailing list to receive a notification when a new version of mailman has been released. However, if a new version has been officially released, this doesn’t mean that it will be immediately included in the Debian repositories to install it from there. Usually any new package is included in the Debian repositories after a certain period of time necessary for testing.