Multipart Emails in Neomutt

2022-05-27

It recently came to my attention that mutt now supports sending multipart emails. I thought that this would mean that in half an hour or so I would have html emails working. Turns out, I was wrong. What instead happened was weeks of trial and error and reading RFCs.

I now have a system I am happy with. I write an email in markdown and mutt (along with some surrounding scripts) will convert that markdown to html, attach inline images and create a multipart email.

A note about HTML emails

If you do need to send HTML emails, please spare a thought for your recipient; it is not just “weird” terminal email client users that could suffer. According to the National Eye Institute, one in twelve men are colour blind; so please don’t use only colour to distinguish items. Additionally, many people suffer from vision-loss blindness, who are likely to use screen readers or braille displays. Complex layouts or heavy use of images are going to give these people a poor experience.

That being said, HTML emails can be used for aesthetic purposes, whilst still providing a plain text version for those who want it. That is the method I suggest adopting if you need html emails, and the method this blog post will describe.

To begin with, we need to understand a little bit about how emails are structured. Below is an example tree structure of a standard email.

Multipart Related
├─>Multipart Alternative
│ ├─>Plain Text Email
│ └─>HTML Email
└─>Inline Image Attachment
Non-Inline Attachment

Starting at the lowest level, we see a plain text email and an HTML email. These are both wrapped in a multipart alternative wrapper. This tells email clients receiving the email that they are alternative versions of the same document. The email client will normally choose which to display based on the mime type and user preferences.

The multipart alternative wrapper and an image attachment are then wrapped in a multipart related wrapper. This tells the email client that the contents are related to one another, but not different version of the same document. This is where inline images are attached.

Finally, there is another attachment that is outside of the multipart related wrapper. This will show up as another attachment but cannot be displayed inline.

Neomutt Configuration

The conversion from markdown to html will be handled by an external script. It will create files and instruct mutt to attach them.

We can start with the following:

macro compose Y "<first-entry>\
<pipe-entry>convert-multipart<enter>\
<enter-command>source /tmp/neomutt-attach-macro<enter>

We specify a macro to run when Y is pushed. First, we select the first entry. This is in case we have attached anything manually, the first entry should be the markdown file.

We then pipe the selected entry (the markdown file) to an external script, in this case a bash script called convert-multipart. Finally we source a file called /tmp/neomutt-commands. This will be populated by the script and will allow us to group and attach files inside neomutt.

Converting to HTML

Let’s start with a simple pandoc conversion.

#!/usr/bin/env bash

commandsFile="/tmp/neomutt-commands"
markdownFile="/tmp/neomutt-markdown"
htmlFile="/tmp/neomutt.html"

cat - > "$markdownFile"
echo -n "push " > "$commandsFile"

pandoc -f markdown -t html5 --standalone --template ~/.pandoc/templates/email.html "$markdownFile" > "$htmlFile"

# Attach the html file
echo -n "<attach-file>\"$htmlFile\"<enter>" >> "$commandsFile"

# Set it as inline
echo -n "<toggle-disposition>" >> "$commandsFile"

# Tell neomutt to delete it after sending
echo -n "<toggle-unlink>" >> "$commandsFile"

# Select both the html and markdown files
echo -n "<tag-entry><previous-entry><tag-entry>" >> "$commandsFile"

# Group the selected messages as alternatives
echo -n "<group-alternatives>" >> "$commandsFile"

The above bash script will create an html file using pandoc, and create a file of neomutt commands. This instructs neomutt to attach the html file, set its disposition, and group the markdown and html files into a “multipart alternatives” group.

Neomutt’s attachment view should look something like this.

  I   1 <no description>                             [multipa/alternativ, 7bit, 0K]
- I   2 ├─>/tmp/neomutt-hostname-1000-89755-7    [text/plain, 7bit, us-ascii, 0.3K]
- I   3 └─>/tmp/neomutt.html                      [text/html, 7bit, us-ascii, 9.5K]

Inline attachments

The next part of the puzzle is inline attachments. These need to be attached and then grouped within a multipart related group.

To reference the file from within the html email, each inline image needs a unique cid. I use md5 sums for this. They are not cryptographically secure, but for the purposes of generating unique strings for images in an email, they are fine.

grep -Eo '!\[[^]]*\]\([^)]+' "$markdownFile" | cut -d '(' -f 2 |
    grep -Ev '^(cid:|https?://)' | while read file; do
        id="cid:$(md5sum "$file" | cut -d ' ' -f 1 )"
        sed -i "s#$file#$id#g" "$markdownFile"
    done

We loop through all the images in the markdown file, and replace the paths for cids (assuming they are not already cids or remote images).

As the markdown has changed, we need to attach the new one and detach the old.

if [ "$(grep -Eo '!\[[^]]*\]\([^)]+' "$markdownFile" | grep '^cid:' | wc -l)" -gt 0 ]; then
    echo -n "<attach-file>\"$markdownFile\"<enter><first-entry><detach-file>" >> "$commandsFile"
fi

To attach the images, we loop through the original file and add to the file neomutt sources. Neomutt will be instructed to attach, set the disposition, set the content ID and tag the image.

grep -Eo '!\[[^]]*\]\([^)]+' "${markdownFile}.orig" | cut -d '(' -f 2 |
    grep -Ev '^(cid:|https?://)' | while read file; do
    id="$(md5sum "$file" | cut -d ' ' -f 1 )"
    echo -n "<attach-file>\"$file\"<enter>" >> "$commandsFile"
    echo -n "<toggle-disposition>" >> "$commandsFile"
    echo -n "<edit-content-id>^u\"$id\"<enter>" >> "$commandsFile"
    echo -n "<tag-entry>" >> "$commandsFile"
done

if [ "$(grep -Eo '!\[[^]]*\]\([^)]+' "$markdownFile" | grep '^cid:' | wc -l)" -gt 0 ]; then
    echo -n "<first-entry><tag-entry><group-related>" >> "$commandsFile"
fi

Finally, if there were any images attached, we select the first entry (the multipart alternative we’ve already created), tag it and mark everything tagged as multipart related.

  I     1 <no description>                            [multipa/related, 7bit, 0K]
  I     2 ├─><no description>                      [multipa/alternativ, 7bit, 0K]
- I     3 │ ├─>/tmp/neomutt-markdown           [text/plain, 7bit, us-ascii, 0.3K]
- I     4 │ └─>/tmp/neomutt.html                [text/html, 7bit, us-ascii, 9.5K]
  I     5 └─>/tmp/2022-05-27T15-02-20Z.png              [image/png, base64, 0.5K]

At this point, the user is free to attach additional, non inline documents as normal. This email should be good for both text based and graphical email clients.

For the full source changes, see this commit.

Multipart Emails in Neomutt

A note about HTML emails

Multipart / Related emails

Neomutt Configuration

Converting to HTML

Inline attachments