RFC 2047 header decoding doesn't work

[dannysauer]

Messages with RFC2047-encoded non-ascii characters render as if that header (usually from or subject) are empty. This is not the ideal way to handle international characters for people like me who work with a geographically distributed team.

Header and result image attached since text triggers the "you can't post external URLs" filter. :/

[Kostya Vasilyev]

Looks like we may have a bug in some edge case...

Overall this app has had this type of decoding since... oh, 2011 or so?

Can you send the original email as .eml attachment to kmansoft / gmail (my personal account)?

You can get it in Gmail web mail if you use that.

If not you can do three dots -> view -> show headers in Aqua Mail and there is a copy button there.

PS - what is *your* mail server / service on the receiving side?

[Kostya Vasilyev]

PS - for me, the header in your screenshot decoded to:

"Rafael Fernández López <notifications@github.com>"

[dannysauer]

Sorry; I'm a little slow on the response here.

That's what I expected it to display as.

I'm running postfix as the receiving MTA and dbmail for the IMAP service; filtered through SpamAssassin. These decode properly in squirrel mail (in chrome) and thunderbird on my desktop, so I'm pretty sure the issue is local to my android device (albeit not _necessarily_ aquamail).

I'll forward the full mail in a sec.

[dannysauer]

Just double-checked.  Thunderbird actually doesn't behave completely correctly - in the message list, it shows Rafael =?iso... but in the message display it shows Rafael Fernández...  That could theoretically be a weird protection against phishing, though, and not an accidental bug.

Also, doing some more digging, these messages do show up properly in Aquamail when they're received through a Groupwise IMAP server.  The Groupwise-received messages which work properly in Aquamail also work properly in both message list and message display in Thunderbird.  It might actually be something extra on my end confusing things, or it might be that Rafael is composing the Groupwise messages in mutt on his workstation, while GitHub is doing something differently when it generates messages.

I'm having a harder time finding an example where this happens and where it was generated by something other than GitHub.

[Kostya Vasilyev]

Um these messages are *not* sent by Rafael (I think) - they are sent by GitHub software.

That's plainly visible in your headers:

From: "Rafael ... <notifications@github.com>"

Re: That could theoretically be a weird protection against phishing, though, and not an accidental bug.

I think it's a bug loud and clear and not anything "weird" but "intentional".

Question is - whose bug is it?

Probably can't be an identical (or almost identical) bug in two completely unrelated mail apps, Aqua and Thunderbird.

The server then.

You mentioned that using GroupWise mail server - everything is fine.

Then I assume that using some other mail server - everything is *not* fine?

If so it must mean that your "other" mail server is doing "something bad" to Q/B encoded headers.

You can enable Aqua Mail's debug logging, open a message, Diag: reload (last item in three dots menu) and then examine the log to see what the server returns.

[dannysauer]

I suppose the point of the two different origins is more that Rafael likely typed his own display name into both GitHub and mutt using the same character set and input method, not that he's actually creating the emails.

The curious thing about Thunderbird's involvement here is that, from my IMAP server, it displays the name properly when I open the message, but displays it improperly in the list of messages.  I was hoping that would provide a clue as to the cause.  I'll do a bit more digging to see what actually differs.

[Kostya Vasilyev]

Re: my IMAP server

Oh so there  *is* a different IMAP server after all (wasn't completely sure).

Re: The curious thing about Thunderbird's involvement ... hoping that would provide a clue

It only tells us one thing: "something" is still wrong in TB (I mean "affecting" TB, not wrong in TB itself).

And it correlates with the mail server.

And it correlates with Aqua.

That it shows good in message view in TB tells us almost nothing (I can think of at least two reasons).

---

That is even more data points pointing to "your" (unnamed and mysterious) mail server.

One way to debug it is what I already wrote above. There are others. Good luck.

[dannysauer]

Yeah, there's a different IMAP involved: "I'm running postfix as the receiving MTA and dbmail for the IMAP service; filtered through SpamAssassin".

So I have several moving parts to look at.  I'll follow-up when I've looked at them, because I'm probably not the only one curious about which part is causing the issue.  My intention is to fabricate a message and set postfix to deliver to a local file so I can compare what it does.  Then add Spamassassin in to the mix, and finally dbmail.  If it's my end, which seems likely, I'm strongly inclined to initially suspect DBMail -- specifically that the "split the message up and put it in MySQL" step is doing something to mess up the character encoding.

[Kostya Vasilyev]

Well it's probably not the encoding itself but rather some mess-up of the overall format of that header -

- although "Q/B encoding" aka "encoded words" is a standard (as we both know).

Like I said you can also enable Aqua Mail logging (see my signature for link) and Diag: Reload in message view (last item in the menu).

In the log, look for something like this, Aqua Mail asking the server for specific headers from a message (not all headers or the full original RFC 822 format data):

Sending: kman22 UID FETCH 433 (UID BODY.PEEK[HEADER.FIELDS (DATE MESSAGE-ID LIST-ID SUBJECT FROM REPLY-TO TO CC BCC X-PRIORITY RESENT-FROM IN-REPLY-TO REFERENCES DISPOSITION-NOTIFICATION-TO CONTENT-TYPE AUTHENTICATION-RESULTS)] BODYSTRUCTURE RFC822.SIZE INTERNALDATE)

You should get back something like this, note the "From:" is exactly how it was in the original email:

* 222 FETCH (UID 433 INTERNALDATE "6-Nov-2018 07:39:51 -0500" RFC822.SIZE 709 BODYSTRUCTURE ("TEXT" "PLAIN" ("CHARSET" "US-ASCII") NIL NIL "7BIT" 411 16 NIL NIL NIL NIL) BODY[HEADER.FIELDS (DATE MESSAGE-ID LIST-ID SUBJECT FROM REPLY-TO TO CC BCC X-PRIORITY RESENT-FROM IN-REPLY-TO REFERENCES DISPOSITION-NOTIFICATION-TO CONTENT-TYPE AUTHENTICATION-RESULTS)] {279}
From: Rafael =?iso-8859-1?q?Fern=E1ndez_L=F3pez?= <notifications@github.com>
Subject: Git Hub Sample
Date: Wed, 30 May 2018 08:51:43 +0200
Content-Type: text/plain; charset="US-ASCII"
To: ...
Message-ID: <7N8TWL4SEKXBXVQXMDTKMMDASSTOI1N9....@fastmail.com>
)
kman22 OK Completed (0.001 sec)

In your case - with DBMail - I suspect you'll see something different.

[Kostya Vasilyev]

These are both rather old, but who knows...

https://sourceforge.net/p/squirrelmail/mailman/squirrelmail-users/?limit=250&viewmonth=200911&style=flat&viewday=15

https://www.mail-archive.com/dbmail@dbmail.org/msg16578.html

[Kostya Vasilyev]

This looks quite relevant:

https://www.mail-archive.com/dbmail@dbmail.org/msg16610.html