Is there an option to set "Content-Transfer-Encoding: quoted-printable"?

[nagmat84]

AquaMail sends plain text e-mails with

Content-Type: text/plain; format=flowed; charset="UTF-8"
Content-Transfer-Encoding: 8bit

if "flowed" is enabled by the particular option. In that case AquaMail still inserts hard line break after 76 characters, but the receiving client is allowed to re-flow the message. However, many clients do not respect the "flowed" option and still break the lines. This leads to annoying double line breaks and half filled lines on small screens.

Is there any option to either prevent AquaMail from inserting the line breaks or alternatively switch to Content-Transfer-Encoding: quoted-printable?

The first option is not perfectly in accordance to the RFCs because lines must no be longer than 76 characters, but this is what most clients do and understand. The second option is perfectly valid, lines are broken up after 76 characters but "soft line" breaks, i.e. line breaks that should not be visible to the reader, are marked with a trailing  = just before the  <CR><LF>.

Does AquaMail support any of these?

[Kostya Vasilyev]

I don't follow how changing the transfer encoding would help at all.

the "=<newline>" when decoded would turn into a newline proper, which is what you have now with transfer-encoding=8bit.

The flowed encoding is "backwards compatible", email apps that don't understand it should display the line breaks, and I'm not sure where the "double line breaks" are coming from.

Basically, "format=flowed" is a way to mark some of the line breaks as "soft", so the receiving mail app can join it (if appropriate). But there are no extra line breaks.

But yes, an email app that doesn't understand flowed format will show the "76 column" line break and also possibly break the lines somewhere else, before that boundary is reached.

This is not perfect, but is quite normal. The proper fix for this is for the receiving mail app to understand and decode "flowed", so we're back to square one.

Yes, one way to "hack it" on the sending side would be to wrap at some larger boundary, not 76 characters. You'd have less wrapping on the receiving side then, but you'd still have it in long sentences (and sorry there is no setting for that right now).

[nagmat84]

Hi,

> the "=<newline>" when decoded would turn into a newline proper, which is what you have now with transfer-encoding=8bit.

No, this is where you err. Have a look at RFC 2045, ch. 6.7, item (5) "soft line breaks". Content-Transfer-Encoding: quoted-printable only uses some basic ASCII letters directly to represent themselves and the "=" as an escape character. A non-escape line break in the e-mail body is a hard line break and must be displayed to the user as is (i.e. end of paragraph). A line-break preceded by an "=" indicates a soft-line break and must be removed from the text before the text is rendered on the user's screen. Those soft line breaks can also be placed in the middle of a word without introducing a space.

> and I'm not sure where the "double line breaks" are coming from.

Speaking of "double" line breaks is misleading, better call them superfluous annoying line breaks. Assume you have a narrow screen that approximately can display 50 characters and the e-mail body has hard line breaks every 76 characters. Then the email viewer renders a first line of text with 50 characters, then wraps the line, because the right border of the display is reached and the second line which is the rest of the first line of the email body is only half-filled with 26 characters, because the email body contains a hard line break. The result is an email with alternating half-filled lines.

> Yes, one way to "hack it" on the sending side would be to wrap at some larger boundary, not 76 characters. You'd have less wrapping on the receiving side then, but you'd still have it in long sentences

No, you would not, if you would not insert additional hard line breaks anywhere, not after 76 characters, not after 100, never. Only put hard line breaks into the email body where the user who typed the email inserted them by purpose, i.e.usually at the end of a paragraph. However, as I already admitted this is not compliant to any RFC but it works and for example is also the way how the auto-generated emails of this forum are formatted.

So let my sum up what options exists:

Option 1: No additional hard line breaks at all

This option is basically what I discussed last. If a user composed a new email never insert additional hard line breaks into the email body but only keep the line breaks that were introduced by the composer by purpose. This basically means "one paragraph per line". This solution is cheap, simple and works with any charset and any Content-Transfer-Encoding. However, it is not compliant to any RFC which request that no line of an email body must exceed 76 characters.

Pros: simple, cheap solution, "just works"
Cons: incompliant to RFCs

Option 2: The quoted-printable-solution

Prior to format=flowed there has been already a solution to allow the client to re-flow the content as specified by RFC 2045, sec. 6.7 The idea is to indicate "soft" line beaks by a preceding =. This kind of line breaks are no real line breaks that should be considered as a part of the content but line breaks that are only inserted to follow the 76 characters per line limitation. This approach is widely supported and chosen by all major desktop email clients.

Pros: Totally compliant to any RFC, widely supported
Cons: Blow up of data volume because only octets with decimal values of 33 through 60 and 62 through 126 are represented literally, everything else is escaped by = followed by hexadecimal digests

Option 3: The format=flowed-solution

"Modern" alternative to option 2. Any octet can be represented literally, because format=flowed can be combined with Content-Transfer-Encoding: 8bit. Single line breaks indicate "soft" line breaks, i.e. line breaks that are only inserted to adhere to the 76 character limitation; double line breaks indicate "hard" line breaks, i.e. end of paragraph. In some way this resembles the LaTeX approach.

Pros: Compact representation of octets, can be combined with 8-bit encoding, totally compliant to any RFC
Cons: Not widely supported, most e-mail clients to not understand the format=flowed directive

[StR]

Let me chime in with my perceptive on the proposed options:

Option 1: No additional hard line breaks at all

This option is basically what I discussed last. If a user composed a new email never insert additional hard line breaks into the email body but only keep the line breaks that were introduced by the composer by purpose. This basically means "one paragraph per line". This solution is cheap, simple and works with any charset and any Content-Transfer-Encoding. However, it is not compliant to any RFC which request that no line of an email body must exceed 76 characters.

Pros: simple, cheap solution, "just works"
Cons: incompliant to RFCs

Cons: May not work with some servers, mail filters (Spam, Anti-virus, ... scanners). Can lead to unclear consequences, e.g. truncation of the line when the line exceeds some internal software limit, message rejection due to RFC noncompliance, message being silently dropped (for the same reason), possibly breaking with some implementation of dkim-checking filter. If/when Kostya implements encryption/signing, that may break too (or give plenty of headache).
And, the RFCs, if not adhered to, will stop being the common denominator (aka standard) for all participants, which will lead to a chaos.

And I know some people who are reluctant to insert any line breaks, saying that the program should insert by itself, so they wouldn't bother. AS a result you get a full-page message with no line breaks.

Option 2: The quoted-printable-solution

Prior to format=flowed there has been already a solution to allow the client to re-flow the content as specified by RFC 2045, sec. 6.7 The idea is to indicate "soft" line beaks by a preceding =. This kind of line breaks are no real line breaks that should be considered as a part of the content but line breaks that are only inserted to follow the 76 characters per line limitation. This approach is widely supported and chosen by all major desktop email clients.

Pros: Totally compliant to any RFC, widely supported
Cons: Blow up of data volume because only octets with decimal values of 33 through 60 and 62 through 126 are represented literally, everything else is escaped by = followed by hexadecimal digests

I personally hate quoted-printable. (It's a burp of the 7-bit era servers.)
It complicates "grep" on the raw messages, reading raw messages. I suspect it can also interfere with message signing (if/when implemented).

Option 3: The format=flowed-solution

"Modern" alternative to option 2. Any octet can be represented literally, because format=flowed can be combined with Content-Transfer-Encoding: 8bit. Single line breaks indicate "soft" line breaks, i.e. line breaks that are only inserted to adhere to the 76 character limitation; double line breaks indicate "hard" line breaks, i.e. end of paragraph. In some way this resembles the LaTeX approach.

Pros: Compact representation of octets, can be combined with 8-bit encoding, totally compliant to any RFC
Cons: Not widely supported, most e-mail clients to not understand the format=flowed directive
[/quote]

I am not sure how this solution is much different (in the outcomes) from the existing situation.
But I think the mail client should start

[nagmat84]

Option 1: No additional hard line breaks at all
[...]

Cons: May not work with some servers, mail filters, [...] And, the RFCs, if not adhered to, will stop being the common denominator (aka standard) for all participants, which will lead to a chaos.

I am totally with you. Under no circumstance option 1 should be implemented. I only listed that option for the sake of completeness and to get this thread organized by making unambiguously clear what "option 1" is about.

Option 2: The quoted-printable-solution
[...]

I personally hate quoted-printable. (It's a burp of the 7-bit era servers.) It complicates "grep" on the raw messages, reading raw messages. I suspect it can also interfere with message signing (if/when implemented).

Yes, that is a true point. However, this is the only widely implemented solution how to get soft line breaks. From a developer's point of view the "modern" approach (see option 3) is much more desirable. Clearly, we have some sort of chicken-egg problem here. If you support the legacy approach (option 2) then there is no need for other email clients to adopt the new approach (option 3). If you only offer the modern approach this increases the pressure of other mail clients to follow the path, however you loose compatibility in the mean time. Unfortunately, none of the big players (MS Outlook, OS X Mail, Gmail web client, etc.) do support option 3, yet.

Option 3: The format=flowed-solution
[...]

I am not sure how this solution is much different (in the outcomes) from the existing situation. But I think the mail client should start

I do not get what you want to tell me here. Let me recapitulate: Both option 2 and 3 offer a solution how to mark certain line breaks as "soft". These are line breaks that are artificially inserted into the email body in order to be compliant to the 76 character limit but are not meant as an actual part of the message and the email viewer is allowed to re-flow the text. This is the benefit of it. Option 3 is the modern variant, plays nicely with 8-bit encoding and is already optionally supported by AquaMail. (See Settings -> e-mail, advanced -> flow text). The drawback of option 3 is that it is not widely supported. Option 2 also supports a re-flow mechanism, hence it offers the same benefit from the user's perspective, but at the expense of using the quoted-printable encoding.

Now, back to what I am talking about and I am really loosing my patience, because I feel as if people try to misunderstand me by purpose.

AquaMail already offers an option to enable the modern approach (option 3) using format=flowed (see Settings -> e-mail, advanced -> flow text). The result is a wonderful 8-bit encoded plain text e-mail body. This is nice, this is wonderful and it would be like living in paradise if every email client in the world (or at least the major ones) would support it. Then that is everything you would ever need. In that case you even could drop the check box at all and proceed as if it was always enabled. Unfortunately, this is not how the real world works. Hence, it would be nice if AquaMail would support option 2 as the fallback method.

	Current situation	Improvement
Checkbox "flow text" enabled	8-bit encoding with format=flowed	8-bit encoding with format=flowed
Checkbox "flow text" disabled	8-bit encoding, just omits the format option and forcefully inserts hard breaks into the e-mail body	quoted-printable, insert "soft line breaks" acc. to RFC 2045, sec. 6.7, (5)

[Kostya Vasilyev]

quoted-printable, insert "soft line breaks" acc. to RFC 2045, sec. 6.7, (5)

Oh I now see what you meant. Yes this would work.

These aren't really "soft line breaks" in the same sense as they are in format=flowed.

It's just a way to have a line break that's only present in the raw MIME message (what gets transmitted) but is not present at all in the message's content.

Base64 encoding would work too, but would be even less "parseable by a human".

I will enter this into the task tracker as an idea for future versions.

Still not sure why you're seeing double line breaks in another mail app

[StR]

Option 3: The format=flowed-solution
[...]

I am not sure how this solution is much different (in the outcomes) from the existing situation. But I think the mail client should start

I do not get what you want to tell me here. <...>
 Option 3 is the modern variant, plays nicely with 8-bit encoding and is already optionally supported by AquaMail.

That was the key information that was missing for me in your previous description.
I thought that was the case, but it sounded like all 3 options were talking about changes from the existing Aquamail functionality.

It may have been me, being slow, or your post was not perfectly clear. That's why I was asking you to clarify this.
So, there is no reason for loosing your calm: when people have problems understanding, questions are asked. You wouldn't want people to dismiss your suggestion just because they didn't quite understand it, would you?

Now, back to what I am talking about and I am really loosing my patience, because I feel as if people try to misunderstand me by purpose.

	Current situation	Improvement
Checkbox "flow text" enabled	8-bit encoding with format=flowed	8-bit encoding with format=flowed
Checkbox "flow text" disabled	8-bit encoding, just omits the format option and forcefully inserts hard breaks into the e-mail body	quoted-printable, insert "soft line breaks" acc. to RFC 2045, sec. 6.7, (5)

It sounds like Kostya has already understood your suggestion.
But let me just mention that the first row of this table doesn't quite make sense to me: "Current Situation" and "Improvement" contain the same text (How is that an "improvement"?). I am guessing, you are not suggesting any improvement in the first row, just showing what is happening (no changes) in case when that option ("flow text") is enabled. Right?

[Kostya Vasilyev]

I believe the suggestion was to do quoted-printable with "soft" line breaks *only if* "flowed text" is off (otherwise there'd have to be a whole new setting, ouch).

[nagmat84]

But let me just mention that the first row of this table doesn't quite make sense to me: "Current Situation" and "Improvement" contain the same text (How is that an "improvement"?). I am guessing, you are not suggesting any improvement in the first row, just showing what is happening (no changes) in case when that option ("flow text") is enabled. Right?

I believe the suggestion was to do quoted-printable with "soft" line breaks *only if* "flowed text" is off (otherwise there'd have to be a whole new setting, ouch).

Finally, yes! That what I mean. If the option format=flowed is enabled, everything is just fine. No need to change anything here. This is the modern variant of how "soft line breaks" are implemented. But if the option is disabled the fallback should be to use quoted printable so that soft line breaks still work and not just to omit "format=flow" from the content type.

By the way: Having two options does not make real sense. Combining both format=flowed together with quoted printable is pointless. You get the drawback of increased size due to an artificial encoding, loose the ability to easily grep in message body and gain nothing - probably worse you have two interfering sorts of soft line breaks. On the other hand the current implementation of the option "Use flowed text" is pointless, too. At the moment you should leave the option always on. Disabling the option has no benefit right now. Clients that do not understand the format=flowed ignore the setting anyway and show hard line breaks. At the moment, if the option is disabled you only loose the benefit that modern clients nicely reflow the text but you do not gain anything.

However, if the option would be used to toggle between  format=flowed with true 8-bit encoding and  quoted printable then the option would really offer a choice. Then the option offers a choice between a (A) modern variant of reflowing text with a compact representation (true 8-bit encoding) but with the drawback that some older clients to not understand the option and show text with hard line breaks or (B) the legacy variant with a bulky encoding but the benefit of broader support.

[Kostya Vasilyev]

Better late than never I suppose, if anyone's still watching:

https://www.aqua-mail.com/forum/index.php?topic=6249.0

HTML messages are now sent quoted-printable (better compat) always.

Plain text messages are now sent quoted-printable if app settings -> mail, other -> flowed text is off.