Lost Mail

[Davey126]

Not really ... but one might think so if only using the AquaMail client with a Gmail account. Gmail does some wonky things with conversations, labels and disambiguating messages that do not translate well to 'traditional' mail clients that depend on discrete mail messages, imprecise mappings and limited information from Gmail to determine what to do with each item. Gmail's unique features work well with Google properties but step into the 3rd party world and things get interesting. AquaMail does an AMAZING job handling most of the nuances...better than any other non-Google mail client I have used to date. But there are still opportunities for improvement.

The problem of 'lost mail' stems from the mapping of Gmail labels to folders and the implicit actions taken when items are relabeled/moved. At present AquaMail only supports a 'move' function between folders which can be disruptive to Gmail's use of multiple labels...including system labels such as 'important', 'sent', etc. When an item is moved to a different folder in AquaMail it sometimes looses other 'system' labels. Gmail restores a few key system labels when the messages are resynced to the server. Most are not.

A prime example occurs when moving an item from the 'sent' folder. Aquamail reliably moves the item to the target folder and removes it from the sent folder. However, this breaks Gmail's multi-label model and can lead to much head-scratching when looking for an item you previously sent which is no longer in the sent folder. Fortunately, AquaMail's search function usually comes to the rescue to locate the 'lost' item.

In an interesting twist the only way to reassociate an item to the 'sent' folder/view is by using AquaMail or a similar client. It can not be done from the Gmail web interface or Google apps.

I have proposed (twice) the inclusion of 'copy' in addition to 'move' within the AquaMail client. However, this has not been supported by the community (understand the rational) so I am angling for a more limited capability: do not strip any existing labels from a message when an item is moved to a new folder except, of course, the label associated with the source folder. This includes all system labels like 'sent', etc.

If implemented this change would be transparent from a UI perspective (no confusing choice between copy and move) but should lead to a more consistent experience for Gmail users. It still does not offer multi user label/folder capability but one can use native Google properties for that when needed.

[Davey126]

In Gmail when you relabel an item (message) it does not remove the other labels unless you untick them. Similarly, when you shuffle messages between folders it NEVER removes system assigned labels like 'sent' and 'important'.

If you use the AquaMail client exclusively one rarely has multiple labels assigned to a message. The item is either in one folder or another. In that context the move function behaves exactly as expected.

A small fraction of my mail items have multiple labels assigned by filters that get applied when a message arrives. Some are automatically filed and never see the inbox. In that context the move function in AquaMail also functions well. If an item carries labels A, B and C and I move it to folder D while viewing it from folder C it retains labels A, B and D. However, I must return to the Gmail client if I want to add label E without loosing A, B and D. AquaMail can't do this and the community has indicated no interest in this capability. Fair enough. 

An example of where AquaMail runs into trouble is when an item is moved to a different folder while viewing it from a system folder such as sent. It losses its 'sent' identity making it difficult to later find in any client. You know you sent a message to Joe but can't find it in the sent mail folder.

The change I am requesting is very limited: never remove system assigned labels such as 'sent' as those labels are assumed to be perpetually present in the Gmail model. This obviously has to be moderated by common sense. If I move an item out of the trash I certainly don't want it to retain that label.

Hope this helps.

[StR]

@Davey126:
I understand the rational of your suggestion.
I agree that a "copy" function could be useful. (And, I might be wrong, but it seems to me that Kostya also might think so, but it's just not a high-priority feature.)

However, I do not think that your suggestion to keep the message in the previous folder while moving (that is what it means in the IMAP folders language) is a good idea. Definitely, not by default.
Here is why.
I (and many other users) use mail clients that are not truly working with Gmail's labels (Thunderbird, Aquamail). When I move a message to a different folder, I expect that it is not left behind in the old one. What you are suggesting would break that behavior. Besides, if your suggestion were implemented, I'd end up with multiple copies of the same message in different folders of my IMAP clients.
You really want to have a separate option copy for that, so that you deliberately create multiple copies. Or, alternatively, a true implementation of Gmail's labels in the client.


In the past several months, I've been thinking about Gmail's "label" approach. And while I might be unhappy with some other "innovations" that Gmail pushes onto its users, I came to think that this one is actually a good approach. Unfortunately, majority of the main-stream IMAP servers and clients do not support that feature. So, it is hard to switch to the workflow that would be consistent between different providers.

[Davey126]

@StR - Thanks for your response. However, I believe you may be misinterpreting my request.

In brief:
- remove the the message from the previous folder (as happens today) when the previous folder is not a special/system folder
- do NOT remove the message when the previous folder is 'sent' or an handful of others that have special meaning to Gmail

The change I am requesting only deals with system folders. The default behavior remains as-is and is the action expected from a move operation between two standard folders.

One may argue that removing items from the 'sent' folder during a move operation is simply honoring the users wishes. However, this is not how it works in the browser version of Gmail or in the Gmail apps. AquaMail does not offer a choice as only action available is move in v1.6.1.4 and below.

Again, I am in no way suggesting that items remain in the previous folder when moved to another folder (that's what copy is for) except in special cases that are consistant with the way Gmail natively behaves.

[StR]

@Davey126:
I understand what you are asking for.

You feel that your suggestion is different because you see "Sent" folder (label) as a "system" one.
(The only other "special" ones are "Drafts" and "Deleted", and I don't think that even for you it would make sense to keep a messages in those after moving it elsewhere. What Gmail considers "System labels" (Starred, Important, Chats, even Spam ...) is only Gmail's notion: there are no such special folders from a general IMAP client's point of view.)
To me, it is one of the folders that is present in all my e-mail clients. Yes, it has an added functionality (sent messages get deposited into it), but I expect all other functionality of all folders to be the same.

For the people with whom I have regular e-mail communication, I have folders dedicated to them. So, I move messages from "Sent" to those folders. My "Sent" folder contains only those messages that haven't been moved to those other folders.
So, even for "Sent", when I am moving my messages from it, I want them to be moved, not duplicated.
(I am inconvenienced enough by Gmail that the way how it works with the sent messages breaks the plugin I am using in Thunderbird, "Copy Sent to Current": https://www.ggbs.de/extensions/#cs2c . But, fortunately, I don't use Gmail as much as other accounts at this point.)

I hope you can see how your suggestion would break my work flow and functionality on which I rely.
That's why I am saying that this "half measure" is not a good idea. Either one creates a true "copy" action (or supports full label functionality), or doesn't do this intermediate "kludge".

[Davey126]

@Davey126:
I understand what you are asking for.

You feel that your suggestion is different because you see "Sent" folder (label) as a "system" one.
(The only other "special" ones are "Drafts" and "Deleted", and I don't think that even for you it would make sense to keep a messages in those after moving it elsewhere. What Gmail considers "System labels" (Starred, Important, Chats, even Spam ...) is only Gmail's notion: there are no such special folders from a general IMAP client's point of view.)
To me, it is one of the folders that is present in all my e-mail clients. Yes, it has an added functionality (sent messages get deposited into it), but I expect all other functionality of all folders to be the same.

For the people with whom I have regular e-mail communication, I have folders dedicated to them. So, I move messages from "Sent" to those folders. My "Sent" folder contains only those messages that haven't been moved to those other folders.
So, even for "Sent", when I am moving my messages from it, I want them to be moved, not duplicated.
(I am inconvenienced enough by Gmail that the way how it works with the sent messages breaks
https://www.ggbs.de/extensions/#cs2c

I hope you can see how your suggestion would break my work flow and functionality on which I rely.
That's why I am saying that this "half measure" is not a good idea. Either one creates a true "copy" action (or supports full label functionality), or doesn't do this intermediate "kludge".

Interesting. So you have developed a workflow based on the way your mail system operates which is different than Gmail, Outlook and the other big providers. That does not invalidate what you are doing (not at all!) but it does present a compatibility problem where any solution breaks someone elses work flow.

Fact remains: the AquaMail move function as currently implemented can break Gmail's model. How serious is that? IDK. For me I understand what has happened and can deal with the consequences. Annoying but not a show stopper. As you are well aware most users are not familiar with the inner workings of Gmail, IMAP, POP and mail clients in general. They just want things to work and raise big red flags when messages appear to go missing.

I certainly respect your position - and I hope you appreciate mine. Discussion is always good but I think it has run its course. As with other such requests probably best for Kostya to weigh in. I suspect things will remain 'as-is' as there is not a large clamoring for either 'copy' or alignment with Gmails presumptuous use of common word labels (like 'sent').

Thanks again @StR and @Paris Geek. Appreciate the thoughtful responses.

[StR]

Davey126,

First of all, - don't worry, I am enjoying this intelligent (I hope) discussion too.

Interesting. So you have developed a workflow based on the way your mail system operates which is different than Gmail, Outlook and the other big providers.

That statement might benefit from the clarification: My workflow developed based on the way (many, if not most) e-mail clients and servers (based, in turn, on the RFC's defining IMAP standards) have been operating before these "big providers" have appeared.
(So far, I've been spared from using Lotus Notes and Microsoft's e-mail programs and protocols, even though the threat from MS e-mail products came very close a few times.) And I could speak SMTP fluently long before I could combine two words in Spanish together.

Note, that many of those e-mail clients are open-source software, based on open standards, as opposed to proprietary innovations and kludges from Google and Microsoft that are mostly incompatible with the software outside of their ecosystem.
And, I believe, it is done on purpose, - to keep people tied up to that ecosystem.
While it is not impossible that some proprietary systems can become prevailing (e.g. FB messaging system became e-mail replacement for some people), but those also come and go. (How many people are using ICQ these days? And at some point it seemed to be THE messaging platform.)
Surprisingly, IMAP, or rather IMAP4 has been in existence for over 20 years (since ~1992), and considering its predecessor, IMAP2 (1988), - 27-28 years.

There have been various attempts to make IMAP servers more database-based and oriented (which is what Gmail essentially is). Unfortunately, none of them became a de-facto standard.

But clearly, the original IMAP4's (or even IMAP4rev1) capabilities are limited. That's why there are various extensions for mobile devices proposed, such as IMAP IDLE.
Maybe, RFC 6154 that Gmail's model relies on for the implementation of its labels will become more widely accepted and implemented, and maybe that would include Gmail's use of labels.
[/quote]

I would like to see a Copy action in Aquamail, but I have understood that Kostya is busy with something else right now.

I agree with both Paris Geek's assertions.


PS. I've just learned that Thunderbird has implemented some level of working with labels that allows avoiding downloading duplicated messages with multiple labels. Well, maybe my view about support of Gmail's labels is also somewhat outdated... but from what I can tell, it's still mapping of labels onto folders. Not using labels per se.
I am reading that Sparrow and Postbox are using Gmail's labels more properly.

[Davey126]

@StR -  Great post. I may shamelessly steal your SMTP quip as it could also describe my IT journey - although I probably did not delve as deeply into the gory protocol details when managing a handful of mail servers. Spent a fair bit of time pouring over RFCs as the company I worked for implemented a fairly extensive Lotus Notes infrastructure (thousands of seats) that smoothly interfaced with exactly nothing. Trying to figure out who was non-conforming and own up to/fix the problem was a constant battle.

Like it or not Google (possibly Microsoft) "interpretation" of RFCs will become the defacto standard as smaller players find it economically infeasible to run a private service. Think you are seeing some of that today as mid-level providers adapt to user expectations (and advertiser demands) established by the big boys. Ditto for business. Who wants to run their own mail service unless it's an extension of your core business.

[Kostya Vasilyev]

The change I am requesting is very limited: never remove system assigned labels such as 'sent' as those labels are assumed to be perpetually present in the Gmail model. This obviously has to be moderated by common sense. If I move an item out of the trash I certainly don't want it to retain that label.

But it doesn't. It executes a "move" operation from source folder / label to destination folder / label, and isn't even aware of the other folders / labels.

Seems like a Gmail server "glitch" or "um, it just sort of came out that way".

Gmail does provide access to labels as an extension, but Aqua currently doesn't use this, and as you already know full well, doesn't let the user manage flags (and also isn't aware of "this message in folder A and this other message in folder B is really one and the same").

[Davey126]

So it would seem the condition (trying not to say problem) is rooted in the ability of AquaMail to perform actions on emails that are disallowed in the native Gmail apps/browser. Since Gmail never exposes the 'sent' label one can't remove it using Google's tools. AquaMail (and probably most other mail clients that leverage label -> folder mapping) permit items to be removed from the sent folder which implicitly removes the sent label (from GMail's perspective) and prevents the item from being properly cataloged as 'sent'.

More Google weirdness.

I think the original request retains merit but I certainly understand the reluctance to restrict moves from the sent folder (which would break some workflows ask noted by @StR) and/or create more Gmail specific exceptions which no one is a fan of.

Adding a copy function would offer a non-Gmail specific solution while not restricting existing capabilities - but we've already had that discussion.

[Kostya Vasilyev]

From my perspective, it's a standard IMAP MOVE command, to move a message from one folder to another.

Yes, it seems like some sort of weirdness that Gmail has, but -- I'm sure their response (if I ever got one) would be "just implement proper support for Gmail labels, we do provide a way", and they'd be right...