Author Topic: How to initially download ALL emails from imap server?  (Read 9961 times)

dvhttn

  • Guest
How to initially download ALL emails from imap server?
« on: February 12, 2017, 10:21:11 pm »
Hi. First post so maybe this has been answered somewhere but not found anything 'searching' (though maybe used the wrong search terms ...). I want to be able to download, initially, ALL my emails found on my imap server. I have many saved that could be useful information for me - they were saved so that this is information that I then don't have to go looking for again ! I have no problem with (phones) internal storage and I have no problem with any data connection, wifi or 4G, to do the initial download(s). What I do have is a geographic location where phone service is a little *spotty* to say the least, but being able to search all my email would be very useful. I also don't expect to have these folders sync regularly - just 'manually' occasionally.

Is there a way to do this? I have seen comments from the developer to the effect that 'desktop PCs do things differently' (i.e. Outlook, for example, does download everything) so expect the answer is 'no' but  am hopeful.

Thanks in advance

Dave

StR

  • Hero Member
  • *****
  • Posts: 1558
Re: How to initially download ALL emails from imap server?
« Reply #1 on: February 13, 2017, 05:05:45 pm »
A mobile IMAP client is really a "window" into what is on your server. I am not aware of any Android IMAP e-mail client that stores all messages on the phone. (It doesn't mean there aren't any, even though I doubt) You have a certain number of messages stored locally, and then you can pull older messages "on demand".
Desktop e-mail apps are designed to work differently (But on my laptop, for some accounts, I have an essentially similar setup: only recent messages are downloaded, while only the headers for the old ones are stored on the laptop.)

I understand your situation with the spotty connectivity, - but if you really need offline access to your e-mail, the correct solution for your is a laptop with a program like Thunderbird. You can get a simple and relatively cheap one. And if you need it to access new e-mail messages on mobile networks, you can tether it via the phone, if you have a sufficient data plan. (Some providers on some small plans do not allow tethering, but there are workarounds for that as well, e.g. Fox-Fi that is now PdaNet+.)

While your phone can have large built-in memory, you will start seeing problems when the size of the database becomes too large (beyond certain size). You'll have slow operation while opening messages, searching, etc.
There is no magic there...

dvhttn

  • Guest
Re: How to initially download ALL emails from imap server?
« Reply #2 on: February 13, 2017, 05:39:49 pm »
....
I understand your situation with the spotty connectivity, - but if you really need offline access to your e-mail, the correct solution for your is a laptop with a program like Thunderbird. You can get a simple and relatively cheap one. And if you need it to access new e-mail messages on mobile networks, you can tether it via the phone, if you have a sufficient data plan. (Some providers on some small plans do not allow tethering, but there are workarounds for that as well, e.g. Fox-Fi that is now PdaNet+.)

While your phone can have large built-in memory, you will start seeing problems when the size of the database becomes too large (beyond certain size). You'll have slow operation while opening messages, searching, etc. ...

No. For what I want, that is when I'm out and maybe need some information that is in my emails, a laptop is NOT the solution. I do not, and would not want to, carry a laptop everywhere. I do have my phone though, hence the request. As for ther database getting slow, why should it if a) it's designed correctly and b) is on reasonably fast memory?

Dave

StR

  • Hero Member
  • *****
  • Posts: 1558
Re: How to initially download ALL emails from imap server?
« Reply #3 on: February 13, 2017, 07:42:43 pm »
No. For what I want, that is when I'm out and maybe need some information that is in my emails, a laptop is NOT the solution. I do not, and would not want to, carry a laptop everywhere. I do have my phone though, hence the request.

The laptop was a suggestion if you need reliable offline access.

If you have a phone and have connectivity, if you need to access your old messages, you can do that with IMAP without downloading all messages to your phone.
I think that the most efficient way would be probably to do a search on the server for the message, and then opening it. But you can always scroll your messages down until you get to the one you are looking for. (Aquamail will download additional messages when you reach the bottom of the list of the downloaded messages.)

In principle, you can configure your account as "POP3", and then Aquamail will download all your messages. But I wouldn't recommend that way, especially if you are accessing your account from multiple devices. There are several different "complications" that you might encounter (e.g. depending on the mail provider, your messages might be deleted from the server upon you downloading them via POP3, so, should something happen to your phone, you won't have access to those messages in the future).

Quote
As for ther database getting slow, why should it if a) it's designed correctly and b) is on reasonably fast memory?

As far as I understand, there could be more than one reason.
1. It is in the way how the messages are stored.
On a desktop system, the messages are stored as separate files (if they are stored locally) - (one per message or one per folder, that doesn't matter). On Android (at least in Aquamail) the messages are stored in a database. Many databases (at least unless you go to specialized ones available on large servers) are not easily scalable: when they become large (beyond certain threshold) the operations become slow.

2. Limited resources available for an app on Android devices.
Even on the most "powerful" phones, you still have resources/specs inferior to a reasonable modern laptop.
Even if you have a large size of storage on your device, you still have limited amount of RAM available to your app (Android and other apps are using some as well, and then, Android might also limit how much a single app can be allocated -- I am not sure about this aspect, but it doesn't matter much for this question).
E.g. the "mighty" Samsung Note 7 had only 4 GB of RAM, while, say Samsung Galaxy 5 has only2 GB. Modern laptops and desktops start from 4GB, and can go way up. Besides, Windows and MacOS can swap the memory out (to the HDD). The memory management on Android is different, - but then, you are dealing with the speed of your flash memory (which is also limited), which is much slower than RAM.

The bottom line is that operating on a large-size index takes toll on mobile devices. And when that happens, you would be the first to complain that the app puts the phone on its knees.
So, the developer (and as I said, - that's very similar with all Android e-mail clients that I know), - made the correct decision to limit on how many messages could be stored locally.

Think this way: You can calculate the volume of you sedan car's trunk and cabin, and you figure out that it's large enough compared to the volume of a full-size bed. But you still cannot expect a sedan car to have a full-size bed in it, like you can have in an RV vehicle. You can do some modifications and maybe even physically fit that bed, but it will be awkward, and you cannot demand that from the car manufacturer, right?
So if you want to always have a comfortable bed on a road, you have to drive an RV (or one of those WV vans). If you insist on driving a sedan, you'd  have to use a hotel or camp in a tent.

dvhttn

  • Guest
Re: How to initially download ALL emails from imap server?
« Reply #4 on: February 13, 2017, 11:13:22 pm »
Thanks for all the "possible" reasons app developers may not do this but .... you haven't actually answered my original question with a straight 'yes' or 'no, which was all I wanted. I'm assuming from your answers and my own checking out of the app that the answer is 'no. You haven't given me one good reason why though, apart from the fact the the developer hasn't done it yet, for whatever reason.

I myself have been in IT for 30 years+ so am familiar with things from BBC Micros, VAX and VMS up to todays Intel CPUs and Smart Phones (I actually worked for Intel for 3 years) and  there is no reason why todays phones can't handle the download/storage/compute for a basic job such as email. Back in the day a 486 laptop was perfectly adequate and many of todays phone storage devices are on a par with or exceed the disk capabilities from back in the day. So we'll agree to disagree on that score.

As the answer is 'no' I'll look elsewhere then .....

Dave

mikeone

  • Hero Member
  • *****
  • Posts: 2762
Re: How to initially download ALL emails from imap server?
« Reply #5 on: February 13, 2017, 11:33:05 pm »
Thanks for all the "possible" reasons app developers may not do this but .... you haven't actually answered my original question with a straight 'yes' or 'no, which was all I wanted. I'm assuming from your answers and my own checking out of the app that the answer is 'no. You haven't given me one good reason why though, apart from the fact the the developer hasn't done it yet, for whatever reason.
...
As the answer is 'no' I'll look elsewhere then .....

Dave
Well, there is no "yes" or "no", because it depends on the amount of messages that are stored on your mail server.

However, there are options available in the settings:

-> Mail, receiving -> Messages to cache (please be aware of the term "to cache" vs. "to store" 😊) -> increase the value according to your requirements (up to the maximum of 10,000).

Possibly this setting could fit your needs.
« Last Edit: February 14, 2017, 12:14:28 am by mikeone »

dvhttn

  • Guest
Re: How to initially download ALL emails from imap server?
« Reply #6 on: February 14, 2017, 01:23:19 am »

Well, there is no "yes" or "no", because it depends on the amount of messages that are stored on your mail server.

However, there are options available in the settings:

-> Mail, receiving -> Messages to cache (please be aware of the term "to cache" vs. "to store" 😊) -> increase the value according to your requirements (up to the maximum of 10,000).


Nope. Seems to want to download only qhen you choose that folder. I want to download everything and store it locally.

Thanks anyway.

StR

  • Hero Member
  • *****
  • Posts: 1558
Re: How to initially download ALL emails from imap server?
« Reply #7 on: February 14, 2017, 06:34:32 pm »
Thanks for all the "possible" reasons app developers may not do this but .... you haven't actually answered my original question with a straight 'yes' or 'no, which was all I wanted.

Dave,
You expect a definitive answer, but you have not given any information about the volume of your e-mails: number of messages and total volume/storage used. If you have worked in IT for that long, you should realize that those factors matter.
If you have 5 messages stored, then the answer: "yes you can".
If the number of your messages and their size are the same as they were 20 years ago (text-only, no attachments), then the answer: "yes you can".
If it is more, then the answer is "it depends".

I have an account with messages from the last 16 years. It has about 13 GB of messages. Attempting to download all of them onto my phone  (even if it were possible) would likely cripple it.
(And if it is not yet explicit enough, in that case the answer is "no".)


I'm assuming from your answers and my own checking out of the app that the answer is 'no. You haven't given me one good reason why though, apart from the fact the the developer hasn't done it yet, for whatever reason.

Excuse me, Your IT Highness that my peasant responses were not on par with Your IT Highness' requirements.
With your unmatchable pedigree, I suspect that the only reason Your IT Highness hasn't written the desired app yourself while awaiting for a response here, is Your IT Highness' were too kind determination to avoid intimidating us, savages, by Your IT Highness' might .

And in the mean time, maybe Your IT Highness could ask Google, why their GMail app cannot download 15 GB worth of messages to a mobile phone. Please, share with us the reasons they'll give you ("apart from the fact  [that that pitiful bunch of developers] hasn't done it yet").


Disclaimer: I am not related to MobiSystems or the chief Aquamail developer Kostya, and do not represent them. I express my own opinion while trying to help other Aquamail users learning this great e-mail app.

Kostya Vasilyev

  • Hero Member
  • *****
  • Posts: 12740
Re: How to initially download ALL emails from imap server?
« Reply #8 on: February 14, 2017, 11:12:10 pm »
Actually you will want to increase both "messages to sync" and "messages to cache" to the max values -- and then refresh the account.

Please note that:

- Aqua Mail, being a mobile app, downloads the text and attachments separately from each other.

To have offline access, you'll want to get those by long pressing on the account -> options and folders -> preload.

- Aqua Mail, being a mobile app, downloads the text of each message only up to a certain limit (and the rest when you open a message).

App settings -> network -> "load when checking mail", this is for text portions specifically.

- Aqua Mail, being a mobile app, will only show you those message it's "sure" about vs. the server.

The "messages to sync" setting will make it "sync" more messages than it usually does, but on the flip side, it will be asking the server for the state and existence of those messages every time it checks mail (traffic).

---

Q:

Is is possible, in 2017, to write a mobile app for mobile devices that's not optimized for the typical 2017 "always connected" mobile user?

A:

Yes it is possible, but that's not what Aqua Mail was ever intended to be.
Creating debug logs for diagnostics: https://www.aqua-mail.com/troubleshooting/

The official FAQ: https://www.aqua-mail.com/faq/

Лог-файлы для диагностики: https://www.aqua-mail.com/ru/troubleshooting/

Вопросы и ответы: https://www.aqua-mail.com/ru/faq/

dvhttn

  • Guest
Re: How to initially download ALL emails from imap server?
« Reply #9 on: March 05, 2017, 07:22:55 pm »
Sorry for the late reply but thanks Kostya for the explanation. Seems AM doesn't do what I'd like an email client to do.  Hey, each to their own .....

@StR: not sure what's got you so riled, it's just that we have differing ideas of what we'd like an email client to do and mine don't agree with your's. Get over yourself ........