Gmail IMAP Implementation

[lexluthor]

From what I understand, Gmail's IMAP implementation is somewhat buggy.

Let's say I get a new email in my inbox.  If I mark it as read via gmail web interface, it never gets marked as read automatically in Aquamail (or K9 for that matter).  If I delete the email out of my inbox via gmail web interface, it does get deleted from the Aquamail inbox.

Basically, message read state does not get passed properly via IMAP idle when using gmail.

Can you comment on this?  Any tricks you might be able to think of to get this working correctly?  I could do poll at some interval in addition to push and when it polls, it should get the read state, but at the interval that I'd have to set polling to to make it useful for this purpose, it'd use up more battery than I'd be willing to do.

By the way, that's one of the reasons that it'd be great if you ever got exchange activesync working.  Setting gmail up that way does pass read state properly.  Unfortunately for me, no client that I've found that does exchange activesync works the way I need it to (mainly the need to allow me to have my from and reply to addresses NOT be my gmail address).

[lexluthor]

Just making sure this one didn't get lost in all the spam.

[Kostya Vasilyev]

Gmail's IMAP server does two things differently (and I'm only talking about IMAP Push here...)

1 - It does not send message flag changes, although it's *required* by the spec to do this

2 - It "tickles" the app every 5 minutes, even if there were no changes

Because of the above, I have to perform a full "check 25 or whatever most recent messages" on those 5 minute tickles just to get the flags.

Was it an oversight on Google's part, or an intentional way to steer users towards the native Gmail app?

[lexluthor]

Ok. So using this, i could see the read flag updated anytime between immediate and 5 minutes after the message is marked as a elsewhere?

[Kostya Vasilyev]

Most of the time, yes:

-- unless there are new messages during that time, in which case the app tries to optimize things a bit and only downloads the new messages.

Performing a full (N recent messages) every time there is a new message would be too expensive.

At the latest, you should be seeing the flags update after the IMAP Idle connection expires and is refreshed, max. 25 minutes, or 12 minutes 30 seconds on the average.

[ggggg]

I am using Llama to switch data and flight mode on my SM-T705, 4.4.2. During the day data is on every 20 minutes for 1 minute or when the user is active (display on) data stays on.

1 I have one gmail account, IMAP but no push (up till now). How should I set up sync options, to reduce wake ups / power consumption ?
Maybe event option mobile, Intervall 18/20, delay 10s ? How is this setting combined with the intervall setting further up on the same page ? It is 15Min.

2. What is less consuming: System settings: Watch or sync even if AutoSync disabled. I have watch right now.

[Kostya Vasilyev]

If you set Aqua to sync when auto-sync is disabled, but there is no network connection, then that won't work (no network).

If this battery optimizer enables auto-sync, then I'd set Aqua to trigger on that and disable scheduled mail sync -- hopefully, auto-sync will be enabled only once the network connection is fully available.

[ggggg]

LLama doesnt change auto-sync, but mobile-data on/off. (auto-sync stays enabled all the time)

What about:
System settings: watch auto-sync
Poll Intervall: 15min
Start sync on: event option mobile, shortest intervall 18min, Delay 10s

[Kostya Vasilyev]

Yes, that should work -- unless your mobile data takes more than 10 seconds to "get ready", and then you'll want to increase the "settle in delay".

[ggggg]

OK, is there any log, where I can see when last sync was and if it succeeded ?

[Kostya Vasilyev]

The app's main window has "last sync time" under each account.

[ggggg]

Problems with long wake locks ... please analyse attached log (every time one email was polled)

sync with 2s lock:

Code: [Select]

15:02:07.726 +0200 LockManager Releasing all locks
15:02:07.727 +0200 LockManager Releasing the Wake lock, time held = 2.41 sec
15:02:07.730 +0200 LockManager After release: Wake lock isHeld = false
15:02:07.731 +0200 LockManager WiFi lock is not held
15:02:07.732 +0200 StartSyncService All locks have been released

sync with 13s lock:

Code: [Select]

15:14:47.458 +0200 UIModeTracker onScreenOff
15:14:47.460 +0200 UIModeTracker Notifying listeners
15:17:51.414 +0200 LostConnectionReceiver {}{}{}{}{} onReceive ConnectivityManager.CONNECTIVITY_ACTION: ni = NetworkInfo: type: mobile[HSPA+], state: CONNECTED/CONNECTED, reason: connected, roaming: false, failover: false, isAvailable: true, isConnectedToProvisioningNetwork: false,  isConn = true, isConnOrConn = true
15:17:51.446 +0200 LostConnectionReceiver {}{}{}{}{} Currently active network info: NetworkInfo: type: mobile[HSPA+], state: CONNECTED/CONNECTED, reason: connected, roaming: false, failover: false, isAvailable: true, isConnectedToProvisioningNetwork: false
15:17:51.449 +0200 LostConnectionReceiver {}{}{}{}{} onConnectionEstablished: curr -1, new 0
15:17:51.453 +0200 LostConnectionReceiver {}{}{}{}{} onConnectionEstablished: new connection type is 0
15:17:53.191 +0200 KickSyncReceiver onReceive, Intent = Intent { act=android.net.conn.CONNECTIVITY_CHANGE flg=0x4000010 cmp=org.kman.AquaMail/.core.KickSyncReceiver (has extras) }
15:17:53.205 +0200 KickSyncReceiver Connectivity change: NetworkInfo: type: mobile[HSPA+], state: CONNECTED/CONNECTED, reason: connected, roaming: false, failover: false, isAvailable: true, isConnectedToProvisioningNetwork: false
15:17:53.209 +0200 KickSyncReceiver Last kick sync event time: 15:01:43
15:17:53.264 +0200 KickSyncReceiver Will kick off sync in 15 seconds
15:17:53.268 +0200 AlarmCompat_api19 setWindow: 0, 1440595088208 (2015-08-26 15:18:08) 5000, PendingIntent{434bfb90: android.os.BinderProxy@4311b060}
15:18:09.244 +0200 KickSyncReceiver onReceive, Intent = Intent { act=org.kman.AquaMail.KICK_SYNC flg=0x14 cmp=org.kman.AquaMail/.core.KickSyncReceiver (has extras) }
15:18:09.332 +0200 KickSyncReceiver Kicking off sync
15:18:09.334 +0200 [POWER.1] Acquired special flag 0x00000001, result 0x00000001
15:18:09.338 +0200 LockManager Acquiring the Wake lock
....
15:18:22.658 +0200 LockManager Releasing the Wake lock, time held = 13.31 sec

the last one has 31s ... without receiving any new email !!!!

Code: [Select]

2015.08.26 18:02:49.351 +0200 LockManager Releasing the Wake lock, time held = 31.67 sec


[ggggg]

Open questions:

As said Llama enables data only once an hour for 1 minute or when the screen is on ... I am syncing 1 gmail account and I have one alias (via gmx) for sending like in gmail.

0) what about wake lock logs from post above ?

1) Under androids account settings I had one Aquamail account (gmail, google apps). After changing the language setting it said appliing settings and now I have a 2nd account for IMAP,POP3. I double checked ... no IMAP in Aqua general settings, no IMAP in Aqua account setting and no IMAP in accounts folder settings !

2) I cant see any difference whether it is set the Aqua gmail account to sync or not (it still syncs if unchecked). In Aqua the sync is set to "observe all" and I have one sync event for mobile connections.

3) What happens if data is enabled (delay=0) and data not yet stable ... does aqua wake lock and for how long does it try ?

4) It looks if one sync event fails (time written in red) Aqua doesnt do any further sync ? How can I overcome that ?

5) When does it sync if enable the sync event "Auto-Sync setting"

6) What is the best setting to have no wake locks and minimum battery drain (please keep my data strategy in mind)

 

[Kostya Vasilyev]

Re: the last one has 31s ... without receiving any new email !!!!

Nothing unusual, you should be able to follow the log yourself.

Time to establish new network connection to imap.gmail.com -- 3 seconds

CAPABILITY command -- almost 3 seconds

SELECT INBOX -- 2 seconds

Changing a message's flags (STORE command) -- 2 seconds

And these things do add up.

And then the network connection breaks off, and a bunch of apps are woken / started to process this event (you won't see it in the AquaMail log, but that's what happens on connectivity changes).

One thing you can try is Menu -> Compact the database in Aqua's main window, think I'm seeing some database slowness too in a couple of places.

[Kostya Vasilyev]

1 - "AquaMail (IMAP, POP3)" is a category for both IMAP and POP3 accounts.

2 - Right, AquaMail doesn't use the Android "Sync Adapter" framework to do its work.

3 - The attempt to check mail without a network connection won't work and will have network errors

4 - It will check mail next time based on the mail check schedule (by default, 15 minutes).

5 - Toggling auto-sync, the actual setting name and location varies by Android version, but in terms of code, it's this:

   com.android.sync.SYNC_CONN_STATUS_CHANGED

6 - NO wake locks? Uninstall all mail apps, all non-mail apps, power down your phone, and you'll have NO wake locks.