Author Topic: Version 1.14.0-750-dev - "work in progress", not in Google Play  (Read 8638 times)

Kostya Vasilyev

  • Hero Member
  • *****
  • Posts: 12740
Version 1.14.0-750-dev - "work in progress", not in Google Play
« on: January 16, 2018, 07:57:02 pm »
https://www.aqua-mail.com/download/AquaMail-market-1.14.0-747-dev-22e5464249df.apk

https://www.aqua-mail.com/download/AquaMail-market-1.14.0-750-dev-95ece5523b20.apk

---

+ Fixed issue with Gmail OAUTH2, sometimes not able to add account which is not present in phone Settings

+ Build 750: OAUTH2 sign-in for Office 365 now working, including two-factor (MFA, code over SMS). Big thanks to @swreg.

---

+ Исправление для Gmail OAUTH2, иногда не получалось добавить учётку которой нет в Настройках телефона

+ Сборка 750: OAUTH2 авторизация для Office 365 заработала, в том числе двух-факторная (SMS коды). С благодарностью @swreg.

 
« Last Edit: February 02, 2018, 06:48:34 pm by Kostya Vasilyev, Aqua Mail »
Creating debug logs for diagnostics: https://www.aqua-mail.com/troubleshooting/

The official FAQ: https://www.aqua-mail.com/faq/

Лог-файлы для диагностики: https://www.aqua-mail.com/ru/troubleshooting/

Вопросы и ответы: https://www.aqua-mail.com/ru/faq/

swregs

  • Newbie
  • *
  • Posts: 7
Re: Version 1.14.0-747-dev - "work in progress", not in Google Play
« Reply #1 on: January 18, 2018, 03:17:16 am »
Hello,

I have installed this dev version (also Pro enabled) to try the Office 365 OAUTH2 authentication because I need MFA support.  I have tried it against two different Office 365 tenants (one with and one without MFA required), but both fail after putting in e-mail address and password to the Microsoft generated "new experience" login web pages.  In both cases the error page returned says:

User account 'myname@company.com' from identity provider https : // sts.windows.net/GUID_of_my_company_AAD_tenant/ does not exist in tenant 'Aqua Mail' and cannot access the application 906be9aa-2843-47e6-a01d-ab9361ca7009 in that tenant.  The account needs to be added as an external user in the tenant first.  Sign out and sign in again with a different Azure Active Directory user account.

I guess it probably works for you since you have Aqua Mail as a registered app in your 'Aqua Mail' tenant/AAD portal.

I searched in the AAD enterprise app gallery for Aqua Mail to add it as an authorized app in my tenant, but did not find a match.

Am I missing some piece of the puzzle?  Of course I understand you emphasized it was "mostly" working and has not been formally released yet.

Just excited to see MFA with Office 365 is near.

Regards,

swregs

Kostya Vasilyev

  • Hero Member
  • *****
  • Posts: 12740
Re: Version 1.14.0-747-dev - "work in progress", not in Google Play
« Reply #2 on: January 18, 2018, 01:37:22 pm »
Aaaargh. MS never makes anything easy :)

Yes you're right, I was testing with an account that's owned by same "tenant".

1 - I will ask our QA to try logging in from a different tenant (they have a separate subscription).

2 - Found this:

https://blog.mastykarz.nl/configuring-multi-tenant-authentication-azure-app-service-authentication-options/

It's obvious that you know more about "Azure Directories" and "tenants" and all that than me -- could you possibly take a look at that link and tell me if it makes sense?

Creating debug logs for diagnostics: https://www.aqua-mail.com/troubleshooting/

The official FAQ: https://www.aqua-mail.com/faq/

Лог-файлы для диагностики: https://www.aqua-mail.com/ru/troubleshooting/

Вопросы и ответы: https://www.aqua-mail.com/ru/faq/

Kostya Vasilyev

  • Hero Member
  • *****
  • Posts: 12740
Re: Version 1.14.0-747-dev - "work in progress", not in Google Play
« Reply #3 on: January 18, 2018, 02:02:02 pm »
LOL - I went to portal.azure.com to see that "allow multi-tenant" setting - and Aqua Mail app registration isn't even showing!

It was a few days ago. The rest of the portal seems to be operational as usual.
Creating debug logs for diagnostics: https://www.aqua-mail.com/troubleshooting/

The official FAQ: https://www.aqua-mail.com/faq/

Лог-файлы для диагностики: https://www.aqua-mail.com/ru/troubleshooting/

Вопросы и ответы: https://www.aqua-mail.com/ru/faq/

Kostya Vasilyev

  • Hero Member
  • *****
  • Posts: 12740
Re: Version 1.14.0-747-dev - "work in progress", not in Google Play
« Reply #4 on: January 18, 2018, 08:43:34 pm »
@swregs

Could you please try this build?

https://www.aqua-mail.com/download/AquaMail-market-1.14.0-749-dev-3ee56a3afaa9.apk

The app's registration was already "multi-tenant", I added "prompt=consent" on the approval URL per the article I linked above. Let's see if this resolves the issue.

Creating debug logs for diagnostics: https://www.aqua-mail.com/troubleshooting/

The official FAQ: https://www.aqua-mail.com/faq/

Лог-файлы для диагностики: https://www.aqua-mail.com/ru/troubleshooting/

Вопросы и ответы: https://www.aqua-mail.com/ru/faq/

swregs

  • Newbie
  • *
  • Posts: 7
Re: Version 1.14.0-747-dev - "work in progress", not in Google Play
« Reply #5 on: January 18, 2018, 08:51:18 pm »
I believe we will have to learn on this together.

I am using this URL to get to the app registration blade for my tenant.
https : //portal.azure.com/#blade/Microsoft_AAD_IAM/ActiveDirectoryMenuBlade/RegisteredApps
Of course I do not see Aqua Mail since it is an app you registered in your tenant.

For my tenant, in the Office 365 Admin Center under the Services and add-ins settings I have "Integrated Apps" turned on, which supposedly allows end-user accounts in my tenant to give 3rd party apps access to their data.  This may only apply to website based services with SSO.
https : //support.office.com/en-us/article/turning-integrated-apps-on-or-off-7e453a40-66df-44ab-92a1-96786cb7fb34

The link you posted seems oriented to the web app side, not mobile client side.  If I understand Aqua Mail properly, it interacts directly with the Outlook/Exchange Online REST APIs without using an intermediate web app.

According to this https : //docs.microsoft.com/en-us/azure/active-directory/develop/active-directory-dev-glossary#multi-tenant-application
if indeed you created your app registration as a "native" type app (not "web app / API"), then it should have defaulted to multi-tenant enabled.  I believe "native" is the correct type to use for installed mobile apps.  I just created a AquaTest "native" app registration and the properties don't even give a multi-tenant choice. (Picture attached) If I look at some of my existing "web app / API" app registrations, they do have a selection to switch the tenancy mode.  It looks like you would have to delete an existing registration to remake it as "native" type registration where you add in the requested permissions to the Office 365 Exchange Online APIs and then use the new Application ID (client_id) and Redirect URI in the app build.

Supposedly upon first login from the app, it will authenticate and then do the one-time consent prompting if I allow the app to use the permissions it needs to interact with Exchange Online APIs using my account.

It also seems confusing because it appears there are two different portals where apps can be registered.  The older way is in Azure portal which only does app auth against AAD tenants.  They seem to be pushing to use the newer "Microsoft Application Registration portal" https : //apps.dev.microsoft.com as place to register apps which are able use either Microsoft account or AAD accounts as well as converged Graph APIs to most all MS services.

Was typing this as you replied.  Will try that new build in a few minutes and report back.

Thanks,

swregs

Kostya Vasilyev

  • Hero Member
  • *****
  • Posts: 12740
Re: Version 1.14.0-747-dev - "work in progress", not in Google Play
« Reply #6 on: January 18, 2018, 09:02:08 pm »
@swregs

Appreciate it, since in "MS land" I'm a stranger :)

- Aqua Mail is not cloud based, so there is no "intermediate web app"

It's only the app on the device connecting directly to a mail server.

- It is registered on Azure Portal as a "native" app...

... which are always "multi-tenant enabled" and this cannot be changed

- As a test, I also created a web based app...

... and there was a switch for multi-tenant on/off (sanity check)

- The new 2.0 APIs they're pushing do not support EWS (Exchange Web Services) or IMAP.

There is an all new, REST / JSON based, Microsoft-specific, Mail / Calendar / Contacts API that can be used to access Office 365 or Hotmail (Outlook.com) accounts the same way (just like EWS, ironically).

---

And so I believe the only thing we were missing was the addition of "prompt=consent" on the approval URL. I added that in build 749 above.

---

Quote
I have "Integrated Apps" turned on, which supposedly allows end-user accounts in my tenant to give 3rd party apps access to their data.

Is this (assuming it's relevant) enabled by default, do you happen to know?

I'm wondering if new users would be able to sign into their Office 365 accounts without having to ask their admins to enable this setting first... If you happen to know...
« Last Edit: January 18, 2018, 09:11:16 pm by Kostya Vasilyev, Aqua Mail »
Creating debug logs for diagnostics: https://www.aqua-mail.com/troubleshooting/

The official FAQ: https://www.aqua-mail.com/faq/

Лог-файлы для диагностики: https://www.aqua-mail.com/ru/troubleshooting/

Вопросы и ответы: https://www.aqua-mail.com/ru/faq/

swregs

  • Newbie
  • *
  • Posts: 7
Re: Version 1.14.0-747-dev - "work in progress", not in Google Play
« Reply #7 on: January 18, 2018, 10:27:29 pm »
Unfortunately, no change.  In AquaMail, I tried upgrading an existing account to OAUTH2 login and also tried adding a new account directly as Office 365 type.  In both cases, after username and password sequence I get the same failed response saying my account is not in the 'Aqua Mail' tenant and thus cannot access the application id 906be9aa......

I am fairly sure the "Integrated Apps" is on by default.

My understanding of OAUTH2 flows is thin.  Are you using one of the MS provided Android SDKs like ADAL or MSAL?

I read that you have to use a multi-tenant aware auth endpoint instead of tenant specific endpoint.
It says https : //login.microsoftonline.com/common    instead of      https : //login.microsoftonline.com/tenant_domain   towards the end of this section in the docs https : //docs.microsoft.com/en-us/azure/active-directory/develop/active-directory-authentication-scenarios#basics-of-registering-an-application-in-azure-ad

If using ADAL, the authority endpoint seems to get specified when making an AuthenticationContext.
http : //simonjaeger.com/get-started-with-adal-for-android/


Kostya Vasilyev

  • Hero Member
  • *****
  • Posts: 12740
Re: Version 1.14.0-747-dev - "work in progress", not in Google Play
« Reply #8 on: January 18, 2018, 10:44:06 pm »
@swregs

No we don't use any SDK's - I wrote our OAUTH2 code originally for Gmail in 2014 and we've been building on top of that ever since.

But that's not the issue - we currently support OAUTH2 for Gmail, Hotmail, Yahoo, Yandex and they're all fine.

OAUTH2 is a pretty basic and low-level thing and every company does it a little differently.

In this case MS has some logic on top. "Tenant" and "Azure Directory" are not even OAUTH2 concepts, they're MS concepts, so that's what we're dealing with here, this "MS specific stuff on top of OAUTH2".

---

Long story short:

I now replaced tenant_id with "common" in the approval and token URLs (and kept the "prompt=consent" change from 749 which I believe is also required).

This still works for my account (so it's not completely broken by the "common" change :) ) - let's see if it makes any difference for your account:

https://www.aqua-mail.com/download/AquaMail-market-1.14.0-750-dev-95ece5523b20.apk


« Last Edit: January 18, 2018, 10:46:44 pm by Kostya Vasilyev, Aqua Mail »
Creating debug logs for diagnostics: https://www.aqua-mail.com/troubleshooting/

The official FAQ: https://www.aqua-mail.com/faq/

Лог-файлы для диагностики: https://www.aqua-mail.com/ru/troubleshooting/

Вопросы и ответы: https://www.aqua-mail.com/ru/faq/

swregs

  • Newbie
  • *
  • Posts: 7
Re: Version 1.14.0-747-dev - "work in progress", not in Google Play
« Reply #9 on: January 18, 2018, 11:59:52 pm »
Ok, so far so good.  I tried 3 different accounts in three different directories/tenants.  One was an in-place upgrade to OAUTH2, one was adding new account, and one was in-place upgrade to OAUTH2 of account that required me to MFA as part of the process.  All three are working.

It looks like mail, calendar, and contacts all sync given the "mailbox" access consent I allowed to "Aqua Mail" app when logging into each account.  I guess you get access to all needed data via EWS since those are really just folders in the mailbox.

Thanks for the efforts!

Kostya Vasilyev

  • Hero Member
  • *****
  • Posts: 12740
Re: Version 1.14.0-747-dev - "work in progress", not in Google Play
« Reply #10 on: January 19, 2018, 12:22:49 am »
Quote
All three are working.

Wonderful!

Quote
It looks like mail, calendar, and contacts all sync given the "mailbox" access consent I allowed to "Aqua Mail" app when logging into each account.  I guess you get access to all needed data via EWS since those are really just folders in the mailbox.

The permission is called "Access mailboxes as the signed-in user via Exchange Web Services" -

- which covers all off the EWS protocol (which is what we use).

There is a similar "inclusive" permission for ActiveSync.

( by contrast, the "new" APIs that you'd mentioned earlier use finer grained permissions )

Thanks for all the help (I remember seeing "common" in the docs , but was't 100% sure if it was required).
Creating debug logs for diagnostics: https://www.aqua-mail.com/troubleshooting/

The official FAQ: https://www.aqua-mail.com/faq/

Лог-файлы для диагностики: https://www.aqua-mail.com/ru/troubleshooting/

Вопросы и ответы: https://www.aqua-mail.com/ru/faq/

swregs

  • Newbie
  • *
  • Posts: 7
Re: Version 1.14.0-750-dev - "work in progress", not in Google Play
« Reply #11 on: January 19, 2018, 03:47:47 am »
Was curious for myself to read more about low-level OAUTH2 flows, which got me to this docs page, which I am sure you have already seen.
https : //docs.microsoft.com/en-us/azure/active-directory/develop/active-directory-protocols-oauth-code

In the "Request an authorization code" section, looking at the parameters table, the "prompt=" value choices are listed.  If the documentation is to be believed (which seems to often be wrong or stale compared to pace of feature updates and changes in Azure) and depending on your implementation, I wonder if using "prompt=consent" will always require a re-consent, instead of just the first time a user connects it to their account.  For example, my MFA enabled tenant requires re-login after 14 days or sometimes more frequently if it detects the same account is accessing resources from different or new locations.  With most apps, the AAD login page gets the password again and redoes MFA verification (Microsoft Authenticator app in my case), but does not re-do the consent step again.

I'll watch out to see what happens when my next re-login happens to be triggered.

Kostya Vasilyev

  • Hero Member
  • *****
  • Posts: 12740
Re: Version 1.14.0-750-dev - "work in progress", not in Google Play
« Reply #12 on: January 19, 2018, 12:40:04 pm »
Code: [Select]
I wonder if using "prompt=consent" will always require a re-consent, instead of just the first time a user connects it to their account.
...
I'll watch out to see what happens when my next re-login happens to be triggered.

According to this:

https://blog.mastykarz.nl/configuring-multi-tenant-authentication-azure-app-service-authentication-options/

prompt=consent is required for the initial login:

Quote
...the first time you try to authenticate to a multi-tenant application it isn't registered with your organization's AAD. To fix this issue you have to trigger the consent flow which will allow the user to login with their organizational account and grant the application the necessary permissions. The consent flow can be triggered by adding &prompt=consent to the Azure login URL.

...although I'm unable to find direct evidence of this in official docs.

So, sure, please keep me posted about the "14 day password expiration" case - we could try to omit "prompt=consent" on repeat authorizations. On the other hand, once the user has entered the password (and possibly dealt with MFA) - approving the permissions again is just a single tap :)
Creating debug logs for diagnostics: https://www.aqua-mail.com/troubleshooting/

The official FAQ: https://www.aqua-mail.com/faq/

Лог-файлы для диагностики: https://www.aqua-mail.com/ru/troubleshooting/

Вопросы и ответы: https://www.aqua-mail.com/ru/faq/

swregs

  • Newbie
  • *
  • Posts: 7
Re: Version 1.14.0-750-dev - "work in progress", not in Google Play
« Reply #13 on: February 02, 2018, 05:34:21 pm »
Yesterday I hit the "14 day password expiration" case.  I was able to re-auth with MFA without issues, although it takes a few steps.  This surfaced as a notification indicating "Incoming server login error".  Going to the account list screen the same message is shown for the account.  Clicking into the account enters the account setup screen where you have to hit "NEXT" to trigger the re-auth, enter password, possible MFA, and then re-prompt/accept for permissions, to be at end of setup stage where you have to hit "SAVE" to get back into e-mail.

I suspect I am sensitive to the number of steps since most end-users already complain about the interruption of having to re-login on a regular occasion or due to conditional access invalidating tokens.

Thanks, wanted to report back that it was working during re-auth scenario.

Kostya Vasilyev

  • Hero Member
  • *****
  • Posts: 12740
Re: Version 1.14.0-750-dev - "work in progress", not in Google Play
« Reply #14 on: February 02, 2018, 06:37:25 pm »
Alright, let's take this "in reverse order":

- The uses has to re-auth because that's your server's policy

- To indicate this to the user, we show a login error on the app's main screen

- But the app may not be running (started, visible on the screen) at all, so we show a status bar notification

Do you have suggestions on improving this "flow"?
Creating debug logs for diagnostics: https://www.aqua-mail.com/troubleshooting/

The official FAQ: https://www.aqua-mail.com/faq/

Лог-файлы для диагностики: https://www.aqua-mail.com/ru/troubleshooting/

Вопросы и ответы: https://www.aqua-mail.com/ru/faq/