Author Topic: No checking for absence of _legitimate_ addresses  (Read 8155 times)

StR

  • Hero Member
  • *****
  • Posts: 1558
No checking for absence of _legitimate_ addresses
« on: February 04, 2016, 07:58:11 pm »
It is not really a bug, more like a lack of a feature.  ;)

I know that Aquamail is making sure that at least one of the address fields is non-empty.
I discovered that Aquamail doesn't perform any tests for checking that the content is a legitimately-looking e-mail address. I am talking about a simple test for the presence of "@" (and maybe some other simple consistency checks, such as absence of spaces breaking up otherwise a legitimate address.

I had one letter entered in the address field, and the app attempted to send it, predictably resulting  in an error.
While many programs don't do that, I'd think it is an easy and relatively simple test to do. I believe I've seen some programs that do that, but I don't remember off the top of my head.

While we are at it, - may I propose that a "nudge" about an empty Subject would also be helpful.
Even many "historic" e-mail programs, such as elm, pine/alpine ... warn about that.
(And Thunderbird goes further, helping to prevent a common mistake, forgetting to attach an attachment that is mentioned in the text. Even though I appreciate that, I am not suggesting to go that far here. )



And one more thought in relation to the first issue above.
I have contacts that consist of just one letter in the name.
There are several reasons, but let me give you two:
1. Some people in the US that use they initial(s) instead of the name.
2. For privacy reasons (some Androids apps (including those by Google) insist on snooping into Contacts).

Unfortunately, Aquamail doesn't start picking any contacts until you enter two letters.
I remember your response in the discussion about this a few months ago, and I understand why you might be reluctant to start offering e-mail addresses with just one letter. But I wonder if you would consider doing that for the Names in the Contacts? I don't know if that would be too complicated to implement. If it is, don't worry, I have some workarounds (e.g. entering the first two letters of the e-mail address for these people, which requires me remembering those).




Kostya Vasilyev

  • Hero Member
  • *****
  • Posts: 12740
Re: No checking for absence of _legitimate_ addresses
« Reply #1 on: February 05, 2016, 09:20:12 pm »
Re: one letter contacts and filtering:

There is no distinction (auto-completing based on email vs. display name), it's a question of when filtering kicks in at all.

Re: malformed emails:

I've been suspecting as much for a few months but never did investigate -- will do so now, thanks for the nudge.
« Last Edit: February 05, 2016, 10:09:41 pm by Kostya Vasilyev, Aqua Mail »
Creating debug logs for diagnostics: https://www.aqua-mail.com/troubleshooting/

The official FAQ: https://www.aqua-mail.com/faq/

Лог-файлы для диагностики: https://www.aqua-mail.com/ru/troubleshooting/

Вопросы и ответы: https://www.aqua-mail.com/ru/faq/

StR

  • Hero Member
  • *****
  • Posts: 1558
Re: No checking for absence of _legitimate_ addresses
« Reply #2 on: February 06, 2016, 12:42:13 am »
Re: one letter contacts and filtering:

There is no distinction (auto-completing based on email vs. display name), it's a question of when filtering kicks in at all.

I understand that there is no distinction. I am proposing to actually make a distinction (if it does not require too much work), to start filtering after one letter for the name and after two letters for the address as opposed to after 2 for all).

And to add two more use cases:
1. when the "display name" (e.g. as saved from an incoming message) might have initials: F. S. Last. If I remember that person as "First Last", and I start entering "F"... I'd see him. But currently, I have no way of seeing him by entering his first name.
2. If I don't remember his name, but I remember it starts with "F" (while the name is stored in the contacts as "First S. Last"), I can enter "F" and I'd see a subset from which I might recall his full name.

Also, note, that the stock Contacts app starts "filtering" from the first letter entered. I have a record for "P. J."  By entering P in the contacts app, I already see that person's entry.

No pressure.  ::)

Kostya Vasilyev

  • Hero Member
  • *****
  • Posts: 12740
Re: No checking for absence of _legitimate_ addresses
« Reply #3 on: February 06, 2016, 12:43:50 am »
Re:  I am proposing to actually make a distinction

But there isn't one -- let's say you've entered one character -- at this point, I the code can't possibly know if this one character is going to match a first name, a last name, or an email.

It just decides at this point if it should start looking for matches or not.
Creating debug logs for diagnostics: https://www.aqua-mail.com/troubleshooting/

The official FAQ: https://www.aqua-mail.com/faq/

Лог-файлы для диагностики: https://www.aqua-mail.com/ru/troubleshooting/

Вопросы и ответы: https://www.aqua-mail.com/ru/faq/

Kostya Vasilyev

  • Hero Member
  • *****
  • Posts: 12740
Re: No checking for absence of _legitimate_ addresses
« Reply #4 on: February 06, 2016, 01:22:22 am »
@StR:

Please try this one:

https://www.aqua-mail.com/download/AquaMail-market-1.6.1.0-dev4.4-filter-1.apk

It queries system Contacts after one character.

Previously my concern was for Exchange contacts (the built-in database):

- First, querying it used to be quite slow, but I made it very fast a few (maybe 6) months ago

- Second, querying the server with a single character criteria could come up with *a lot* of matches (esp. if if you consider the corporate directory). I now made it only run the server search at 2 or more characters.
Creating debug logs for diagnostics: https://www.aqua-mail.com/troubleshooting/

The official FAQ: https://www.aqua-mail.com/faq/

Лог-файлы для диагностики: https://www.aqua-mail.com/ru/troubleshooting/

Вопросы и ответы: https://www.aqua-mail.com/ru/faq/

StR

  • Hero Member
  • *****
  • Posts: 1558
Re: No checking for absence of _legitimate_ addresses
« Reply #5 on: February 06, 2016, 05:47:39 am »
Thank you!
I just  tried it once.
It works. It seemed a bit slower than based on two letters,  but it could've been my phone at that moment.
I'll report if any problem shows up.



Kostya Vasilyev

  • Hero Member
  • *****
  • Posts: 12740
Re: No checking for absence of _legitimate_ addresses
« Reply #6 on: February 06, 2016, 06:44:26 pm »
Well, yes, it is a bit slower -- the filtering is in Android Contacts provider, not in my code, but it is more work for the database.

And so I just made a change to only enable this for Android 4.4, to hopefully cut out the really slow devices.
Creating debug logs for diagnostics: https://www.aqua-mail.com/troubleshooting/

The official FAQ: https://www.aqua-mail.com/faq/

Лог-файлы для диагностики: https://www.aqua-mail.com/ru/troubleshooting/

Вопросы и ответы: https://www.aqua-mail.com/ru/faq/

StR

  • Hero Member
  • *****
  • Posts: 1558
Re: No checking for absence of _legitimate_ addresses
« Reply #7 on: February 06, 2016, 08:30:46 pm »
And so I just made a change to only enable this for Android 4.4, to hopefully cut out the really slow devices.
Oh, no! Please, no! Put it back!
It was just slightly slower (on my Droid Razr M with 4.1.2), maybe a second or even less. and I mentioned it just as a fact.
Besides, as I wrote, it could've been the state of the phone (other things running) that contributed as well.

BTW, something that I experienced last night:
I tired to download it with that https: link using Opera Mobile. And I've done that in the past, but last night Opera was claiming that the link doesn't exist.
The only thing that helped was changing it to http:. (While on my desktop, I was able to start downloading the apk.)
I don't know - it could be a glitch of the browser, or something... I was falling asleep, so, I had not investigated the problem. I am just letting you know in case somebody else had other related problems...

mikeone

  • Hero Member
  • *****
  • Posts: 2762
Re: No checking for absence of _legitimate_ addresses
« Reply #8 on: February 06, 2016, 11:43:44 pm »
Hi Kostya
I agree with StR to include at least Android Jelly Bean (4.1.x, 4.2.x, 4.3.x). Devices with these version(s) should be fast enough.
 8)

StR

  • Hero Member
  • *****
  • Posts: 1558
Re: No checking for absence of _legitimate_ addresses
« Reply #9 on: February 07, 2016, 06:17:30 am »
I checked today a few times while sending messages  and it works just fine, no significant delay.

Kostya Vasilyev

  • Hero Member
  • *****
  • Posts: 12740
Re: No checking for absence of _legitimate_ addresses
« Reply #10 on: February 07, 2016, 10:30:39 pm »
Re: link

I use "client cert validation" on the https port, it's optional, but isn't handled by Android browsers very well...

Chrome takes you to the "set up PIN code" screen, as the first step to installing said client cert (which nobody but me uses) -- and Opera apparently can't handle it at all.

Not sure what I should do, I only need it for one specific web service, but it's on the same port.

Re: put it back, etc.

I'm seriously thinking about disabling it altogether.

I'm seeing a delay of about 200-300 ms on a 5.1 device, and 400-500 ms on a 4.4 low-end device.

When using two characters (same devices, same contact list) takes about 30-50ms, it's instant.

The (larger) delay is definitely very noticeable, and I just don't want users to complain.

For you, StR, it's not a problem -- because you *already know* what's happening and why.

For others, it's just going to be another case of "oh no, what have you done with your app".

Creating debug logs for diagnostics: https://www.aqua-mail.com/troubleshooting/

The official FAQ: https://www.aqua-mail.com/faq/

Лог-файлы для диагностики: https://www.aqua-mail.com/ru/troubleshooting/

Вопросы и ответы: https://www.aqua-mail.com/ru/faq/

StR

  • Hero Member
  • *****
  • Posts: 1558
Re: No checking for absence of _legitimate_ addresses
« Reply #11 on: February 07, 2016, 10:46:37 pm »
Kostya, I understand why users' reaction for such a delay is a problem.
And I wouldn't blame you if you decided to disable it. (Well  I will still blame you ans those dumb users :) but not too strongly.)
I might stay on the current -dev version for a while... at least until some new major features (that of interest to me) are added. So, thanks for producing this short-lived version.

Facts for your consideration:
1. I am serious when saying that the subsequent lookups are much faster,  comparable to the prior behavior.  I suspect some caching takes place.
I Haven't measured,  but estimate it to be under 200 ms, more likely 100 or faster.

2. Users who don't know about the new functionality, can continue typing the second letter.They might never notice the difference.
I tested that... It works on my underpowered 4.1.2 Razor M.
« Last Edit: February 07, 2016, 10:50:09 pm by StR »

Kostya Vasilyev

  • Hero Member
  • *****
  • Posts: 12740
Re: No checking for absence of _legitimate_ addresses
« Reply #12 on: February 09, 2016, 10:47:37 pm »
Quote
Facts for your consideration:
1. I am serious when saying that the subsequent lookups are much faster,  comparable to the prior behavior.  I suspect some caching takes place.
I Haven't measured,  but estimate it to be under 200 ms, more likely 100 or faster.

2. Users who don't know about the new functionality, can continue typing the second letter.They might never notice the difference.
I tested that... It works on my underpowered 4.1.2 Razor M.

1 - Not surprising, there is caching (in SQLite and the file system), but this caching is very very far from my code. So I can't preload.

In addition, matching on two characters considers fewer contacts (before trimming the result), so it is also much faster for that reason.

2 - Before they can type the second letter, the first letter match does come up, and it is slow, noticeably so.
Creating debug logs for diagnostics: https://www.aqua-mail.com/troubleshooting/

The official FAQ: https://www.aqua-mail.com/faq/

Лог-файлы для диагностики: https://www.aqua-mail.com/ru/troubleshooting/

Вопросы и ответы: https://www.aqua-mail.com/ru/faq/

StR

  • Hero Member
  • *****
  • Posts: 1558
Re: No checking for absence of _legitimate_ addresses
« Reply #13 on: February 09, 2016, 11:07:08 pm »
1 - ...
In addition, matching on two characters considers fewer contacts (before trimming the result), so it is also much faster for that reason.
Just in case: I was still comparing matching on one character to matching on one character, but just the very first I've done that with the current version with all subsequent attempts (I just tried for the first time since I did it two days ago, - and it was fast, <100 ms.)
So, I don't know how long its caching for.

Quote
2 - Before they can type the second letter, the first letter match does come up, and it is slow, noticeably so.
It might that it's just doing it so fast that I am noticing it, - but I just tried this, - I am typing 3 letters in a raw without any delay.
I've tried with a different first letter, and it's always the same.
Is there any way to clear whatever that SQL cached so that I can try typing these letters in an "uncached" state?

Kostya Vasilyev

  • Hero Member
  • *****
  • Posts: 12740
Re: No checking for absence of _legitimate_ addresses
« Reply #14 on: February 09, 2016, 11:26:45 pm »
Quote
Just in case: I was still comparing matching on one character to matching on one character, but just the very first I've done that with the current version with all subsequent attempts (I just tried for the first time since I did it two days ago, - and it was fast, <100 ms.)
So, I don't know how long its caching for.

SQlite has cache, and the kernel has a file system cache.

And those are really low level things which an application -- several API layers away -- absolutely cannot control.

I really feel for you, but have to make decisions based on "what others (note the plural) might think".
Creating debug logs for diagnostics: https://www.aqua-mail.com/troubleshooting/

The official FAQ: https://www.aqua-mail.com/faq/

Лог-файлы для диагностики: https://www.aqua-mail.com/ru/troubleshooting/

Вопросы и ответы: https://www.aqua-mail.com/ru/faq/