Yes, I do understand your use case, and I'd want the same if I was getting messages with no text and attachments.
But the user who asked for the automatic switching to be removed (except for calendar attachments, for which I've kept the auto-switching), had his reasons too, although we didn't go into details.
So now I have two users (two respected, valuable users, who I appreciate having) who are trying to pull me in opposite directions. With good intentions, and a good use case.
Another setting? Ugh.
Ok, what type of mail account do you use, IMAP / POP3 or EWS?