Using DMARC information is complicated in practice in the real world
As part of a planned switch to rspamd as our anti-spam system (well, our spam recognition system), I’ve been taking a closer look at how our test rspamd scores some email and what it reports about why. This has given me a new and unhappy view of DMARC in the real world, building on how DKIM looks for our ‘good’ email. So let me tell you a story, starting with the background.
The university is now
a big user of Microsoft Teams. The university’s UTmail+ institutional email system for staff, faculty, and so on is also “powered by Office 365”, which is to say that it is hosted in Microsoft’s ‘Outlook/Office 365’ cloud (with a bunch of contractual terms around what datacenters our data actually lives in and so on). However, while everyone here has a UTmail+ account (which is also your Teams account), some people forward their email to other systems. In particular, any number of people here forward their UTmail+ email to us.
Various activity in Microsoft Teams generates email to you, which normally is sent to your institutional email account in UTmail+ and may then get forwarded to us. As a good email citizen in the modern world, Microsoft Teams DKIM signs its email:
DKIM: d=email.teams.microsoft.com s=selector1 c=relaxed/relaxed a=rsa-sha256
Such email comes from ‘noreply@email.teams.microsoft.com’ as both the envelope sender and the From: header address.
As a good email citizen in the modern world, the Outlook/Office 365 environment also DKIM signs outgoing email, this time under the ‘onmicrosoft.com’ domain instead of microsoft.com:
DKIM: d=utoronto.onmicrosoft.com s=[…] c=relaxed/relaxed a=rsa-sha256
Unfortunately, when Microsoft Teams email transits either our hosted Office 365 environment or at least one other institution’s hosted one, something breaks the Microsoft Teams DKIM signature. At the same time, Office 365 does not change the From:, which means that the message is covered by whatever DMARC policy applies to that ‘noreply@email.teams’ address.
The email.teams.microsoft.com sub-subdomain does not have its own DMARC policy. Instead it falls under the general microsoft.com DMARC policy:
v=DMARC1; p=reject; pct=100; […]
This says that anything with a From: of a microsoft.com address that fails DMARC and SPF should be rejected. Since Microsoft advertises this DMARC policy, rspamd takes them at their word and applies a spam score penalty to all of this entirely legitimate email.
There is a standard that is supposed to deal with this problem, ARC. The Office 365 forwarding environment appears to put on various ARC headers, but in a sample Teams email I have these headers seem to claim not to have validated the original DKIM signature, just the SPF results. In any case, it feels like there would need to be some explicit configuration somewhere so that either we or Microsoft would give ARC signatures from these Office 365 environments the power to give a pass on microsoft.com’s DMARC policy. In practice rspamd sometimes applies a mild bonus (ie, not-spam) for ARC_ALLOW, and sometimes gives it a neutral result of ‘no change in score’ (a 0.00 result, at least according to the logs).
(It appears that rspamd 2.5 adds an undocumented option to its ARC module to configure a whitelist of trusted ARC signers, per here, here and here, but without documentation I’m wary of touching it even if I could figure out what exactly to put where.)
Beyond the specific problem here and the opacity of what is going on in evaluating DKIM, DMARC, and ARC results in rspamd, notice how we have wound up in a situation where none of these things can be checked in isolation. In practice, you cannot just look at the From: domain and see whether it passes DMARC and DKIM checks; you need additional validation that is conditional on, well, something. In practice, to reliably accept Microsoft Teams email that transits a hosted Office 365 environment we would probably have to identify all of the outgoing sources for this, including for people who forward their Microsoft Teams email through another place that hosts their email in Office 365.
The practical result of all of this is that today I reconfigured rspamd to not assign any spam score penalty for a DMARC failure. DMARC failures are clearly going to happen with legitimate email to our users that transits through their UTmail+ addresses (at least), which in practice makes them useless.
(One comment.)
Author: cks
Date: 2020-10-01
URL: https://utcc.utoronto.ca/~cks/space/blog/spam/DMARCPracticalComplexity
utoronto.ca
| Why I write recursive descent parsers (despite their issues) (2020-09-16) | Today I read Laurence Tratts Which Parsing Approach? via which has a decent overview of how parsing computer languages including little domain specific languages is not quite the well solved problem wed like it to be As part of the article Tratt discusses how recursive descent parsers have a number of issues in practice and recommends using other things such as a LR parser generator I have a long .. |
| Disabling DNF modules on Fedora 31 so they don’t mess up package updates (2020-08-11) | Fedora 31 DNF modules and probably Fedora 32 ones as well are currently broken as covered here and here Fedora is not insuring that DNF modules that claim to be the latest for example ripgrep have versions of their packages that are as recent as the non-modular packages the ripgrep module has 1102; Fedora 31 has 1201 and the mere existence of a DNF module for something will block updates even if y.. |
| Some thoughts on how I still miss DTrace (and also mdb) (2020-11-30) | Although Im generally happy with our Linux fileservers every so often we run into an issue where I miss OmniOSs DTrace and mdb; DTrace for dynamic visibility into what the system was doing and mdb for static inspection and tracing through kernel data structures In theory Linux has equivalents of both of these In practice this Linux future is unevenly distributed Its likely that our Linux fileserve.. |
| Whether extra disks should be live or spare now depends on HDs versus SSDs (2020-10-09) | Suppose not entirely hypothetically that you have a server with at least three or four spare drive bays and you want to build a mirrored storage setup that can maintain redundancy without requiring an in-person drive swap should a drive fail Lets say you go with three drives in the system in total Obviously two of them have to be mirrored in order to get your basic redundancy but the third one cou.. |
| The better way to make an Ubuntu 20.04 ISO that will boot on UEFI systems (2020-11-27) | Yesterday I wrote about how I made a 2004 ISO that booted on UEFI systems It was a messy process with some peculiar things that I didnt understand and places where I had to deviate from Debians excellent documentation on Repacking a Debian ISO In response to my entry Thomas Schmitt the author of xorriso got in touch with me and very generously helped me figure out what was really going on The shor.. |
| Important parts of Unix’s history happened before readline support was common (2020-08-17) | Unix and things that run on Unix have been around for a long time now In particular GNU Readline was first released in 1989 as was Bash which is long enough ago for it or lookalikes to become pretty much pervasive especially in Unix shells Today its easy to think of readline support as something thats always been there But of course this isnt the case Unix in its modern form dates from V7 in 1979 .. |
| Console blanking now defaults to off on Linux (and has for a while) (2020-11-07) | For a long time if you left a Linux machine sitting idle at a text console for example on a server the kernel would blank the display after a while Years ago I wrote an entry about how you wanted to turn this off on your Linux servers where at the time the best way to do this was a kernel parameter For reasons beyond the scope of this entry I recently noticed that we were not setting this kernel p.. |
| We need to start getting some experience with using Ubuntu 20.04 (2020-10-18) | Under normal circumstances we would have a decent number of machines running Ubuntu 2004 by now probably including our login servers But the situation is not normal because ongoing world and local events still have us working from home making it not so simple to install and deploy a new physical server with a new version of Ubuntu However it really looks like this is the new normal so we should st.. We need to start getting some experience with using Ubuntu 20.04 |
| Sorting out what the Single Unix Specification is and covers (2020-10-08) | Ive linked to the Single Unix Specification any number of times for various versions of it when I first linked to it it was at issue 6 in 2006; its now up to a 2018 edition But Ive never been quite clear what it covered and didnt cover and how it related to POSIX and similar things After yesterdays entry got me looking at the SuS site again I decided to try to sort this out once and for all My pri.. |
| The issue of how to propagate some errors in our Django web app (2020-08-03) | Much of what our Django application to handle Unix account requests does is only available to special people such as professors who can actually approve account requests instead of just making them Following our usual we protect the management section of the web app with Apache HTTP Basic Authentication where only people in designated Unix groups such as the sponsors group have access However the .. |
