SPF Too Many DNS Lookups issue

In our scenario, We're running into an issue with our SPF records in route53. We are receiving SPF Too Many DNS Lookups because we are using multiple mail services. Is there a way to resolve this? I saw one thread stating to run a look up on the domains and include them by IP but that doesn't seem too stable of a solution. Another said to us PTR but that seems like the same path. Is there a better way you know of that we could achieve this?

"Error! 15 DNS lookups required to evaluate the SPF record. The maximum is 10."

v=spf1 ip4:52.209.21.221 include:spf1.abc.com include:spf2.abc.com ~all

We have 7 mailing platforms which is causing too many DNS lookups.

spf1.abc.com    -
    
"v=spf1 include:_spf.salesforce.com include:spf.protection.outlook.com include:mail.zendesk.com ~all"

spf2.abc.com    -
    
"v=spf1 include:amazonses.com include:sendgrid.net include:abc-com.spf.smtp25.com include:_spf.google.com ~all"

Is this modification will help

"v=spf1 ip4:52.209.21.221 include:spf1.abc.com -all"

Do we need to go with SPF flattening technique, how can we achieve that can anyone advise me on this?

As you're aware there's a hard maximum of 10 lookups. A lookup is counted each time you see the word include; so say your DNS TXT records were like this:

In addition to your root domain you have 5 other DNS records referenced in your includes, so your SPF lookup count is 5 (the root isn't included in the count); so you could have up to 5 more.

You can get a good view of your SPF records (recursive) via this tool: https://www.dmarcanalyzer.com/spf/checker/ (many others are also available).

Note: This count is the total recursive count; not just the number of lookups on any one record, or down any single branch; but across the entire tree/graph.

Flattening

There are some obvious things we can do to reduce the number of lookups; e.g. spf1.exmple.com and spf2.example.com aren't really adding any value; so we can just drop those records and update example.com to refer directly to the thirdparty SPF records; thus removing 2 from our count.

One thing to be careful of when doing this: consider why those were created in the first place, given they're just increasing complexity... Maybe they're referred to by other domains. E.g. My company owns many domains, some of which will all share the same service (e.g. we use Office/Exchange 365 for mail and ZenDesk for support; both configured to send emails from the countries' specific domains). Because we have this sharing, we have an SPF record in our primary domain which has the includes for those services which are shared by all these domains, so we can manage them all in one place / update all referencing domains via a single record.

To flatten things further you can fetch the third party's records and put their values into your records instead of having an include to dynamically reference it. The issue here is that if the third party changes the value of this record, your implementation will break as you'll have the wrong values. You can mitigate that by either having scripts monitor the original record (and automatically correct your records with updates, or alert you of the issue when this is detected), or some companies implement their own management layer over DNS meaning you can define your lookups, and they'll flatten the DNS records for you. I've no experience with those so won't link to anyone; but a simple search for DNS Management Service SPF Flattening should give you a few options.

Note: DNS TXT values also have a max length of 255 characters in a string. However, you can have multiple strings per record; so just create additional values (essentially you can have an array of <=255 character strings across which you can hold your value).

Alternative Options

Remove Services

Often multiple SPF records are needed because different teams within a company have purchased different solutions which fulfil the same requirement (e.g. SMTPaaS solutions). If you can consolidate providers you'll be able to remove some of these. That said, this often required buy in from more seniour management, who often don't appreciate the implications of this scenario.

Subdomain per Service

Though your example.com domain can only have 10 lookups against it, you can have an entirely separate 10 lookups for support.example.com or sales.example.com, so you could partitionl your lookups by function (or even service; e.g. zendesk.example.com to have:

Then work with addresses specific to those subdomains (e.g. [email protected] for your service desk). If those addresses need to be associated with a real mailbox you'd need to include your outlook.com value on the relevant subdomain too (as shown in the Sales example above), but it should be rare for you to need ALL lookups for any domain (e.g. SalesForce AND ZenDesk AND Outlook).

Note that you can also use email aliases to give the same mailbox multiple addresses, so even if you do have a mailbox that needs to be associated with multiple systems, maybe you can have different sender addresses for that mailbox, but still have replies to any of those addresses resolve to the same target mailbox.

Documentation

Not so much a solution to a current problem, but useful to help with future issues. Every time you add a service to your SPF record, record why those entries were added. Often people in the business request these without understanding the implications, then forget about them once all's working. Very often they'll be setup for a temporary requirement but no one will think to inform you when that requirement ceases to exist. By documenting things you can review your records periodically & find the people to contact to check whether the requirement still exists (or at least have some idea of where to look to see if ServiceX still has any relevancy), helping you to remove outdated services or know who to contact if you need to move a service off to its own sub domain / etc.