I’ve been a .com purist for over two decades of building. Once, I broke that rule and bought a .online TLD for a small project. This is the story of how it went up in flames.
↫ Tony S.
An absolute horror story about Google’s dominance over the web, in places nobody really talks about. Scary.
Same story with custom e-mail servers; one can configure all the ARCs & DMARCs, get 100% delivery everywhere else, straight to spam on Gmail.
This isnt true. My company handles ~2800 domains with private email domains with SMTP handled by various different companies. As long as you have valid SPF, DKIM and DMARC records and a source IP with a valid PTR record then its very rare to get spam filtered on gmail. Even for bulk senders
If you are having problems sending to gmail then you have made a mistake in your setup. Its pretty tolerant, even for cases where you have multiple DMARC entries and one of them doesnt work, as long as one of them is correct then they allow to inbox
The ..online farce is very real though and you should avoid domains that are not official country codes, .net, .org or .com
corestaq,
For better or worse, email filtering heuristics often punishes the innocent for having bad neighbors on the network. I once had significant gmail delivery issues that needed to be solved by physically moving the server and getting a new IP. The configuration was never the problem. Alas because of heuristics, two people following the exact same email hosting guidelines can experience radically different blocking behavior.
And the same rules don’t apply to everyone. There was a period a few years ago when google were originating the majority of spam that we would have blocked had it come from a no-name provider. But given google’s monopoly on email….we just had to put up with it (fortunately no longer the case). A separate time a client of mine complained about not receiving emails from government contacts. I checked and the government emails were bouncing because automated DNS and SPF checks were failing. The bounces were functioning correctly as intended: the configuration at the sender was at fault. Yet despite escalating the issue, who do you think won that battle? That’s right, the government won, getting a whitelist exemption from the rules that apply to everyone else. I bet that same government agency got privileged treatment by google as well. It’s just the way the game goes. IRL rules get bent depending on who you are. VIPs get special treatment all the time.
You are trying to equate a bizarre edge case with standard behaviour. Those things are not the same
Email is not difficult as long as its setup correctly and you are not running from a provider that sends a lot of spam or a home internet IP. Even a home broadband IP is doable if you have a static and a valid PTR record
Ive cleaned up a lot of spam filtered problems for email domains and almost every time its missing or badly configured SPF/DKIM/DMARC records. Its VERY rare to get blacklisted IP’s unless you are doing your mail hosting in Russia or Eastern Europe
corestaq,
Budget data centers probably have a higher ratio of scammers. There is at least some luck involved. Ironically some of the more popular providers like godaddy (I know it’s hard to believe, but they’re very popular with US companies) are some of the worst I’ve found for blacklisting.
I agree it’s not technically too difficult when you know what you are doing, but realistically when you sign up for a service there’s a chance you’ll end up in a poor reputation block, which you may end up fighting despite following best practices for email.
Personally I would not recommend it for people who actually care about email delivery. If you’re just experimenting then go for it, I am a fan of DIY. It will probably work with some email providers, yet other providers won’t like your residential IP.
Don’t get me wrong, if things are technically wrong on your side, then of course fix it. However people should be aware that this isn’t always enough. Some blacklists actually demand payment to delist IPs, even if they were blacklisted before being assigned to your server. Sometimes moving is a better strategy rather than trying to fight with money and time to improve the reputation of IP blocks that were damaged by others. YMMV.
You probably have an IP subnet or earned reputation from long operation. The GMail problem is that they play by the rules up to delivery, and then the spam filter executes the fishy IP-based policies without any feedback to the sender. It is easier to get actual spam through.
Same here. I’ve been successfully hosting email for years from 2 residential lines with fixed IPs and correct PTR. I had to pray to the carriers to get me a nice IP with good reputation but else, it was about being very careful with the setup not to miss any step before going live and I don’t even have to think about it anymore.
FWIW, I’ve had that experience much more often with Microsoft’s EMail hosting than Google’s. IME, they (MS) are particularly bad for mindlessly blocking the entire IP ranges of VPS providers, and then taking 2-3 weeks to apply “mitigation.” And that’s best case, when they don’t just reject the delisting request the first 4 or 5 times.
It’s not just MS, though; in the past 15 years or so, the entire “EMail security” industry seems to have gradually moved away from prioritizing deliverability — to prioritizing minimizing liability (“Better to have a 1,000 false-positives/legitimate EMail messages blocked as spam, then to be potentially liable for letting one malicious message through”).
Google safebrowsing may be a useful flagging tool, but to basically put google in charge.of taking down domains without so much as a notification, due process, or review… What irresponsible bullshit by RADIX! I didn’t have any opinion of them before, but they’ve lost my respect as a registry operator even though this didn’t effect me.
Alfman,
Yes, it is a very valuable signal, but obviously misused here.
How can a (root level) registrar block a customer’s domain without any possible direct avenue of remediation?
“Please go see a third party which has 0,01% false positive rate. That is 100 in a million, and there are billions of websites, which means there would be 100,000s false positives, but we don’t care”
The problem here is not Google’s though. It is more on the registrar for being so obtuse.
sukru,
Yeah, I can’t blame google for this. False positives can happen of course, but it’s RADIX’s fault they disabled the domain without any investigation. Fortunately their TLD’s are less common, but just think if it were your business.
RADIX’s resolution process needs to be called out. Since they’ve disabled the domain, how the hell do they expect anyone to review it to delist it? Hopefully someone at RADIX is following the social media and realizes they are being asshats! This guy was lucky that someone at google saw his post on hacker news, otherwise he might still be offline in a catch-22.
Alfman,
I believe every process should have an escalation path.
We can never have perfect algorithms especially when humans are mixed. It does not even have to be free (maybe you pay an additional fee), but one should be able to off the script.