Ask HN: Trying to be more Gmail-independent with Gandi and email forwarding
6 comments
I used the setup you suggest for a long time. It's an excellent intermediary step but long term, I do recommend you move to either GSuite if you want to keep the gmail interface, or otherwise another provider such as fastmail.
My primary Google account is now a GSuite (legacy) account, and my secondary one is a paid GSuite account. I honestly think that if you want to remain in the google ecosystem, paying for it is a generally good idea. $5/month is cheap.
My primary Google account is now a GSuite (legacy) account, and my secondary one is a paid GSuite account. I honestly think that if you want to remain in the google ecosystem, paying for it is a generally good idea. $5/month is cheap.
Thanks for your reply. Yes, I might decide to become a GSuite customer at some point (maybe soon).
Since you say you used my setup for a long time, are you referring to the forwarding feature or the POP3 feature?
If POP3: how did you deal with lack of security?
If forwarding: how did you deal with basically all forwarded emails failing SPF validation when reaching the google server?
Thanks
Since you say you used my setup for a long time, are you referring to the forwarding feature or the POP3 feature?
If POP3: how did you deal with lack of security?
If forwarding: how did you deal with basically all forwarded emails failing SPF validation when reaching the google server?
Thanks
I actually use both. I use a POP3 enabled account on my hosting provider, but because Gmail only polls for new mail from external accounts on its own schedule (you can't specify the frequency, and when you're on the phone and someone says "I've emailed that through, is X correct?", waiting 5 or 15 minutes for it to appear in your inbox in Gmail isn't ideal, so I also have mail forwarding turned on. This means that for mail where the DKIM policy allows forwarding, you get the email almost instantly, but if the DKIM fails, you will still get it just a few minutes later. I've been doing it this way for years without an issue. I set a very strong password on my POP account (since I only have to enter it once into the Gmail interface to setup the account) and accept the security risk of no 2FA.
Thanks.
What you are implying in your reply is that Google ignores its own SPF validation results for spam purposes/mail acceptance then?
When you enable forwarding, even the emails that don't fail DKIM still show up in gmail with the SPF fail (under "Show Original"/"SPF"), and I'm sure they do in your setup as well since it's a logic consequence of doing the forwarding.
I'm essentially trying to figure out if any of these conditions can happen with forwarding enabled:
1) SPF validation will fail, the Google mail server will refuse the forwarded message, so the Gandi mail server will bounce it back to the sender without me even knowing, rather than depositing it in the associated mailbox (especially with the domains set with SPF "-all", such as several of the financial institutions I use). So, no POP3 fallback even if it's enabled along with the forwarding.
2) SPF validation will fail, the Google mail server will still accept the forwarded message, but will be more inclined to mark it at spam.
Thanks
What you are implying in your reply is that Google ignores its own SPF validation results for spam purposes/mail acceptance then?
When you enable forwarding, even the emails that don't fail DKIM still show up in gmail with the SPF fail (under "Show Original"/"SPF"), and I'm sure they do in your setup as well since it's a logic consequence of doing the forwarding.
I'm essentially trying to figure out if any of these conditions can happen with forwarding enabled:
1) SPF validation will fail, the Google mail server will refuse the forwarded message, so the Gandi mail server will bounce it back to the sender without me even knowing, rather than depositing it in the associated mailbox (especially with the domains set with SPF "-all", such as several of the financial institutions I use). So, no POP3 fallback even if it's enabled along with the forwarding.
2) SPF validation will fail, the Google mail server will still accept the forwarded message, but will be more inclined to mark it at spam.
Thanks
Where Gmail refuses to accept the forwarded email (because the DMARC policy doesn't allow forwarding) I receive the 'Mail Forward Failure' notification but the email has been accepted into the original mailbox, so will be pulled into my Gmail mailbox the next time the POP3 check runs. I have a rule on the Gmail side to remove those failure notifications.
I haven't noticed any impact to Gmail's actual spam filtering, either garbage slipping through or false positives.
I've just checked a few random emails and Gmail shows "SPF: PASS with IP xxx.xxx.xxx.xxx". For most emails, the DKIM and DMARC checks also show PASS, but this is obviously dependent on the sender.
I haven't noticed any impact to Gmail's actual spam filtering, either garbage slipping through or false positives.
I've just checked a few random emails and Gmail shows "SPF: PASS with IP xxx.xxx.xxx.xxx". For most emails, the DKIM and DMARC checks also show PASS, but this is obviously dependent on the sender.
Last question: do you use Gandi as a forwarder? If yes, I truly can't understand how it is possible that the forwarded emails (non POP3) can pass the spf on the google side, since your mail server acts as a forwarder and most definitely is not allowed to do so, according to the SPF records.
Thanks for engaging!
Thanks for engaging!
Sorry, no, I'm not using Gandi, I actually have a legacy account in Zoho that I use (from back when POP3 was available on free accounts), but the logic is the same.
The emails get accepted into the Zoho mailbox, then forwarded to Gmail. If Gmail rejects the forwarded email (due to DMARC policy) then I get the failure notification, not the original sender (as the failure is that Zoho was unable to forward the email to Gmail, not that Zoho didn't accept the original email from the sender), and that email will be retrieved by Gmail when it next does it's POP3 check.
The emails get accepted into the Zoho mailbox, then forwarded to Gmail. If Gmail rejects the forwarded email (due to DMARC policy) then I get the failure notification, not the original sender (as the failure is that Zoho was unable to forward the email to Gmail, not that Zoho didn't accept the original email from the sender), and that email will be retrieved by Gmail when it next does it's POP3 check.
Thanks.
The part that I'm not understanding, and at this point I will put it in my todo list to sign up for a free forwarding account in Zoho to try it out, is how it is possible for the Zoho mail server to forward emails to Gmail without the messages being flagged as SPF fail, since your Zoho server is not authorized to dispatch emails from the sender domain when they have SPF records.
In other words, this is a real example from an email that just got flagged on my Gmail from apple.com, as part of being forwarded:
1) The email is being sent from [email protected]
2) The email is received by the Gandi mail server from Apple mail server
3) The email is received by the Google mail server from Gandi mail server
Step 3 is where the forwarding happens but, since the Gandi mail server is not supposed to originate emails from apple.com (according to the SPF records on the apple.com domain), then it gets marked as SPF fail, and the Google mail server tells why (ip address of the Gandi mail server is not allowed to generate mail from [email protected]).
I don't understand how that cannot happen with Zoho, and I'll study it asap.
The part that I'm not understanding, and at this point I will put it in my todo list to sign up for a free forwarding account in Zoho to try it out, is how it is possible for the Zoho mail server to forward emails to Gmail without the messages being flagged as SPF fail, since your Zoho server is not authorized to dispatch emails from the sender domain when they have SPF records.
In other words, this is a real example from an email that just got flagged on my Gmail from apple.com, as part of being forwarded:
Received: from relay5-d.mail.gandi.net (relay5-d.mail.gandi.net. [217.70.183.197] by mx.google.com ...
Received-SPF: softfail (google.com: domain of transitioning [email protected] does not designate 217.70.183.197 as
permitted sender) client-ip=217.70.183.197;
Received: from nwk-txn-msbadger0502.apple.com (nwk-txn-msbadger0502.apple.com [17.151.1.69]) by spool.mail.gandi.net ...
Date: Mon, 21 Jan 2019 03:18:35 +0000 (GMT)
From: Apple <[email protected]>
Reply-To: [email protected]
1) The email is being sent from [email protected]
2) The email is received by the Gandi mail server from Apple mail server
3) The email is received by the Google mail server from Gandi mail server
Step 3 is where the forwarding happens but, since the Gandi mail server is not supposed to originate emails from apple.com (according to the SPF records on the apple.com domain), then it gets marked as SPF fail, and the Google mail server tells why (ip address of the Gandi mail server is not allowed to generate mail from [email protected]).
I don't understand how that cannot happen with Zoho, and I'll study it asap.
Here's the relevant headers from a recent email from Apple, including the Authentication-Results and Received-SPF lines:
Delivered-To: (ME)@gmail.com
Received: by 2002:a9d:650d:0:0:0:0:0...
Received: from sender-of-f72.zoho.com (sender-of-f72.zoho.com. [135.84.80.227]) by mx.google.com...
Received-SPF: pass (google.com: domain of (ME)+aml_=(MY DOMAIN)=gmail.com@(MY DOMAIN) designates 135.84.80.227 as permitted sender) client-ip=135.84.80.227;
Authentication-Results: mx.google.com;
dkim=pass [email protected] header.s=id0517 header.b=S5Hlyy1T;
spf=pass (google.com: domain of (ME)+aml_=(MY DOMAIN)=gmail.com@(MY DOMAIN) designates 135.84.80.227 as permitted sender) smtp.mailfrom="(ME)+aml_=(MY DOMAIN)=gmail.com@(MY DOMAIN)";
dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=id.apple.com
Received-SPF: pass (zoho.com: domain of email.apple.com designates 17.151.1.66 as permitted sender) client-ip=17.151.1.66; [email protected]; helo=nwk-txn-msbadger0405.apple.com;
Authentication-Results: mx.zohomail.com;
dkim=pass;
spf=pass (zoho.com: domain of email.apple.com designates 17.151.1.66 as permitted sender) [email protected];
dmarc=pass(p=reject dis=none) header.from=id.apple.com
Received: from nwk-txn-msbadger0405.apple.com (nwk-txn-msbadger0405.apple.com [17.151.1.66]) by mx.zohomail.com...
From: Apple <[email protected]>Thanks, really appreciate you taking the time to extract those headers, very helpful.
Looking at the headers, it seems clear that the Zoho mail server does something different when forwarding the message: it also sets some parts of the headers (like smtp.mailfrom) to instruct the recipient mail server that the mail is sent from your email address, and not really from @id.apple.com. Google then validates not @id.apple.com, but (ME)+aml_=(MY DOMAIN)=gmail.com@(MY DOMAIN), which makes it work.
I will ask Gandi support.
Edit: after doing some research, it appears Zoho is implementing SRS (http://www.openspf.org/SRS) as the diagrams in the web page show, whereas Gandi is not. I just learned a new thing!
Thanks again.
Looking at the headers, it seems clear that the Zoho mail server does something different when forwarding the message: it also sets some parts of the headers (like smtp.mailfrom) to instruct the recipient mail server that the mail is sent from your email address, and not really from @id.apple.com. Google then validates not @id.apple.com, but (ME)+aml_=(MY DOMAIN)=gmail.com@(MY DOMAIN), which makes it work.
I will ask Gandi support.
Edit: after doing some research, it appears Zoho is implementing SRS (http://www.openspf.org/SRS) as the diagrams in the web page show, whereas Gandi is not. I just learned a new thing!
Thanks again.
I've just learned something too :)
I've known for years it "just works", but now I know why :D
I've known for years it "just works", but now I know why :D
Forwarding. So, I haven't actually had the SPF issue you mention. I stopped using that setup about 2 years ago in favour of my proper gsuite account and I believe that, at some point since that time, Google's setup changed causing this difference in validation. I had heard of some setups being grandfathered in. I was at the time using the "additional account" email setting in gmail.
So in the emails I would send, my @gmail.com would actually appear in the headers, even though the From: was my custom domain.
I don't know if this setup is still available.
So in the emails I would send, my @gmail.com would actually appear in the headers, even though the From: was my custom domain.
I don't know if this setup is still available.
Does anyone have experience with getting their paid G Suite account locked? If paying them means that I can un-suspend my Google Account easily, I would be more than happy to shell out $5/month just for that privilege.
In GSuite, you don't get your "account" locked because the account is the responsibility of the organization you sign up with. Now, you might get your organization kicked out as a customer if you really piss Google off (you'd have to do some pretty illegal shit I suspect).
There's definitely less risk and more recourse as you would be able to talk to customer support should issues arise. I absolutely think that paying for GSuite is very much a way to "vote with your wallet" that you are willing to pay for support for a critical account.
There's definitely less risk and more recourse as you would be able to talk to customer support should issues arise. I absolutely think that paying for GSuite is very much a way to "vote with your wallet" that you are willing to pay for support for a critical account.
A few options:
+ Use another email service, such as Fastmail or GSuite (has Gmail!)
+ Get Gmail to do POP3. The password will only be exposed to Google, so the attack surface is low.
+ Set up an email filter in Gandi to forward to your Gmail.
+ Use another email service, such as Fastmail or GSuite (has Gmail!)
+ Get Gmail to do POP3. The password will only be exposed to Google, so the attack surface is low.
+ Set up an email filter in Gandi to forward to your Gmail.
I have not heard about Gandi, added it to my list of useful products. Thanks for discovering.
If you will be thinking about changing email service, here I've made a short review on most popular options: https://tarasmatsyk.com/posts/3-how-to-custom-email-address/
PS. There are more services which I review in next posts, currently most popular ones are covered.
If you will be thinking about changing email service, here I've made a short review on most popular options: https://tarasmatsyk.com/posts/3-how-to-custom-email-address/
PS. There are more services which I review in next posts, currently most popular ones are covered.
Is it normally quite easy to get your account unsuspended, or are there cases where it's an absolute final decision and you can't do anything?
This is assuming that you didn't deliberately do anything bad/illegal, but maybe you accidentally violated the ToS or your test emails were detected as spam.
This is assuming that you didn't deliberately do anything bad/illegal, but maybe you accidentally violated the ToS or your test emails were detected as spam.
I luckily never got my account suspended, but I've certainly done some minor violations of the TOS here and there (e.g. in college I downloaded a few copyrighted books and stored them in my Drive for my strictly personal consumption, and I'm sure those files could be fingerprinted and detected by Google, ...).
From what I've heard, when it happens it's not quite easy to regain access, in the best case it takes many days, and the frustrating thing is that there's no support to which you can appeal to, you're mostly left to yourself (e.g. Google employees connections, twitter noise, ...).
I find it silly that my electronic identity would be held hostage in such cases, so with an external domain I can flip a switch and keep receiving my email somewhere else.
From what I've heard, when it happens it's not quite easy to regain access, in the best case it takes many days, and the frustrating thing is that there's no support to which you can appeal to, you're mostly left to yourself (e.g. Google employees connections, twitter noise, ...).
I find it silly that my electronic identity would be held hostage in such cases, so with an external domain I can flip a switch and keep receiving my email somewhere else.
I also primarily use a 'flip a switch' setup specifically for this reason. To be honest I think that using your own domain name is the only way to go these days, as even other email providers can have issues resulting in you being completely locked out.
At least with a domain name you have multiple paths for recourse or regaining access should something go wrong (domain registrar, hosting provider, DNS provider, mail provider, etc).
Now that I've moved to Cloudflare Registrar, I guess in some ways I've made it harder for myself should something go wrong, since my domain and DNS are now hosted in the same place.
At least with a domain name you have multiple paths for recourse or regaining access should something go wrong (domain registrar, hosting provider, DNS provider, mail provider, etc).
Now that I've moved to Cloudflare Registrar, I guess in some ways I've made it harder for myself should something go wrong, since my domain and DNS are now hosted in the same place.
I've already started moving stuff to GMX (Mail), SpiderOak (Drive), DDG (Search), !Chrome (Browser). Been on Android (non Google phone), trying to work without Google Play Services - contemplating switching to iOS or Jolla. I wish that Apple Maps/Here Maps were on more platforms.
While admirable, I’m not there yet: let’s just say I don’t have a problem at the moment with Google reading all my activity (I like the ecosystem too much), as long as I can take out my data periodically (Takeout works well enough for me) and an account suspension wouldn’t mean me losing my email address for more than a few hours (which is what I’m trying to fix now).
I heavily depend on Gmail and still plan on using it since I love the interface and the mobile app, but I don't want to risk the potential massive damages of Google deciding to inadvertently suspend my account for TOS violation, locking me out of my other 300+ accounts that depend on my email address.
So, on top of doing periodic data exports with Google Takeout, I also did the following:
1) Registered "lastname.com" domain with Gandi (and enabled u2f authentication)
2) Setup email forwarding of "[email protected]" to my Gmail address in Gandi
3) Gradually plan on moving all my identity to "[email protected]", while using Gmail as backend
The problem is, I don't like the email forwarding feature, it screws up SPF validations since now there is one mail server in the middle (the Gandi one) that forwards mail it isn't supposed to according to SPF, so all the emails forwarded to Gmail become essentially a SOFTFAIL/NEUTRAL/FAIL (and they show up as that).
At the same time, I don't want to create a full mail account on Gandi and then enable the POP3 fetching from Gmail, because that feels very insecure (Gandi doesn't have 2FA/access auditing for mail accounts).
If I want to stay in the Google ecosystem, do I really have to convert to GSuite? Any suggestion?
Thanks