Changing the Android captive portal page (2020)(encrypted.at)
encrypted.at
Changing the Android captive portal page (2020)
https://www.encrypted.at/android-captive-portal/
13 comments
> And yet I am still tempted to add this 204 service, but maybe then I also don't have internet on my phone anymore because it keeps believing it's behind a captive portal when the service is offline
Try it. Normally Android will say "This connection doesn't give you internet, connect anyway?" then you press yes, and everything works. (Except if you're on a captive portal, in which case you'll need to type example.com in your browser to access the captive portal).
Try it. Normally Android will say "This connection doesn't give you internet, connect anyway?" then you press yes, and everything works. (Except if you're on a captive portal, in which case you'll need to type example.com in your browser to access the captive portal).
What do you use to self-host your grocery lists?
I've been using the homeassistant shopping list integration but I'm planing to switch to "Specifically Clementines" [1] it's not as complete as grocy [2] but that's a feature. My partner will add to a list, not use an inventory managment system.
1. https://github.com/davideshay/groceries
2. https://github.com/grocy/grocy
1. https://github.com/davideshay/groceries
2. https://github.com/grocy/grocy
Thanks for sharing this. I've tried to get on with Grocy but it's UI really put me off, it felt very unintuitive, cluttered and a bit like an app that had long since outgrown the initial hacked together layout.
Yeah I had a similar reaction to Grocy, just too much. If I was single and on a super tight budget like I was as a student, maybe useful. Now it's just too stressful
not OP, but I use tandoor to host my recipes and it generates a shopping list based on what I've picked to cook this week
this was nearly impossible to find on duckduckgo https://tandoor.dev What a name, the only thing you get when looking for 'tandoor' or 'tandoor grocery list' are recipes for tandoori
In my quest, I've also come across other promising-looking things https://kitchenowl.org and https://recipesage.com
In my quest, I've also come across other promising-looking things https://kitchenowl.org and https://recipesage.com
dead trees, strained
Something I wrote across about eight years, trying different things and never cleaning up unused parts, so the code is a mess but this is it: https://github.com/lgommans/mpgroceries
The name (multiplayer groceries) refers to the original idea of having the software assign random items to random people, so you can both go through the store independently and not get duplicate items, for maximum efficiency, but I don't think I've used that more than once.
Probably I could write a small book about all the things that you shouldn't do, like making a store layout and mapping where all the items are and then making it sort by nearest item...which, turns out, is somehow always on the other side of a wall. This routing is better than nothing, but it turns out that categorizing ("freezer", "candy", "fruit/veg") is: much simpler for the user, portable across stores, and more effective for routing as well.
I've started on a rewrite with a friend, but it's very early stages and the primary purpose of the new project is to turn a list of meals into a grocery list rather than being a grocery list that you add random items to: https://codeberg.org/lucg/MealsUnveiled The old project can also add a recipe to your grocery list, but I never wrote the code to add up amounts (it'll ignore duplicates instead)
The name (multiplayer groceries) refers to the original idea of having the software assign random items to random people, so you can both go through the store independently and not get duplicate items, for maximum efficiency, but I don't think I've used that more than once.
Probably I could write a small book about all the things that you shouldn't do, like making a store layout and mapping where all the items are and then making it sort by nearest item...which, turns out, is somehow always on the other side of a wall. This routing is better than nothing, but it turns out that categorizing ("freezer", "candy", "fruit/veg") is: much simpler for the user, portable across stores, and more effective for routing as well.
I've started on a rewrite with a friend, but it's very early stages and the primary purpose of the new project is to turn a list of meals into a grocery list rather than being a grocery list that you add random items to: https://codeberg.org/lucg/MealsUnveiled The old project can also add a recipe to your grocery list, but I never wrote the code to add up amounts (it'll ignore duplicates instead)
The fridge. Integrations aren’t great, but 9x9’s uptime is incredible.
The attack surface could probably be lowered a bit by configuring the HTTP server (nginx, etc.) to return 204 instead of running PHP behind it.
Still, neat :)
Still, neat :)
I don't think this has anything to do with "attack surface". This is simply to avoid google collecting ip addresses.
Running nginx might be marginally faster than PHP, but I believe the latency to your server will be far greater than the time it takes the server "to spin up the php interpreter" (doesn't even happen anymore when using FastCGI).
Running nginx might be marginally faster than PHP, but I believe the latency to your server will be far greater than the time it takes the server "to spin up the php interpreter" (doesn't even happen anymore when using FastCGI).
Not to be conspiratorial, but captive portals always felt like a little broken piece of internet technology that's never been fixed bc it benefits Google
This has a very cryptic failure mode if you travel to China. Android will just tells you you can't connect to the internet :)
It's pretty easy to work around. You just go to detectportal.firefox.com/
But naturally you need to know what's gone wrong in the first place !
This has a very cryptic failure mode if you travel to China. Android will just tells you you can't connect to the internet :)
It's pretty easy to work around. You just go to detectportal.firefox.com/
But naturally you need to know what's gone wrong in the first place !
> Not to be conspiratorial, but captive portals always felt like a little broken piece of internet technology that's never been fixed bc it benefits Google
Against my better judgement and for my grim fascination, I'd love to hear a full explanation of this conspiracy theory. Can you elaborate?
Against my better judgement and for my grim fascination, I'd love to hear a full explanation of this conspiracy theory. Can you elaborate?
> bc it benefits Google
I doubt this is the reason. It seems more likely that no one has bothered to fix it. I agree that captive portals are a big hack with poor UX. This auto-detection does mitigate it but it is still very annoying especially if you have secure setups like encrypted DNS so the basic hijacking doesn't work properly.
I doubt this is the reason. It seems more likely that no one has bothered to fix it. I agree that captive portals are a big hack with poor UX. This auto-detection does mitigate it but it is still very annoying especially if you have secure setups like encrypted DNS so the basic hijacking doesn't work properly.
I find NeverSSL.com is a short, easy-to-remember URL for "oh crap am I in a captive portal or not".
How do captive portals benefit Google?
> But, why should I change it?
> Well, on Android 6 or higher, every time you switch networks, your phone tries to access the following URL:
> http://connectivitycheck.gstatic.com/generate_204
> As we can see, gstatic.com – Google’s servers. We might not want Google to know our new IP address or when we switched between networks. So… Why don’t we just self-host it? It’s pretty easy.
> Well, on Android 6 or higher, every time you switch networks, your phone tries to access the following URL:
> http://connectivitycheck.gstatic.com/generate_204
> As we can see, gstatic.com – Google’s servers. We might not want Google to know our new IP address or when we switched between networks. So… Why don’t we just self-host it? It’s pretty easy.
Yeah, that's not Google having a say on whether captive portals exist and deciding to keep them there, that's just Google deciding that it wants to know your IP whenever you connect to a network.
I don't know why this is such a big issue, given that they have your more or less exact location at all times, but changing it is so easy that, why not.
I don't know why this is such a big issue, given that they have your more or less exact location at all times, but changing it is so easy that, why not.
> that's just Google deciding that it wants to know your IP whenever you connect to a network.
No, it's Google wanting Android users to be notified when they sign in to a wifi network that uses a captive portal, rather than having all network requests silently fail because they're being redirected.
Fixing the (generally terrible) user experience of captive portals is the primary goal of this feature. Any data collection is secondary.
No, it's Google wanting Android users to be notified when they sign in to a wifi network that uses a captive portal, rather than having all network requests silently fail because they're being redirected.
Fixing the (generally terrible) user experience of captive portals is the primary goal of this feature. Any data collection is secondary.
gstatic.com is a cookieless domain. The persistent push connection needed by FCM (like APNS on iPhones) can tie the IP to a specific account, so this captive portal check gives them less information than they already have. Luckily on Android, you can use UnifiedPush and run your own push server to gain actual privacy.
Google already knows every IP address. There is a fixed amount. Anyone can send a request to that address.
This doesn't give Google any new information either.
This doesn't give Google any new information either.
I guess that is true, but the IP address space today is so large as to be unscannable. Practically speaking, you have to have some evidence that some host is there before you can start prodding at that host.
Do you mean the IPv6 space? The full Ipv4 space has been practically scannable for quite a while now.
https://zmap.io/
ZMap is a fast single-packet network scanner optimized for Internet-wide network surveys. On a computer with a gigabit connection, ZMap can scan the entire public IPv4 address space on a single port in under 45 minutes. With a 10gigE connection and PF_RING, ZMap can scan the IPv4 address space in 5 minutes.
https://zmap.io/
ZMap is a fast single-packet network scanner optimized for Internet-wide network surveys. On a computer with a gigabit connection, ZMap can scan the entire public IPv4 address space on a single port in under 45 minutes. With a 10gigE connection and PF_RING, ZMap can scan the IPv4 address space in 5 minutes.
Yes of course. Most devices on the Internet don't have IPv4 addresses.
AOSP actually has handling when it detects a SIM MCC matching China: https://cs.android.com/android/platform/superproject/+/main:...
This is done on GrapheneOS by default. It also allows you to change the URLs without connecting your phone and using adb.
https://grapheneos.org/faq#default-connections
https://grapheneos.org/faq#default-connections
It's amazing we're still using captive portals instead of a DHCP option that's pushed to the client in 2023. Wtf.
The point of captive portals is to get the user to either click "I agree" (to the terms,..) or enter a code/pay for the internet.
Putting the url in a dhcp option wouldn't change that, wouldn't work with older devices (so a redirect would still be needed), and the only change is, that you'd have two ways to reach the same captive portal page, that you'd have to configure on two (or three, with ipv6) devices/config files.
Putting the url in a dhcp option wouldn't change that, wouldn't work with older devices (so a redirect would still be needed), and the only change is, that you'd have two ways to reach the same captive portal page, that you'd have to configure on two (or three, with ipv6) devices/config files.
It would nevertheless be the cleaner option. Had this been proposed and implemented six years ago, by now almost all devices would have supported it.
IPv6 standard came out in 1998, and a lot of devices still don't support it.
I don't see a lot of devices that don't support IPv6 these days. Linux supports it, so most terrible IoT crapware works perfectly fine, and esp-idf enables it by default (solving the problem for a ton of cheap smart home crap).
There are still devices that don't support hardware IPv6 acceleration but at this point even those are becoming rare.
There are still devices that don't support hardware IPv6 acceleration but at this point even those are becoming rare.
A bunch of devices don't... IoT devices based on esp8266 or esp32 don't (from washing machines to sous vide cookers), many ip cameras, sensors, smart locks etc., don't. Yeah, linux itself might support it, but when compiling the image for those few megs of flash, developers just disable it, because well.. everyone still has ipv4 available.
I'm using Graphene, but with the default Google captive portal settings. While yes, I am technically leaking my current public IP to Google servers, what does it actually tell them apart from the IP?
It's a generic user agent I believe and there's billions of (simple) HTTP requests hitting that endpoint. If you're using a stock Android (or even worse, like Samsung) it's the Play services and unkillable vendor background apps you should be worried about.
I'd argue it's a lot more conspicuous to network operators if you're using non-standard captive URLs.
It's a generic user agent I believe and there's billions of (simple) HTTP requests hitting that endpoint. If you're using a stock Android (or even worse, like Samsung) it's the Play services and unkillable vendor background apps you should be worried about.
I'd argue it's a lot more conspicuous to network operators if you're using non-standard captive URLs.
On the one hand, this denies Google the metadata from your connectivity check.
On the other hand, using your own site uniquely identifies you on each network (negating the privacy improvements through random MAC addresses and such).
I would much rather use a site I can trust. Mozilla offers a 204 service for Firefox, for example, that's a lot less obvious for trackers.
You could also try to confuse the system by using iOS' 204 URL for your Android devices.
On the other hand, using your own site uniquely identifies you on each network (negating the privacy improvements through random MAC addresses and such).
I would much rather use a site I can trust. Mozilla offers a 204 service for Firefox, for example, that's a lot less obvious for trackers.
You could also try to confuse the system by using iOS' 204 URL for your Android devices.
Not all of these return 204, some just return 200 with an "OK" or "success".
$ curl --verbose "https://detectportal.firefox.com"
< HTTP/2 200
successNote that ROMs like GrapheneOS or /e/OS do not connect to the Google connectivitycheck server (instead they use their own servers).
Except /e/OS then throws that out the window because it installs and enables microG by default which immediately connects directly to Google: https://github.com/microg/GmsCore/wiki/Google-Network-Connec...
And /e/OS is still 224 days behind Chromium updates: https://divestos.org/misc/ch-dates.txt
And /e/OS is still 224 days behind Chromium updates: https://divestos.org/misc/ch-dates.txt
> Except /e/OS then throws that out the window
I checked your link, I don't see the connectivity check listed there. Can you please elaborate?
> And /e/OS is still 224 days behind Chromium updates
Did I mention anything at all about Chromium updates? Or is it just to add a link to your website?
I checked your link, I don't see the connectivity check listed there. Can you please elaborate?
> And /e/OS is still 224 days behind Chromium updates
Did I mention anything at all about Chromium updates? Or is it just to add a link to your website?
> check listed there
/e/OS changes the connectivity check from Google to their own server, great, but then it still connects to Google services via microG. So why bother changing the connectivity check?
> Chromium updates
Your comment was recommending an operating system that is actively harmful to its users by falling behind basic security updates that every single other OS has done.
edit: aside from that, what are your thoughts on their upcoming license key system for OTAs that give users a persistent identifier? https://gitlab.e.foundation/e/os/android_packages_apps_Updat...
/e/OS changes the connectivity check from Google to their own server, great, but then it still connects to Google services via microG. So why bother changing the connectivity check?
> Chromium updates
Your comment was recommending an operating system that is actively harmful to its users by falling behind basic security updates that every single other OS has done.
edit: aside from that, what are your thoughts on their upcoming license key system for OTAs that give users a persistent identifier? https://gitlab.e.foundation/e/os/android_packages_apps_Updat...
> but then it still connects to Google services via microG.
So you actually do agree that /e/OS changes the connectivity check, don't you?
> Your comment was recommending an operating system
Not at all. My comment was mentioning that at least some custom ROMs I know do remove the connectivity check, which is the topic of that post ("Changing the Android captive portal page").
In that particular post, I did not recommend /e/OS, I just stated a fact. But if it's about recommendation, I would recommend against DivestOS, because the author is really toxic. That may be harmful to users, too.
So you actually do agree that /e/OS changes the connectivity check, don't you?
> Your comment was recommending an operating system
Not at all. My comment was mentioning that at least some custom ROMs I know do remove the connectivity check, which is the topic of that post ("Changing the Android captive portal page").
In that particular post, I did not recommend /e/OS, I just stated a fact. But if it's about recommendation, I would recommend against DivestOS, because the author is really toxic. That may be harmful to users, too.
Or switch to a custom OS that provides controls for this and more:
- my DivestOS (includes nine presets): https://divestos.org/pages/network_connections
- GrapheneOS (includes two presets): https://grapheneos.org/faq#default-connections
also if you do want to setup your own portal there is a simpler method than invoking php each time:
- my DivestOS (includes nine presets): https://divestos.org/pages/network_connections
- GrapheneOS (includes two presets): https://grapheneos.org/faq#default-connections
also if you do want to setup your own portal there is a simpler method than invoking php each time:
RewriteEngine on
RewriteRule ^/generate_204$ - [END,NE,R=204]Note that some captive portals will break here, because they will use various kinds of redirections for everything except the well-known Google and Apple captive portal check URLs.
Wouldn't such redirections break Google/Apple's checks? The whole point of this URL is to test if you can actually connect to the internet - if random urls are intercepted/broken then the test should fail.
No, those redirections explicitly exempt Google/Apple checks by not redirecting their URLs. All other URLs are redirected to show advertisements usually, either injected into the site you wanted to visit or as a replacement, and a please-login-and-pay-here page. That devices are fooled to assume that internet is working is intentional, otherwise those devices would immediately disconnect.
Yes, what you are describing is breaking the Google/Apple checks by making those devices think that Internet access is available when its not.
gstatic.com - the backbone of the internet.
Just marvel at how much data/metadata they get from that little trick when multiplied by millions users.
I just realized, they (or a government mandate) could force a redirect under specific circumstances to a site under google.com, and if they're using the browser engine for the request, they'd get the cookies of that user as soon as they get on the internet. Maybe even trigger an exploit in the browser.
This is the most fantastic way to target and exploit any single Android user anywhere in the world that I've ever heard of. Automatic, hard to avoid, easy to implement, and the user has no idea.
This is the most fantastic way to target and exploit any single Android user anywhere in the world that I've ever heard of. Automatic, hard to avoid, easy to implement, and the user has no idea.
> could force a redirect under specific circumstances to a site under google.com, and if they're using the browser engine for the request, they'd get the cookies of that user as soon as they get on the internet
At first, I thought this can't be true because, surely, Google marks its cookies as HTTPS-only, right? So I checked, and turns out about half the cookies google.com has in my browser are not HTTPS-only. In fact, the HTTPS-only cookies it does have seem to be the same set of cookies, just with a '__Secure-' prefix. Similarly, about half (different set) of the cookies JS accessible.
At first, I thought this can't be true because, surely, Google marks its cookies as HTTPS-only, right? So I checked, and turns out about half the cookies google.com has in my browser are not HTTPS-only. In fact, the HTTPS-only cookies it does have seem to be the same set of cookies, just with a '__Secure-' prefix. Similarly, about half (different set) of the cookies JS accessible.
Then I remembered how much trouble I already have when my server is offline for any reason: no email, no grocery list, no backups, no VPN, no git, no link shortener, no IRC, no website with various tools like 'what is my IP' or bidirectional latency testing... self hosting is great but you're also on the hook for everything and it all has a single point of failure unless you want to run a dual-uplink server farm.
And yet I am still tempted to add this 204 service, but maybe then I also don't have internet on my phone anymore because it keeps believing it's behind a captive portal when the service is offline