20% of requests for Wikimedia Commons are for one image of a flower(phabricator.wikimedia.org)

1,388 pointsby IfOnlyYouKnewhá 5 anos362 comments

phabricator.wikimedia.org

20% of requests for Wikimedia Commons are for one image of a flower

https://phabricator.wikimedia.org/T273741

362 comments

At the height of the browser wars I once woke up to Microsoft hotlinking a small button for downloading our software from the MSN homepage. I tried to reach someone there for hours but nobody cared enough to do something about it. The image was small (no more than a few K), but the millions of requests that page got were enough to totally kill our server.

Finally, I replaced the image on there with a 'Netscape Now' button. Within 15 minutes the matter was resolved.

IfOnlyYouKnewhá 5 anos

Wikimedia is unique in running some of the most popular websites with open access to almost all systems. As someone who has never been on the inside of FAANG, I found it rather interesting to browse around the backend infrastructure.

See, for example, their statistics at https://grafana.wikimedia.org/d/000000102/production-logging...

tomglynchhá 5 anos

Just added this comment on the issue:

Hi all, I've been doing a bit of research into possible apps that could be causing this and found two potential culprits that I am currently investigating.

The first is Mitron TV, an Indian TikTok alternative which was made available again on the app store June 6th (https://indianexpress.com/article/technology/tech-news-techn...).

The second is Say Namaste, an Indian Zoom alternative which was launched on the app stores June 9th (https://indianexpress.com/article/technology/tech-news-techn...).

Both fall into the timeline of huge increases, have millions of users and may be using '1280px-AsterNovi-belgii-flower-1mb.jpg' to check the users internet connection - especially for Say Namaste to ensure video connectivity. I've reached out to some developers at both companies and will report back. Let me know your thoughts.

EDIT: I have also noticed the dates match the reopening after lockdown for the whole of India: "This first phase of reopening was termed as "Unlock 1.0"[13] and permitted shopping malls, religious places, hotels and restaurants to reopen from *8 June*." (https://en.wikipedia.org/wiki/COVID-19_lockdown_in_India#Unl... )

Tom

bombcarhá 5 anos

I had some random images on a web server years ago - and noticed that something like 99% of my traffic was one image - and searching through refers I realized I was the #1 hit on google images for robot attack cat.

Simpler times.

m463há 5 anos

The "traditional" way of fixing this would be a goatse.cx redirect of the image.

I'm sure there is a more enlightened fix.

gertlexhá 5 anos

After realizing "wiki[p|m]edia" and "flower" triggered a specific image in my head I was guessing it would be a yellow flower, this one in the corner of https://www.mediawiki.org/wiki/MediaWiki but nope, more interesting than that!

dmurrayhá 5 anos

20% of Wikipedia Commons requests to their Singapore servers (EQSIN), not globally. That's still a lot, of course.

dayzehá 5 anos

Sukhbir Singh just commented: Thank you everyone for the comments and suggestions. I just wanted to share that we have identified the app and will update this task tomorrow. (And yes, it is a mobile app.)

V-2há 5 anos

"Please avoid adding drive-by comments such as "hello from Hacker News" to this task as they are not helpful. Thank you"

Why would anyone do such stuff is, as usual, beyond me...

PS. "First!"

batch12há 5 anos

Superficial reversing shows that the ravn app mentioned, com.app.rcn may use the file as part of a speedtest:

com.app.rcn/smali/com/app/rcn/utils/InternetSpeedCalculator.smali: "hxxps://upload.wikimedia.org/wikipedia/commons/1/16/AsterNovi-belgii-flower-1mb.jpg"

edit: defanged the link to maybe save the wikimedia team some bytes

crazygringohá 5 anos

I'll just be looking forward to the follow-up post on HN announcing when they figure out what the culprit was!

Per the comments, right now the top suspect seems to be the app "Josh" or another TikTok clone because of how traffic surged immediately after the TikTon ban:

https://twitter.com/bwaber/status/1358915338637873154

mark-rhá 5 anos

Reminds me of the time Netgear routers were hardcoded with the IP address of a NTP server at the University of Wisconsin. https://en.wikipedia.org/wiki/NTP_server_misuse_and_abuse#Ne...

eyelidlessnesshá 5 anos

You gotta respect the suggested approach to take preventive measures by banning requests to this individual image without a User Agent header and to try to identify who might be affected. I’m sure I’m not the only one here who would just treat it as abuse and ban without followup.

bluntehá 5 anos

And now thanks to this HN post, 21% of requests are for that same flower!

dr_dshivhá 5 anos

This happening now! Some suspect a failure in a CV training pipeline. Others suggest an extremely popular app with a hotlinked image.

tchallahá 5 anos

Could it be the "Good Morning" like greetings on WhatsApp gone viral ?

https://www.wsj.com/articles/the-internet-is-filling-up-beca...

blakesterzhá 5 anos

They did figure it out, a popular chat app in India (they won't name yet) fetches the image but does not display it.

https://phabricator.wikimedia.org/T273741#6815828

simonebrunozzihá 5 anos

Here's the flower in question [0].

[0]: https://upload.wikimedia.org/wikipedia/commons/thumb/1/16/As...

bpicolohá 5 anos

Huh, I worked on a site with a similar issue in ~2019. A massive flood of traffic for a single site from Indian mobile apps (~15kqps at peak iirc).

I think it ended up being a sort of mobile-based botnet with a bizarre target, which luckily was deduced from some of the headers sent (they all had a random common header).

newsclueshá 5 anos

Example code that gets copy pasted into production app somewhere?

BugsJustFindMehá 5 anos

And of course, now that the image is linked in the report, I've just added an additional request for it by clicking.

Donckelehá 5 anos

“ Thank you everyone for the comments and suggestions. I just wanted to share that we have identified the app and will update this task tomorrow. (And yes, it is a mobile app.)”

owlninjahá 5 anos

>>Thank you everyone for the comments and suggestions. I just wanted to share that we have identified the app and will update this task tomorrow. (And yes, it is a mobile app.)

Looks like we will know soon.

ramraj07há 5 anos

I'm in India now, is it possible for me to install some traffic snooper and monitor if any wikimedia requests go out? I can then install some popular apps and see if anything bites!

andrewmattehá 5 anos

90M requests daily from India? I wonder if KaiOS is checking whether it's got internet access.

rozabhá 5 anos

This page someone noticed is very interesting.

https://newshimalaya.com/2021/02/09/%E2%9A%93-t273741-invest...

I was sure I'd seen this website before, and sure enough, it's scraping and rehosting almost everything that's posted on HN...

aquadrophá 5 anos

It's not 20% of all requests, it's 20% of media requests to one of the clusters (it's said in the issue description) There are 5 clusters.

kayxsprehá 5 anos

I remember that MediaWiki installation allowed the configuration that essentially permits the use of Commons files, albeit in that case, the file will be downloaded and cached in the Wiki's own server [1].

That being said, though the image wasn't hotlinked directly, they expressed concerns of DDOS and the possible costs the Foundation has to incur from each load (they even pointed out that it's "fair and reasonable" to point donation link to them).

I would be interested to see how the licensing issue will be handled, though. The photographer licensed this photo as GFDL/CC BY-SA 3.0 [2], and hotlinking may break the term of these licenses.

1: https://www.mediawiki.org/wiki/InstantCommons

2: https://commons.wikimedia.org/wiki/File:AsterNovi-belgii-flo...

[deleted]há 5 anos

blurbleblurblehá 5 anos

Tragedy of the commons