I Could Rewrite Curl (2021)(daniel.haxx.se)
daniel.haxx.se
I Could Rewrite Curl (2021)
https://daniel.haxx.se/blog/2021/05/20/i-could-rewrite-curl/
11 comments
In the defense of some of the particularly jerk-inducing ones, they're conflating libcurl with curl.
Sure if you want to fetch a file from the internet you could do it with one line (see the person I'm replying to);
I'd love to see the look on some of these people's faces when they find out that tool/software/whatever they use is actually using libcurl under the hood.
Good luck even getting on the internet in the first place without that router, switch, PC, modem or anything else that's also running curl/libcurl.
I don't know whether it's just my age showing, but thinking you could do anything remotely like curl in 3 days is .... well, my 1 year old probably talks more sense when he says "no".
Sure if you want to fetch a file from the internet you could do it with one line (see the person I'm replying to);
I'd love to see the look on some of these people's faces when they find out that tool/software/whatever they use is actually using libcurl under the hood.
Good luck even getting on the internet in the first place without that router, switch, PC, modem or anything else that's also running curl/libcurl.
I don't know whether it's just my age showing, but thinking you could do anything remotely like curl in 3 days is .... well, my 1 year old probably talks more sense when he says "no".
> I'd love to see the look on some of these people's faces when they find out that tool/software/whatever they use is actually using libcurl under the hood.
Python dependencies (does not include curl)
https://devguide.python.org/getting-started/setup-building/i...
The "requests" module in Python (does not use curl)
https://github.com/psf/requests
Python dependencies (does not include curl)
https://devguide.python.org/getting-started/setup-building/i...
The "requests" module in Python (does not use curl)
https://github.com/psf/requests
yes, python and some other languages have more-or-less re-implemented parts of libcurl. requests is one example created by a particularly prolific author and it has been evolving for over a decade.
that's perfect evidence that, no, a common developer cannot "re-write" curl in a few days.
that's perfect evidence that, no, a common developer cannot "re-write" curl in a few days.
I was going through my Prius's "software license information" a while ago and I noticed the daniel@haxx email.
No way would I run software in my car that someone hacked together in 3 days.
No way would I run software in my car that someone hacked together in 3 days.
I was updating my car a few months ago and found it runs Linux for the user-facing systems, and of course uses libcurl, according to the open source licenses.
I think this problem is similar to the OpenSSL problem. The idea being that we have a library that has a giant attack surface because it does a LOT of different things and attackers find clever ways to abuse those extra features that most users don't actually want enabled.
Your car probably wants to use some relatively pedestrian things like HTTPS, but do you think it should have any capability of doing GOPHER, SMBS, SFTP, POP3, LDAP, IMAP, TELNET, TFTP, SCP or any of the other stuff curl supports out of the box?
We can hope that the version of curl used for the car was compiled without support for some of these protocols. I don't even know if we have a good way to verify that. More likely is that whatever the stock configuration (all the protocols) is what is being used in most applications.
Your car probably wants to use some relatively pedestrian things like HTTPS, but do you think it should have any capability of doing GOPHER, SMBS, SFTP, POP3, LDAP, IMAP, TELNET, TFTP, SCP or any of the other stuff curl supports out of the box?
We can hope that the version of curl used for the car was compiled without support for some of these protocols. I don't even know if we have a good way to verify that. More likely is that whatever the stock configuration (all the protocols) is what is being used in most applications.
> No way would I run software in my car that someone hacked together in 3 days
So no Tesla for you then.
So no Tesla for you then.
100 lines is such an amateur number. I can actually do this in 1 lines of shell.
curl -O $1 $2
Checkmate 100line python guy!
curl -O $1 $2
Checkmate 100line python guy!
It's worth noting that Microsoft actually does shim "curl" in Powershell to basically be a light wrapper around the "Invoke-WebRequest" object. Is it a full drop-in replacement for curl? Nope, but it does replace it in the simple context(s) of stuff like "curl -O foo bar"
Powershell is really weird in the sense that I can sometimes use Unix-like command like cp, mv, rm... I know it's probablly a completely different thing behind the curtain but it's amusing.
Or, with a dependency on libcurl :)
Also, it's not immediately obvious in the various libraries how to do something right. Most python/requests examples you'll find to do that simple "download a file", for example, would run out of memory on a large file.
Also, it's not immediately obvious in the various libraries how to do something right. Most python/requests examples you'll find to do that simple "download a file", for example, would run out of memory on a large file.
AFAIK the "requests" module in python (the most popular one) doesn't actually depend on curl:
https://github.com/psf/requests
https://github.com/psf/requests
Ah, that's not what I was saying, should have worded it better. The statements were separate.
or wget https://example.com/foo.txt
like those 99% i never had a use for curl because wget covers all my use cases including some that curl doesn't.
but where curl is a swiss-army-knife, wget is a simple hunters or sailors knife. it cuts, and that's all it does.
everyone who complains about curl is not taking advantage of its features and should be using wget instead of complaining.
like those 99% i never had a use for curl because wget covers all my use cases including some that curl doesn't.
but where curl is a swiss-army-knife, wget is a simple hunters or sailors knife. it cuts, and that's all it does.
everyone who complains about curl is not taking advantage of its features and should be using wget instead of complaining.
That must be so demoralizing to read. Not just from the creators side but the maintainers too.
I've always felt really proud when contributing to an OS project and it's approved and merged etc. But never once have I thought to complain or even give my sometimes bad opinions on stuff. I don't know how OS providers and maintainers do it.
I've always felt really proud when contributing to an OS project and it's approved and merged etc. But never once have I thought to complain or even give my sometimes bad opinions on stuff. I don't know how OS providers and maintainers do it.
I hope not, and I doubt it. I mean, the people behind curl/libcurl and the people saying those things linked in OP are on different planes of existence.
It reminds me of a funny story from my university days (sorry I don't tell it too well); I'm in some electronics theory class, I forget the title, the lecturer is writing some equations on the board as he talks us through them;
the guy next to me nudges me and goes, what an idiot, that's not omega; he proceeds to put his hand up like a child, thrusting it repeatedly in the air, so he can share what an idiot this guy is with him and the class;
I ask him quietly not to do it, he ignores me, and carries on until he distracts the prof and the class; prof walks over and asks what's up?
Guy makes his point, loud enough for everyone to hear; prof smiles, hands him the chalk and sits down. Guy sits there for a minute looking confused so the prof gets back up, takes the chalk back and says loud enough for everyone;
"You don't want to teach this lecture? No? Ok. But you DO want to tell a Greek guy how to write Greek? grin" - Dr Ioannis Alexandrou
Great lecturer, cool guy, I hope he's doing well.
It reminds me of a funny story from my university days (sorry I don't tell it too well); I'm in some electronics theory class, I forget the title, the lecturer is writing some equations on the board as he talks us through them;
the guy next to me nudges me and goes, what an idiot, that's not omega; he proceeds to put his hand up like a child, thrusting it repeatedly in the air, so he can share what an idiot this guy is with him and the class;
I ask him quietly not to do it, he ignores me, and carries on until he distracts the prof and the class; prof walks over and asks what's up?
Guy makes his point, loud enough for everyone to hear; prof smiles, hands him the chalk and sits down. Guy sits there for a minute looking confused so the prof gets back up, takes the chalk back and says loud enough for everyone;
"You don't want to teach this lecture? No? Ok. But you DO want to tell a Greek guy how to write Greek? grin" - Dr Ioannis Alexandrou
Great lecturer, cool guy, I hope he's doing well.
>
"You don't want to teach this lecture? No? Ok. But you DO want to tell a Greek guy how to write Greek? grin" - Dr Ioannis Alexandrou
A person who knows Greek quite well once told me that I wrote some Greek letters wrongly. From her (linguistic) knowledge perspective about the Greek alphabet, she was surely right, but some Greek letters are very commonly written somewhat differently by mathematicians than by people who know Greek.
In this sense: I know quite well how some Greek letters are nearly always written by mathematicians, which can be a little bit different from how they are written by people who know Greek.
A person who knows Greek quite well once told me that I wrote some Greek letters wrongly. From her (linguistic) knowledge perspective about the Greek alphabet, she was surely right, but some Greek letters are very commonly written somewhat differently by mathematicians than by people who know Greek.
In this sense: I know quite well how some Greek letters are nearly always written by mathematicians, which can be a little bit different from how they are written by people who know Greek.
I understand how that can happen. This was as a simple case of, the lecturer had written a lowercase omega, and this guy was ignorant of Greek having a lower-case alphabet full-stop.
I wonder if it is demoralising. Imagine any project you worked on for multiple years - and someone suggests it could be rewritten (and better) in a long weekend. How would you feel and respond? I might laugh a bit, but I probably wouldn't spend much time on it.
The part of FOSS I think would be demoralising for me would be dealing with all the people who assume that the maintainer of a piece of software is a customer service operative who exists solely to reply to them in a timely fashion.
The part of FOSS I think would be demoralising for me would be dealing with all the people who assume that the maintainer of a piece of software is a customer service operative who exists solely to reply to them in a timely fashion.
My response would be...don't just talk about it, do it. Looking forward to your follow up post "I rewrote Curl". If it's better, we'll just use your version going forward.
Then maybe for kicks and giggles once a year: "Hi, just a quick followup...did you ever get around to rewriting Curl? Cause I know you could."
The best thing is to just ignore this completely though and focus on the stuff that you are doing.
Then maybe for kicks and giggles once a year: "Hi, just a quick followup...did you ever get around to rewriting Curl? Cause I know you could."
The best thing is to just ignore this completely though and focus on the stuff that you are doing.
The staggering ignorance, posted publicly must be demoralising after a while. E.G.
https://daniel.haxx.se/blog/wp-content/uploads/2023/03/Scree...
https://daniel.haxx.se/blog/wp-content/uploads/2023/03/Scree...
I imagine they have pretty thick skin. There's a lot of dumbasses on the internet.
If I were a maintainer of curl I would LOL at these comments and read them just for fun
Honestly, I do believe that at least some of the people would be able to rewrite what they use curl for in the mentioned weekend. In this sense I think they are mostly right from their perspective.
Of course, curl contains a lot more functionality, but the people who are very aware of it and actually use this functionality actively would not do such claims, because for their use cases, curl cannot be rewritten in a long weekend.
Of course, curl contains a lot more functionality, but the people who are very aware of it and actually use this functionality actively would not do such claims, because for their use cases, curl cannot be rewritten in a long weekend.
Curl filled a big missing hole. How can we code on plain text an HTTP Request in a reproducible and predictable way? The HTTP Protocol describe the request content (method, headers, body...) but do not specify how the call is done (real IP address, port, certificates, timeout, etc.). Curl filled this and I think now it's options are the de-facto standard for describe in text an HTTP Request.
Post prompted by this probably https://news.ycombinator.com/item?id=35228602
While not a rewrite - one recent tool for making http requests which i quite enjoy is:
https://github.com/ducaale/xh
It's basically python httpie rewritten in rust. my only gripe is that i keep forgetting that it exists - and that "xh" is for http and "xhs" is for https.
So i frequently end up with curl anyway:)
https://github.com/ducaale/xh
It's basically python httpie rewritten in rust. my only gripe is that i keep forgetting that it exists - and that "xh" is for http and "xhs" is for https.
So i frequently end up with curl anyway:)
I don't see it mentioned, but this is topical because curl turned 25 today:
https://github.com/curl/curl/discussions/10465
I could rewrite curl with very few lines of code using libcurl as a dependency...
Most of curl (or libcurl for that matter) included every edge case you could ever think of. I like that it packages a wide number of tools that fall under the topic of networking.
Anyone else surprised this isn't the announcement article for the rust rewrite (expected to be named "Rurl")?
I think that the bigger point of what the critics are saying is that Curl as it should be - a simple tool to download a file - is conceptually small.
Of course Curl has evolved into an enormous endeavor, just like the first kernels and browsers were tiny and now are impossible to rewrite because the scope has never stopped growing.
The critics are pointing out that the growth in scope is a smell. If your software went from "all features are used by 50% of users" to just 0.1% of users, then you're probably shaving yaks.
Of course Curl has evolved into an enormous endeavor, just like the first kernels and browsers were tiny and now are impossible to rewrite because the scope has never stopped growing.
The critics are pointing out that the growth in scope is a smell. If your software went from "all features are used by 50% of users" to just 0.1% of users, then you're probably shaving yaks.
Assuming this is the "99%" usage pattern:
Then, yes.