Python switching to Mercurial(mail.python.org)
mail.python.org
Python switching to Mercurial
http://mail.python.org/pipermail/python-dev/2009-March/087931.html
12 comments
Until we get excellent interop between git and mercurial I'm not buying the argument that git vs hg is anything like vi vs emacs.
We've got interop already. Some Mozilla devs use git for work. Official mozilla repo is mercurial, but devs can and do use git if they prefer: http://www.bluishcoder.co.nz/2008/12/changes-to-git-mirror-o...
Git has excellent interop with CVS, so I imagine excellent interop between Git and other DVSs is in our near future.
-- EDIT --
By excellent interop with CVS I obviously mean the one way CVS to Git. I'm hoping Git/Hg will go both ways.
-- EDIT --
By excellent interop with CVS I obviously mean the one way CVS to Git. I'm hoping Git/Hg will go both ways.
Git's pretty good going two way with Subversion. If nothing else, perhaps Subversion could be a poor man's bridge for the time being?
I had thought that git was rolling towards a dominant position in this space, however:
> At PyCon, Brett already announced that Git was no longer being considered -- while it has obviously many fans, it also provokes strong antipathies.
And I'm starting to see why. It has... issues. One I ran into recently:
http://book.git-scm.com/5_submodules.html
> It's not safe to run git submodule update if you've made and committed changes within a submodule without checking out a branch first. They will be silently overwritten:
"Silently overwritten" are not words you want to be associated with a version control system.
> At PyCon, Brett already announced that Git was no longer being considered -- while it has obviously many fans, it also provokes strong antipathies.
And I'm starting to see why. It has... issues. One I ran into recently:
http://book.git-scm.com/5_submodules.html
> It's not safe to run git submodule update if you've made and committed changes within a submodule without checking out a branch first. They will be silently overwritten:
"Silently overwritten" are not words you want to be associated with a version control system.
I think submodules are a bit of a failed experiment, yes. Or, perhaps, a work in progress. Their usability is just awful. That said, if you don't use them the rest of git works just fine. ;)
Does Mercurial support submodules or the equivalent? Not in core, from what I can tell, although a casual search of Google reveals the third-party "Forest Extension":
http://www.selenic.com/mercurial/wiki/index.cgi/ForestExtens...
There's some talk on a wiki about putting this extension's functionality into Mercurial proper, but the wiki link is broken. That doesn't bode well. I don't know Mercurial so I have no idea if Forest is actually being used by folks, if it's akin to git submodules or more akin to Braid, if it's got usability traps of its own, etc. The docs page is filled with TODOs.
Does Mercurial support submodules or the equivalent? Not in core, from what I can tell, although a casual search of Google reveals the third-party "Forest Extension":
http://www.selenic.com/mercurial/wiki/index.cgi/ForestExtens...
There's some talk on a wiki about putting this extension's functionality into Mercurial proper, but the wiki link is broken. That doesn't bode well. I don't know Mercurial so I have no idea if Forest is actually being used by folks, if it's akin to git submodules or more akin to Braid, if it's got usability traps of its own, etc. The docs page is filled with TODOs.
We use Forests and they are pretty awful. I would recommend against using that extension. It is awkward to work with, makes it hard to rollback your repos to a consistent state, and is painful when you have a change that applies to both repos.
Aha, perhaps I have found the Mercurial equivalent of git submodules! ;)
Although, actually, I'm pretty sure you can get git projects with submodules to roll back to consistent states. But it might take ten separate commands, five attempts, and a visit to the FAQ. Submodules are conceptually tricky and their workflow is a mess.
Although, actually, I'm pretty sure you can get git projects with submodules to roll back to consistent states. But it might take ten separate commands, five attempts, and a visit to the FAQ. Submodules are conceptually tricky and their workflow is a mess.
Sound exactly like forests. You CAN get back to a consistent state. But you have to be very careful look at the commit history and hg update a few times.
I've used forests on a few projects. On the upside, the commands don't cause unpleasant surprises; it's easy to guess what they'll do. But there are still far too many extra steps to do simple things like clone a remote forest and get all the sub-projects up to date. They need an "hg jfdi" command.
Forests are out of date, but IIRC mpm is planning on personally tackling submodule support for release some time this summer (whenever the 1.3 release schedule is, I don't remember offhand).
svn's externals work quite well, on the other hand. I know, I know, svn is so 2007, but after fiddling around a bit with git, my impression is that spending time writing code is an order (if not more) of magnitude more important thank screwing around with a version control system, and svn mostly 'just works'. I think that eventually things like git and hg will surpass it, but version control isn't where your startup is going to "make or break" it (unless you're github:-).
I've settled into a pretty happy workflow using Git as a local snapshot of an upstream Subversion repository using git-svn. With a bit of discipline, you can keep things nicely in sync, and get the benefits of cheap-as-hell branching and local/offline commits without forcing your entire project team to migrate to a new VCS.
I've got some notes on the subject on my blog:
http://rcoder.net/content/daily-git-svn
I've got some notes on the subject on my blog:
http://rcoder.net/content/daily-git-svn
Well, I know nothing of SVN externals. But I tend to believe this guy who says that SVN externals are inferior to Piston, and that Piston is the equivalent of git's Braid:
http://codeintensity.blogspot.com/2008/03/svn-externals-are-...
He strikes me as someone who has come by this lesson through direct, painful personal experience.
I've settled on Braid for the moment, and we'll see how that goes. Submodules offer more features, but at the expense of my ability to explain them to anybody else.
http://codeintensity.blogspot.com/2008/03/svn-externals-are-...
He strikes me as someone who has come by this lesson through direct, painful personal experience.
I've settled on Braid for the moment, and we'll see how that goes. Submodules offer more features, but at the expense of my ability to explain them to anybody else.
svn's externals work quite well
I found them quite painful. Links to external paths don't work, committing with multiple externals when one or more of the external repos are down has some sort of problem (though it's been a while and may have been fixed), doing anything from the external to the parent (like moving files) had some sort of problem as well (again, maybe fixed). It was enough of a problem that the project I was working on ended up with two repositories for the same code...and they would manually merge them periodically. Obviously sub-optimal and a big waste of time, but it was less trouble than the alternative of having every developer understand why the external repo had to be treated completely differently from the rest of the repository in many regards.
I found them quite painful. Links to external paths don't work, committing with multiple externals when one or more of the external repos are down has some sort of problem (though it's been a while and may have been fixed), doing anything from the external to the parent (like moving files) had some sort of problem as well (again, maybe fixed). It was enough of a problem that the project I was working on ended up with two repositories for the same code...and they would manually merge them periodically. Obviously sub-optimal and a big waste of time, but it was less trouble than the alternative of having every developer understand why the external repo had to be treated completely differently from the rest of the repository in many regards.
Ok, so they're not perfect, but "hassle" beats "silently discarded" any day.
Aren't we missing the elephant in the room here?
Mercurial = Python-based = runs anywhere Python runs
Git = Unix-like systems only
Doesn't Python shelter a decent sized Windows community? (Free, more accessible than the Windows flavors of Perl / Ruby)
Mercurial = Python-based = runs anywhere Python runs
Git = Unix-like systems only
Doesn't Python shelter a decent sized Windows community? (Free, more accessible than the Windows flavors of Perl / Ruby)
When Unix-like goes so far as to include windows with MSys, what does it exclude? And, for windows users that don't like to type, there's tortoisegit (http://code.google.com/p/tortoisegit/), which appears to be generally functional.
Mercurial is still better supported under Windows. Anecdotal evidence: compare the Eclipse plug-ins (under Windows) for git and hg. (And before there's flak about Eclipse: it is a widely used tool, under Windows at least).
Git not working well on Windows is something that continues to tarnish its reputation even though that hasn't been the case for quite some time.
Curiously, this was part of the reason Python chose not to use Git even after extensive research. One would have thought they would have stumbled across any number of projects like msysgit that would have shown them otherwise.
Curiously, this was part of the reason Python chose not to use Git even after extensive research. One would have thought they would have stumbled across any number of projects like msysgit that would have shown them otherwise.
I say this as a git user: msysgit works well, but that's not the same thing as working well on Windows. hq feels native on Windows in a way that git doesn't.
Having just installed hit on windows, the feeling I got was that there is msysgit, or running it on cygwin. Msysgit's homepage feels... Precarious, like it's part abandoned.
It does work, once you answer a question of which of three ways it should awkwardly integrate into your PATH, but within a cygwinesque bash shell.
So yeah, functional but nowhere near as smooth and seamless as python on windows is, or as one might pick up from your comment.
It does work, once you answer a question of which of three ways it should awkwardly integrate into your PATH, but within a cygwinesque bash shell.
So yeah, functional but nowhere near as smooth and seamless as python on windows is, or as one might pick up from your comment.
I seriously want to know. Have you actually used msysgit for any length of time on a relatively large project?
What happens there is that submodules are checked out as a detached head. You'll see that when committing something in a submodule:
Of course, your changes aren't lost after the update. You can still get access to them from the reflog ('git reflog') or a simple "git checkout @{1}"
Vienna-2:libgit2 pieter$ git commit -am "wawaa"
[detached HEAD 8641889] wawaa
Of course, your changes aren't lost after the update. You can still get access to them from the reflog ('git reflog') or a simple "git checkout @{1}"
First, that sounds pretty damning.
Second, where are all these points whenever someone asks about git? All you ever seem to get in return are fanboys trumpeting git as the second coming, even here on HN.
Second, where are all these points whenever someone asks about git? All you ever seem to get in return are fanboys trumpeting git as the second coming, even here on HN.
where are all these points whenever someone asks about git?
If you go looking for rough edges in an open-source Unix power tool that's under development... you're going to find them.
Based on my research (but, please note, not much actual experience) git submodules are ugly, confusing, and perhaps even dangerous in the hands of anyone but an expert. I avoid them. That said, it appears that lots of Rails developers use them every day and can't live without them. Should this feature have been kept out of the official git release, just because it's still being worked out? Not an easy question to answer. Every project maintainer has a different philosophy about such things, and there are a lot of judgement calls to be made.
If you go looking for rough edges in an open-source Unix power tool that's under development... you're going to find them.
Based on my research (but, please note, not much actual experience) git submodules are ugly, confusing, and perhaps even dangerous in the hands of anyone but an expert. I avoid them. That said, it appears that lots of Rails developers use them every day and can't live without them. Should this feature have been kept out of the official git release, just because it's still being worked out? Not an easy question to answer. Every project maintainer has a different philosophy about such things, and there are a lot of judgement calls to be made.
My issue is not that they exist. I've got no problem with warts in software. I don't think there's a project anywhere that doesn't have some sort of issue.
What I take issue with however is that these warts get conveniently forgotten in the buzz. Even worse, I'd be willing to bet that most of those rails dev's have no idea what is solid and what's a work in progress; they're being done a disservice, really.
I suppose my concern is that there's a fine line between enthusiasm and hype, and it's difficult to know when you've crossed the line.
What I take issue with however is that these warts get conveniently forgotten in the buzz. Even worse, I'd be willing to bet that most of those rails dev's have no idea what is solid and what's a work in progress; they're being done a disservice, really.
I suppose my concern is that there's a fine line between enthusiasm and hype, and it's difficult to know when you've crossed the line.
What I take issue with however is that these warts get conveniently forgotten in the buzz
The problem, to paraphrase Tolstoy, is that happy users are all alike, but every unhappy user is unhappy in her own way.
It's easy to write a review that talks about the core features that a tool gets right. Everyone cares about those features, and everyone uses them, and they're well documented. It is much harder to write a review containing a categorical list of warts. Warts are incidental problems that only bother a subset of users. (If they bothered absolutely everybody, they wouldn't be called warts; they'd have a different name: bugs or misfeatures.)
Worse, one person's wart is another person's feature. Legions of people have complained, over the decades, that rm has a horrible and dangerous UI which lets you accidentally erase your entire machine by typing a stray space at the wrong time. But, somehow, rm persists, because there are plenty of power users who like it.
The problem, to paraphrase Tolstoy, is that happy users are all alike, but every unhappy user is unhappy in her own way.
It's easy to write a review that talks about the core features that a tool gets right. Everyone cares about those features, and everyone uses them, and they're well documented. It is much harder to write a review containing a categorical list of warts. Warts are incidental problems that only bother a subset of users. (If they bothered absolutely everybody, they wouldn't be called warts; they'd have a different name: bugs or misfeatures.)
Worse, one person's wart is another person's feature. Legions of people have complained, over the decades, that rm has a horrible and dangerous UI which lets you accidentally erase your entire machine by typing a stray space at the wrong time. But, somehow, rm persists, because there are plenty of power users who like it.
There are warts and there are warts. When there are warts in some kind of cool experimental toy, that's one thing, but there are other systems that should be as solid and reliable as possible. Things like backups, and, yeah, version control.
I saw that quote re: PyCon and eagerly googled for 'Brett Cannon', pycon, git and only found the same announcement time and again without any explanation as to why git was disqualified.
Is anyone here involved closely in the python community who could maybe shed some light on what made git inadequate for their needs?
Is anyone here involved closely in the python community who could maybe shed some light on what made git inadequate for their needs?
I believe the initial consensus was that a Python-based version control system was greatly preferred over anything else, and that for Git to be considered someone would have to show that it's much more compelling than the other systems. Git was allowed into the PEP later on, however.
Besides being able to say they're eating their own dog food, I think having the ability to easily extend their version control system was a factor in the choice. The Python developers would just be writing Python, after all, and Mercurial has a very simple extension system.
I believe Guido has had direct communication with some of the core developers of Mercurial as well, which was probably another factor. I'm not sure what Brett's DVCS usage survey had to say about Mercurial or Git, but that could also play a role in his decision.
Here's the original thread that led up to the PEP: http://thread.gmane.org/gmane.comp.python.devel/98191. From what I understand, there wasn't supposed to be a lot of discussion until the PEP was written, but people couldn't help but voice their opinions on the matter. I'm guessing this is why Guido put his foot down: to prevent an endless religious debate. Hopefully the PEP will be revised with a more detailed explanation of his rationale.
Besides being able to say they're eating their own dog food, I think having the ability to easily extend their version control system was a factor in the choice. The Python developers would just be writing Python, after all, and Mercurial has a very simple extension system.
I believe Guido has had direct communication with some of the core developers of Mercurial as well, which was probably another factor. I'm not sure what Brett's DVCS usage survey had to say about Mercurial or Git, but that could also play a role in his decision.
Here's the original thread that led up to the PEP: http://thread.gmane.org/gmane.comp.python.devel/98191. From what I understand, there wasn't supposed to be a lot of discussion until the PEP was written, but people couldn't help but voice their opinions on the matter. I'm guessing this is why Guido put his foot down: to prevent an endless religious debate. Hopefully the PEP will be revised with a more detailed explanation of his rationale.
"At PyCon, Brett already announced that Git was no longer being considered -- while it has obviously many fans, it also provokes strong antipathies."
This seems illogical at best. They chose not to use Git because there are a lot of people that dislike it? This decision should have been based on technical details, not on feelings. Anything else is doing a disservice to the developers and to the community. How many other VCS's were passed up because they had a lot of haters?
This seems illogical at best. They chose not to use Git because there are a lot of people that dislike it? This decision should have been based on technical details, not on feelings. Anything else is doing a disservice to the developers and to the community. How many other VCS's were passed up because they had a lot of haters?
I guess the feelings are based on technical details.
Just looking at the python/ruby communities mostly from the outside, I wouldn't discount that many python folks dislike Git simply because many rubyists like it. I'm not saying there aren't valid technical reasons, (and frankly I don't know much technically about either Git or Mercurial) but I'd be surprised if emotion/religion doesn't play in at least a little bit.
That's a good point. Ruby has really become Python's "Other" in spite of their similarities.
You "guess"? That's not a very accurate analysis of the situation.
Since they aren't sharing any of these details, we have no choice but to assume there are none. In fact, their exact reason for not using Git was "it also provokes strong antipathies". That doesn't even _imply_ technicality on any level.
Either way, I couldn't care less personally. The only people they're hurting are themselves, as a gateway can be created between Git and Mercurial. All the projects I work on right now use SVN, and I just use git-svn for everything.
Since they aren't sharing any of these details, we have no choice but to assume there are none. In fact, their exact reason for not using Git was "it also provokes strong antipathies". That doesn't even _imply_ technicality on any level.
Either way, I couldn't care less personally. The only people they're hurting are themselves, as a gateway can be created between Git and Mercurial. All the projects I work on right now use SVN, and I just use git-svn for everything.
I'm not sure why you're being down-modded.
The Python team purporting to do DVCS research and then saying their decision was made on a gut feeling seems disingenuous at best.
Sure, I'm biased because I work for GitHub, but this decision would have felt like much less of a slap in the face had they just said: "We're switching to Mercurial because we like it and it's written in Python, case closed."
The Python team purporting to do DVCS research and then saying their decision was made on a gut feeling seems disingenuous at best.
Sure, I'm biased because I work for GitHub, but this decision would have felt like much less of a slap in the face had they just said: "We're switching to Mercurial because we like it and it's written in Python, case closed."
The way I read it, and the document discussing the VCS switch (PEP 374), was that the Big Three DVCSes (git, hg, bzr) were essentially all just fine. Git is the fastest and hairiest, Bazaar is the slowest and cutest, and Mercurial covers both ends fairly well without introducing any glaring flaws. All three systems have big, well-known projects actively running on them, with developers singing about how much better their respective choices are compared to the SVN or CVS systems. Any choice would have been fine; it was fairly obvious.
Python has a history of fussy programmers. Remember the web framework wars of a few years ago? Everyone complained that there were too many choices, picked favorites to champion and nitpicked at the alternatives (which were not actually bad), and nobody could sleep at night because there were alternatives and nobody knew which one was right. Then Guido announced that Django was right, and the lost souls breathed a sigh of relief and did the Django tutorial, and the folks who were already using Turbogears and Pylons productively continued to do so. Django isn't necessarily the best Python web framework for everyone, but it's rarely a bad choice.
That's pretty much what happened this year with DVCSes. The Biopython project was stalled on a CVS-to-SVN transition for over a year, for example, until some grenades were thrown and tears were shed and finally Git was picked because a few people liked it, nobody really had a problem with it and all the other options were triggering nasty blog posts. Case closed. Sometimes we just need a quick slap to shut us up.
Python has a history of fussy programmers. Remember the web framework wars of a few years ago? Everyone complained that there were too many choices, picked favorites to champion and nitpicked at the alternatives (which were not actually bad), and nobody could sleep at night because there were alternatives and nobody knew which one was right. Then Guido announced that Django was right, and the lost souls breathed a sigh of relief and did the Django tutorial, and the folks who were already using Turbogears and Pylons productively continued to do so. Django isn't necessarily the best Python web framework for everyone, but it's rarely a bad choice.
That's pretty much what happened this year with DVCSes. The Biopython project was stalled on a CVS-to-SVN transition for over a year, for example, until some grenades were thrown and tears were shed and finally Git was picked because a few people liked it, nobody really had a problem with it and all the other options were triggering nasty blog posts. Case closed. Sometimes we just need a quick slap to shut us up.
If you read much of what Guido writes, you'll see that most of his decisions come from a lot of thinking about a topic or problem and then going with what his gut tells him. It just seems to be the way he works, and I'd argue that the results speak for themselves.
And I can understand how you would wish they used git, but if you're taking this as a "slap in the face" at all, I think you're reading it wrong. They did some research on various similar tools and picked one.
And I can understand how you would wish they used git, but if you're taking this as a "slap in the face" at all, I think you're reading it wrong. They did some research on various similar tools and picked one.
[deleted]
Why would you ever force a system that nobody likes on your developers? "Better" doesn't matter if nobody's happy using it.
He didn't say that the developers in particular didn't like it. He didn't even say that _he_ doesn't like it. He only gave a generic statement referring to an unknown group of people.
My problem is not with their decision, but with how they made it. They are taking the "cathedral" approach, rather than the "bazaar" one.
My problem is not with their decision, but with how they made it. They are taking the "cathedral" approach, rather than the "bazaar" one.
He didn't say that the developers in particular didn't like it. He didn't even say that _he_ doesn't like it. He only gave a generic statement referring to an unknown group of people.
Also known as "weasel words": http://en.wikipedia.org/wiki/Weasel_word
Also known as "weasel words": http://en.wikipedia.org/wiki/Weasel_word
Conversely, do you feel they should make decisions based on the number of developers that do like a system?
Good news for Mercurial. It happens to be my favorite DVCS. Bruce Eckel seemed to come to the same conclusion (HG over BZR). It just seems to have the right feel after coming from Subversion. All in all, it is good competition. Git needs to get a smoother feel on Windows, BZR has a lot going for it - but needs to find that polish, hg needs to add first class SVN support, and SVN needs keep from stagnating or changing too much.
Cool, and I like Guido's way of making decisions. But I'm just wondering why they're switching. Did they find that Subversion wasn't working for them?
Oh, I see. PEP 374: http://www.python.org/dev/peps/pep-0374/
I think this is a good choice. I found learning mercurial after using svn for years to be very easy. And why not choose a system written in the language that you are writing? Sort of a recursive dog food feast...
Hedging their bets?
murcurialhub.com is registered by github
murcurialhub.com is registered by github
Github mostly provides the pretty web interface. There's no reason they couldn't do the same for the hg crowd. So, not so much hedging their bets, but looking to expand?
Ha! mercurialhub.org is still available...
The battle has been joined. ;)
Seriously, though, it's probably a good fit. Mercurial is written in Python, after all.
Seriously, though, it's probably a good fit. Mercurial is written in Python, after all.
More importantly, Mercurial is extensible in Python -- you can write extension modules, hooks, etc., all of which have full access to the (albeit poorly-documented) Mercurial internals.
Writing extensions for Git is rather different, since you effectively can't. To extend or build on Git, you end up doing a lot of shelling out to the 'git' binary with use of 'raw' output formats and regexp-based scraping. It's pipes and shell subcommands all the way down, instead of a library-style programming API.
Which you prefer depends a lot on the diversity and preferences of the community using your repositories. Mercurial pushes you towards usability and single-language (Python) solutions, while Git easily drops into a classic UNIX-like "lots of little scripts" solution.
Writing extensions for Git is rather different, since you effectively can't. To extend or build on Git, you end up doing a lot of shelling out to the 'git' binary with use of 'raw' output formats and regexp-based scraping. It's pipes and shell subcommands all the way down, instead of a library-style programming API.
Which you prefer depends a lot on the diversity and preferences of the community using your repositories. Mercurial pushes you towards usability and single-language (Python) solutions, while Git easily drops into a classic UNIX-like "lots of little scripts" solution.
[deleted]
Should be interesting to see whether Mercurial will grow or maintain marketshare.
This is great-- I had recently started my own Python mercurial repo for minor changes we had to make for embedding it into the Source engine. This will make it easier for us to keep in sync with base.
I am thrilled that Python is switching to a python-based DVCS, but I'm bitter about Bazaar not being the choice :-/
Yeah, after reading his explanation, I wished I'd chimed in and said I'm a non-Canonical coder who's been really happy with Bazaar.
Second, I want competition in the SCM space. I think the biggest tragedies with Subversion and CVS before it was that their dominant positions implied that there was "only one way" to do SCM. Hopefully, by keeping at least two competitors around, there will be more stimulus for invention and improvement.
...sure, it'll make developers live a bit harder, as now there is yet another decision that needs to be made. But come now! If developers can decide between Vi and Emacs, surely they are capable of deciding between Git and Mercurial (and no...don't even try to tell me there are more than two choices for text editor ;-).