A note of reflection after 10 years of “A successful Git branching model”(nvie.com)
nvie.com
A note of reflection after 10 years of “A successful Git branching model”
https://nvie.com/posts/a-successful-git-branching-model/
14 comments
I would love all the hours back I spent in discussions around branching strategy, trying to keep complex models understood across the team (and myself), dealing with painful merges and flowing changes through, trying to figure out if a change is in a branch already, etc..
It's so much simpler and more productive if everyone works in master, with new features behind a flag if need be, with just critical fixes ported to release branches.
There will always be arguments for why a more complicated branch strategy needs to be introduced. Resist those arguments. In my experience, the cost rarely pays for any benefits. Writing good software is hard enough already.
It's so much simpler and more productive if everyone works in master, with new features behind a flag if need be, with just critical fixes ported to release branches.
There will always be arguments for why a more complicated branch strategy needs to be introduced. Resist those arguments. In my experience, the cost rarely pays for any benefits. Writing good software is hard enough already.
We work on a branch and merge to master only after code review and (usually) quality assurance. We also deploy asap after merging to master and passing all regression tests. We can and often do deploy the same service many times a day.
Trunk based using short lived feature branches is the only thing I've used that has made any sort of sense in terms of value vs. complexity.
Same here. Trunk based is easy to explain and enforce.
The only challenge we faced was maintaining patch releases. Folding in fixes for common issues which were found in newer releases sometimes involved more than just cherrypicking the fix into a patch release for an older version.
This inevitably led to asking users to update to more up-to-date versions. This further enforced the linear development paradigm.
The only challenge we faced was maintaining patch releases. Folding in fixes for common issues which were found in newer releases sometimes involved more than just cherrypicking the fix into a patch release for an older version.
This inevitably led to asking users to update to more up-to-date versions. This further enforced the linear development paradigm.
This is the one thing that I find lacking in trunk-based development. I work on multiple products that have a release branch for each minor release, and it works incredibly well other than this as a minor pain point.
I find the easiest way is that if you know you want (or even think you might want) a particular fix in a patch version, you can make a branch from the oldest release branch you want to support. When you create a PR, you can easily target to the release branch (or another later release branch) or master, depending on what you want. You can even do multiple PRs to merge it multiple places.
No matter how you go about it, it should always be safe to merge a release branch back into master. In fact, this is a good thing to do after you release a patch version, but can also be done whenever you need a particular fix in newer (master, or other feature branch code) code immediately.
Merging release branches like this makes following a fix significantly easier than if you do it with cherry picking. Consider if you're looking at bug ticket, and trying to figure out what branches/releases it was fixed in. If the PR was merged to master, then later cherry picked to an older release, all you'll see is it was in master. Unless someone manually comments about it or your tooling picks it up based on references, the only other thing to do is check the code itself to see if the fix is there. When you instead merge forward, even if the ticket comments are wrong or not there, you can literally follow the branches from the PR/commits and see every release it got into.
I find the easiest way is that if you know you want (or even think you might want) a particular fix in a patch version, you can make a branch from the oldest release branch you want to support. When you create a PR, you can easily target to the release branch (or another later release branch) or master, depending on what you want. You can even do multiple PRs to merge it multiple places.
No matter how you go about it, it should always be safe to merge a release branch back into master. In fact, this is a good thing to do after you release a patch version, but can also be done whenever you need a particular fix in newer (master, or other feature branch code) code immediately.
Merging release branches like this makes following a fix significantly easier than if you do it with cherry picking. Consider if you're looking at bug ticket, and trying to figure out what branches/releases it was fixed in. If the PR was merged to master, then later cherry picked to an older release, all you'll see is it was in master. Unless someone manually comments about it or your tooling picks it up based on references, the only other thing to do is check the code itself to see if the fix is there. When you instead merge forward, even if the ticket comments are wrong or not there, you can literally follow the branches from the PR/commits and see every release it got into.
> Unless someone manually comments about it or your tooling picks it up based on references.
I don't find this to be overly burdensome on the tooling. If the ticket-tracking system can't automate traceability between git commits and tickets, then it isn't well-suited to git-based software development.
We use the following policy:
- A bugfix commit must fix the bug, the whole bug, and nothing but the bug. If that seems impractical for the task at hand, make it practical by breaking up the bug's ticket into smaller elements.
- Bugfix commits always cite the bug tracking system's ticket number.
- The bug tracking system observes the git tree and maintains its own linkage to it.
- When cherry-picking the bugfix to other affected branches, pass the `-x` argument to `git cherry-pick`.
That's it. The (cherry-picked from) comment helps out when just viewing the git history on its own. And the bug tracking system lets you quickly see all of the commits and pull requests that referred to the bug.
I don't find this to be overly burdensome on the tooling. If the ticket-tracking system can't automate traceability between git commits and tickets, then it isn't well-suited to git-based software development.
We use the following policy:
- A bugfix commit must fix the bug, the whole bug, and nothing but the bug. If that seems impractical for the task at hand, make it practical by breaking up the bug's ticket into smaller elements.
- Bugfix commits always cite the bug tracking system's ticket number.
- The bug tracking system observes the git tree and maintains its own linkage to it.
- When cherry-picking the bugfix to other affected branches, pass the `-x` argument to `git cherry-pick`.
That's it. The (cherry-picked from) comment helps out when just viewing the git history on its own. And the bug tracking system lets you quickly see all of the commits and pull requests that referred to the bug.
> Folding in fixes for common issues which were found in newer releases sometimes involved more than just cherrypicking the fix into a patch release for an older version.
Honestly? This isn't an a problem of accidental complexity that some combination of tooling and process can fix for you. I don't think that anyone has kept the cost as low as log(N_releases). Even that would be too high for most teams.
Honestly? This isn't an a problem of accidental complexity that some combination of tooling and process can fix for you. I don't think that anyone has kept the cost as low as log(N_releases). Even that would be too high for most teams.
An interesting trick is to develop a fix as a feature branch based as far back as practicable, and then merge it into the trunk and any release branches.
+1 to trunk based. There's obviously a lot there, but one of the best bits is the idea that each release deserves a branch. It makes hotfixes, ci/cd, and rollbacks all much simpler to reason about.
TBD is fine for web apps, but not for software that you have to maintain multiple versions in the wild at the same time or when the release cycle takes several weeks.
That isn't true at all. Trunk based is perfect for this because you have the clean release branches from the trunk.
I've worked in large companyies very successfully using TBD and releasing quarterly to enterprise customers using our software on their systems.
I've worked in large companyies very successfully using TBD and releasing quarterly to enterprise customers using our software on their systems.
> I've worked in large companyies very successfully using TBD and releasing quarterly to enterprise customers using our software on their systems.
care to elaborate? curious about those companies and the software you are talking about.
i totally see the trunk based process work for software that doesn't need to maintain multiple versions as the original commenter said.
but for multi-versioned software you can't afford to have a single trunk and let your developers have a go at it. i went and checked the repos for some of the multi-versioned software that i use that are open-source.
well confirmed.
care to elaborate? curious about those companies and the software you are talking about.
i totally see the trunk based process work for software that doesn't need to maintain multiple versions as the original commenter said.
but for multi-versioned software you can't afford to have a single trunk and let your developers have a go at it. i went and checked the repos for some of the multi-versioned software that i use that are open-source.
well confirmed.
Why do you need to maintain multiple versions? You should only need to maintain hotfixes on the last release branch. Once the next release is ready everybody who can update should take that release, any older releases become unsupported.
From a customer point of view a hotfix vs new release should be no difference. You just have to ensure that all releases are interface-compatible with the previous one. The cost of this is well worth it compared to the complexity of juggling multiple branches in parallel.
From a customer point of view a hotfix vs new release should be no difference. You just have to ensure that all releases are interface-compatible with the previous one. The cost of this is well worth it compared to the complexity of juggling multiple branches in parallel.
I think I understand your point, but thinking about releasing a hotfix over an old version is a way of maintaining multiple versions... I work once with a team supporting multiple versions of a really complex app(in the sense of a lot of features and DB entities). They support his clients (on-premises) with eventual hotfixes, one service-pack and one version a year.
Some times they were working on changes on a release before a hotfix was addressed, and with cherry-pick's like operations, they took the changes of the hotfixes and applied to the service-pack and new versions with less effort than before they used the model.
With a dev team of 10 programmers, 20 on IT support and hundreds of client's deployments they build a profitable business (subscription-based and in-site support) where the branching model helps a lot.
They spend less time managing the releases, switching context for priority support and dealing with more changes in less time.
Some times they were working on changes on a release before a hotfix was addressed, and with cherry-pick's like operations, they took the changes of the hotfixes and applied to the service-pack and new versions with less effort than before they used the model.
With a dev team of 10 programmers, 20 on IT support and hundreds of client's deployments they build a profitable business (subscription-based and in-site support) where the branching model helps a lot.
They spend less time managing the releases, switching context for priority support and dealing with more changes in less time.
At the top of this thread is a link to a clear explanation in a post by umvi. Very easy to understand.
So...shouldn't the focus be fixing the fact that release cycles take several weeks?
This is well researched and discussed in Nicole Forsgren's book: Accelerate: The Science of Lean Software and DevOps: Building and Scaling High Performing Technology Organizations
https://www.amazon.com/Accelerate-Software-Performing-Techno...
This is well researched and discussed in Nicole Forsgren's book: Accelerate: The Science of Lean Software and DevOps: Building and Scaling High Performing Technology Organizations
https://www.amazon.com/Accelerate-Software-Performing-Techno...
Sometimes long test cycles are just the nature of the business, especially in hardware-software co-development, and especially in high-reliability development. Nobody will believe that running software tests on a workstation or server somewhere counts for release readiness if it gets deployed to a wind turbine, robot, or megawatt UPS.
The software team's automated tests indicate that the software is ready to start an expensive physical testing cycle, not that its ready to go to the field.
The software team's automated tests indicate that the software is ready to start an expensive physical testing cycle, not that its ready to go to the field.
More accurate to say gitflow is fine for webapps. TBD is broader than gitflow, and i agree that when you have to maintain multiple versions in the wild, i would not use gitflow.
I didn't know there was a name for this, but I've been doing it for 2 years now and I don't think I could go back.
Same here. My team has been managing our git repo like this for years, but I didn't realize there was a named practice around it.
It was the way in the svn days and common before. But folks attracted to git are often the same that like to desimplify things.
Absolutely agree. I think git flow makes sense to begin with however once you start to do CI/CD, you need something a bit more robust.
Trunk based development gives you that. No long lived branches. No merge hell.
It’s considerably better.
Trunk based development gives you that. No long lived branches. No merge hell.
It’s considerably better.
I’d argue that starting simple and adding complexity only when needed is a better strategy.
The thing I don't get about trunk-based development is: if everybody always pushes straight to master, how do you do code review? Or don't you do code review in TBD?
I like having explicit pull requests where a new feature can be reviewed.
I like having explicit pull requests where a new feature can be reviewed.
[deleted]
That link does say that you can use pull requests for merging the short-lived feature branches back into master. It says that, for small teams, pushing directly into master might be acceptable (presumably without code review).
But how short-lived can they be? Feature development isn't necessarily short. Reviewing bits and pieces of a feature without having the whole thing together may not be very informative. And small teams may also care about code review.
Reading the comments, I realize I'm a git neophyte (limited experience working on some open source software, didn't know "gitflow" was a thing) so excuse me for my naive question. But what about this simple workflow. Developers submit PR on top of a master branch (developer is in charge of rebasing, PR must pass integration test). Administrator is in charge of merging, maintains a linear history. Releases are just tags on master. Has this workflow a name? anything wrong with it?
A year after you release v1.0.0 (which you say is a tag), a customer reports a bug in it. They don't want a breaking upgrade to v9.13.0. They want to pay for v1.0.1 (aka "hotfix").
But you can't open a PR against a tag v1.0.0. Hence you discover that v1.0 should be a branch, and both v1.0.0 and v1.0.1 tags. This way you can open a PR against v1.0 and easily create v1.0.1, v1.0.2, etc.
Another problem. You are at "master" and approach v13.0.0. But there are numerous bugs to fix before you release. Testing takes >24 hours. You choose to work on bugs on a branch (v13.0) this way master can proceed with new features and not be feature-frozen for days or weeks.
Another problem. Your CI is deficient in that it allows merging and cloning before it makes sure that previous merge produced good product. Puny as it is, it is the current industry standard. Thus your main branch becomes broken now and then. So you call it "develop" and only merge it to "master" what you are sure is a good product.
If these common situations don't apply to you - proceed as you are. Do the simplest thing possible, but not simpler. What you've described doesn't create any roadblock or dead end street for the project.
But you can't open a PR against a tag v1.0.0. Hence you discover that v1.0 should be a branch, and both v1.0.0 and v1.0.1 tags. This way you can open a PR against v1.0 and easily create v1.0.1, v1.0.2, etc.
Another problem. You are at "master" and approach v13.0.0. But there are numerous bugs to fix before you release. Testing takes >24 hours. You choose to work on bugs on a branch (v13.0) this way master can proceed with new features and not be feature-frozen for days or weeks.
Another problem. Your CI is deficient in that it allows merging and cloning before it makes sure that previous merge produced good product. Puny as it is, it is the current industry standard. Thus your main branch becomes broken now and then. So you call it "develop" and only merge it to "master" what you are sure is a good product.
If these common situations don't apply to you - proceed as you are. Do the simplest thing possible, but not simpler. What you've described doesn't create any roadblock or dead end street for the project.
> A year after you release v1.0.0 (which you say is a tag), a customer reports a bug in it. They don't want a breaking upgrade to v9.13.0. They want to pay for v1.0.1 (aka "hotfix").
> But you can't open a PR against a tag v1.0.0. Hence you discover that v1.0 should be a branch, and both v1.0.0 and v1.0.1 tags. This way you can open a PR against v1.0 and easily create v1.0.1, v1.0.2, etc.
So you could just `git checkout v1.0.0; git checkout -b v1.0`, commit your hotfix and deploy v1.0.1.
> Another problem. You are at "master" and approach v13.0.0. But there are numerous bugs to fix before you release. Testing takes >24 hours. You choose to work on bugs on a branch (v13.0) this way master can proceed with new features and not be feature-frozen for days or weeks.
Or you could use topic branches for new features, and fix bugs on master at the same time.
> Another problem. Your CI is deficient in that it allows merging and cloning before it makes sure that previous merge produced good product. Puny as it is, it is the current industry standard. Thus your main branch becomes broken now and then. So you call it "develop" and only merge it to "master" what you are sure is a good product.
I don't understand your point here. Usually your CI builds a branch, runs some tests on the compiled product and makes sure the `master` branch is ok before deploying to staging or production.
> If these common situations don't apply to you - proceed as you are. Do the simplest thing possible, but not simpler. What you've described doesn't create any roadblock or dead end street for the project.
I think there are a lot of branching strategies for a reason. Your use cases can be covered with other branching strategies as well.
> But you can't open a PR against a tag v1.0.0. Hence you discover that v1.0 should be a branch, and both v1.0.0 and v1.0.1 tags. This way you can open a PR against v1.0 and easily create v1.0.1, v1.0.2, etc.
So you could just `git checkout v1.0.0; git checkout -b v1.0`, commit your hotfix and deploy v1.0.1.
> Another problem. You are at "master" and approach v13.0.0. But there are numerous bugs to fix before you release. Testing takes >24 hours. You choose to work on bugs on a branch (v13.0) this way master can proceed with new features and not be feature-frozen for days or weeks.
Or you could use topic branches for new features, and fix bugs on master at the same time.
> Another problem. Your CI is deficient in that it allows merging and cloning before it makes sure that previous merge produced good product. Puny as it is, it is the current industry standard. Thus your main branch becomes broken now and then. So you call it "develop" and only merge it to "master" what you are sure is a good product.
I don't understand your point here. Usually your CI builds a branch, runs some tests on the compiled product and makes sure the `master` branch is ok before deploying to staging or production.
> If these common situations don't apply to you - proceed as you are. Do the simplest thing possible, but not simpler. What you've described doesn't create any roadblock or dead end street for the project.
I think there are a lot of branching strategies for a reason. Your use cases can be covered with other branching strategies as well.
> Or you could use topic branches for new features, and fix bugs on master at the same time.
One could but it's often very desirable to integrate branches fast. If merging is delayed by a release there will be stale branches to merge. The experience of merging long branches scares people off of refactoring, because a refactoring creates more conflicts the longer it stays unmerged.
> I don't understand your point here. Usually your CI builds a branch, runs some tests on the compiled product and makes sure the `master` branch is ok before deploying to staging or production.
As long as you have sequential merges there is no problem. But when you merge into the main branch and it is ahead of you (due to another merge) the build may become broken. You can merge the other way first, sure. But if you want to enforce this you need support from your infrastructure.
One could but it's often very desirable to integrate branches fast. If merging is delayed by a release there will be stale branches to merge. The experience of merging long branches scares people off of refactoring, because a refactoring creates more conflicts the longer it stays unmerged.
> I don't understand your point here. Usually your CI builds a branch, runs some tests on the compiled product and makes sure the `master` branch is ok before deploying to staging or production.
As long as you have sequential merges there is no problem. But when you merge into the main branch and it is ahead of you (due to another merge) the build may become broken. You can merge the other way first, sure. But if you want to enforce this you need support from your infrastructure.
The only way to be sure: CI checks out master, merges the feature branch, and tests that version. If successful, push to origin. If the push fails (because a parallel pushed happened in between) start from the beginning.
The challenge here is that pull request might be tested again and again. It increases the CI load by some factor for large projects.
The challenge here is that pull request might be tested again and again. It increases the CI load by some factor for large projects.
> `git checkout -b v1.0`, commit your hotfix and deploy v1.0.1
And the code review happens where? You need to:
> I don't understand your point here. Usually your CI builds a branch, runs some tests on the compiled product and makes sure the `master` branch is ok.
Gasp... You are right, you don't understand my point. You show master to everyone before making sure it isn't broken?
And the code review happens where? You need to:
git checkout -b v1.0
git push
ask your colleague for a code review of branch v1.0
> I don't understand your point here. Usually your CI builds a branch, runs some tests on the compiled product and makes sure the `master` branch is ok.
Gasp... You are right, you don't understand my point. You show master to everyone before making sure it isn't broken?
The original post was describing "Administrator is in charge of merging, maintains a linear history. Releases are just tags on master", you seem to discuss the various branching strategy while it seems to me the explanation was about the fact that we need a branching strategy in the first place. Do I miss something?
The reply to the original post was describing branching off at a historic place without merging back into trunk. You can't have a branch without a branch ;-) I think the person replying did not understand that you can easily branch from a commit with a tag (and in fact a branch is really just a special kind of tag on a commit in git).
> So you could just `git checkout v1.0.0; git checkout -b v1.0`,
git checkout can take an optional tag/branch argument so
`git checkout -b v1.0 v1.0.0` works just as well for creating branch `v1.0` off of tag `v1.0.0`
git checkout can take an optional tag/branch argument so
`git checkout -b v1.0 v1.0.0` works just as well for creating branch `v1.0` off of tag `v1.0.0`
Yes, in those situations you will need additional branches beyond just master (and short-lived/personal feature branches). But,
1. You can implement the necessary branching with much less complexity than git-flow. Just have master, plus possibly some stable development branches created when you need it. Tag v1.0.0 off master, then make the v1.x branch and tag v1.0.1 on it. Or branch v13.x off master and tag v13.0.0 once it's stable. No "develop" branch, no long-running feature branches, nothing.
2. Importantly, you can implement the necessary branching when you need it. If you never run into these situations, you can keep having just a single master branch and not worry about it. You do not need to future-proof yourself for fear that you might run into a complicated situation. If you do, then you can make release branches at that point.
You don't need a separate "develop" / "master" branch to cover up weakness es in your CI. If your CI is good enough that you can automatically merge "develop" into "master" when tests pass, it's good enough that you can actually implement the Not Rocket Science Rule and automatically merge everything into master when tests pass. If it's not good enough, then just keep track of the last-known good commit on master, and fix build/test failures on master urgently, don't shove them off onto another branch so you can ignore then. The industry standard is the idea that "who broke the build" is a well-defined question - but it means the right step is to fix the build.
1. You can implement the necessary branching with much less complexity than git-flow. Just have master, plus possibly some stable development branches created when you need it. Tag v1.0.0 off master, then make the v1.x branch and tag v1.0.1 on it. Or branch v13.x off master and tag v13.0.0 once it's stable. No "develop" branch, no long-running feature branches, nothing.
2. Importantly, you can implement the necessary branching when you need it. If you never run into these situations, you can keep having just a single master branch and not worry about it. You do not need to future-proof yourself for fear that you might run into a complicated situation. If you do, then you can make release branches at that point.
You don't need a separate "develop" / "master" branch to cover up weakness es in your CI. If your CI is good enough that you can automatically merge "develop" into "master" when tests pass, it's good enough that you can actually implement the Not Rocket Science Rule and automatically merge everything into master when tests pass. If it's not good enough, then just keep track of the last-known good commit on master, and fix build/test failures on master urgently, don't shove them off onto another branch so you can ignore then. The industry standard is the idea that "who broke the build" is a well-defined question - but it means the right step is to fix the build.
Parent. I totally agree, especially with the CI remark.
I hate "develop" and I wish I could stop pointing out how crappy idea it is.
I hate "develop" and I wish I could stop pointing out how crappy idea it is.
Your model is called trunk based development and personally I like it a lot. It keeps the history clean and is very easy to use even for junior devs.
"Trunk based development"
It is fine, unless you do a lot of back ports or bug fixes for older releases, at which point the version branches are very useful.
It is fine, unless you do a lot of back ports or bug fixes for older releases, at which point the version branches are very useful.
Trunk based development, as described at https://trunkbaseddevelopment.com/, can be used with release branches, where bug fixes can be back ported to.
Are there any reasons at all for using git flow over trunk based development with release branches? The latter seems far simpler but just as flexible.
Are there any reasons at all for using git flow over trunk based development with release branches? The latter seems far simpler but just as flexible.
That works until you have to support lots of old versions.
I don't understand why you think that. Can you explain why having many release branches is harder in trunk based development?
In a previous job, we would just cherry pick bug fix commits over from master to all the active release branches. It wasn't hard.
In a previous job, we would just cherry pick bug fix commits over from master to all the active release branches. It wasn't hard.
It's not hard, but it's a manual process which is always prone to errors or oversights. Having to support lots of versions just smells a little fishy to me, maybe justifiable if you have some type of firmware/hardware tie-in.
Why can't you, if you need to hotfix, git checkout the tag from master then git branch -b to something designating that it's a hotfix branch, and then when the hotfix is verified git tag the head of that branch with a specific version?
You can then merge that hotfix branch down into the head of master, or into any other later versions by doing a similar branch from the newer version tag first. Essentially it makes version branches lazy in that they're only created when they're actually needed, but you can still support lots of older versions to your hearts content with bugfixes, hotpatches and minor updates.
The big downside to this strategy, though, is that since you don't have a version branch until you decide you need it, new features for testing can't be merged into master until the current version is release ready (ie all outstanding bugs are fixed).
This isn't really a huge issue for smaller teams/companies or ones that can keep a fast continuous development pace, especially if your release schedule is short, because at most the merge freeze is only a couple days. But this will result in developers needing to handle merge conflicts if their feature branch exists for more than a single deployment cycle. IMO you want to keep the merge window to no more than the number of weeks between deployments in days (eg 1 week between deployments requires a merge freeze no longer than 1 day, and 3 weeks requires a merge freeze no longer than 3 days).
However, it's also really easy to grow from this into what this article describes as the team grows or the testing/deployment/merge freeze window lengthens, because you can switch from waiting to need a hotpatch for making a version branch to making a version branch every time you'd otherwise begin a merge freeze. You do have to remember to merge the commit tagged with that release back into master so that master gets all the bug fixes, but that's no more onerous than requiring hotpatches get merged into master as well.
You can then merge that hotfix branch down into the head of master, or into any other later versions by doing a similar branch from the newer version tag first. Essentially it makes version branches lazy in that they're only created when they're actually needed, but you can still support lots of older versions to your hearts content with bugfixes, hotpatches and minor updates.
The big downside to this strategy, though, is that since you don't have a version branch until you decide you need it, new features for testing can't be merged into master until the current version is release ready (ie all outstanding bugs are fixed).
This isn't really a huge issue for smaller teams/companies or ones that can keep a fast continuous development pace, especially if your release schedule is short, because at most the merge freeze is only a couple days. But this will result in developers needing to handle merge conflicts if their feature branch exists for more than a single deployment cycle. IMO you want to keep the merge window to no more than the number of weeks between deployments in days (eg 1 week between deployments requires a merge freeze no longer than 1 day, and 3 weeks requires a merge freeze no longer than 3 days).
However, it's also really easy to grow from this into what this article describes as the team grows or the testing/deployment/merge freeze window lengthens, because you can switch from waiting to need a hotpatch for making a version branch to making a version branch every time you'd otherwise begin a merge freeze. You do have to remember to merge the commit tagged with that release back into master so that master gets all the bug fixes, but that's no more onerous than requiring hotpatches get merged into master as well.
Sure, it sucks at that point, but the question is - does git-flow work better?
Sounds like GitHub Flow. https://githubflow.github.io/
As others have stated, this works well when there aren't "releases" but once you have releases with versions and may have to backport things, it starts to breakdown. For how most web-based software is developed it seems to be a descent for.
As others have stated, this works well when there aren't "releases" but once you have releases with versions and may have to backport things, it starts to breakdown. For how most web-based software is developed it seems to be a descent for.
Here is what is wrong with it.
It will work fine 99% of the time. But when you get into trouble (and a single merge done badly may cause trouble) there is no way to recover it because you lost history. And trouble is often of the form, "A month of my work went poof and nobody noticed for weeks after."
Another pain point is that the Administrator becomes a bottleneck. And when the Administrator runs into a conflict they often have no context from which to sort it out.
But on the positive side, your history will be nice and linear. Which makes tools like git-bisect very, very happy.
It will work fine 99% of the time. But when you get into trouble (and a single merge done badly may cause trouble) there is no way to recover it because you lost history. And trouble is often of the form, "A month of my work went poof and nobody noticed for weeks after."
Another pain point is that the Administrator becomes a bottleneck. And when the Administrator runs into a conflict they often have no context from which to sort it out.
But on the positive side, your history will be nice and linear. Which makes tools like git-bisect very, very happy.
In most cases, it is perfectly fine.
It may be different if you develop something which is a library (rather than app) AND it is big enough you consider backporting changes to older versions.
If you don't, release branches (rather than tags) are nothing by a burden.
It may be different if you develop something which is a library (rather than app) AND it is big enough you consider backporting changes to older versions.
If you don't, release branches (rather than tags) are nothing by a burden.
Please correct me if I'm wrong, but I believe this is the model the Linux kernel uses.
The kernel at least used to use something like gitworkflows(7) [1]
1: https://gitirc.eu/gitworkflows.html
1: https://gitirc.eu/gitworkflows.html
the risk of this workflow is that PRs will languish for too long, and the work required to rebase them when they're actually needed will cause friction and mistakes. this will often manifest when your software reaches a certain level of maturity and you have established a release model that distinguishes between a release that includes only bugfixes and a release that introduces new features.
you definitely want to get your PRs merged in to _something_ as soon as they are complete, to avoid bitrot. but just because they're complete doesn't mean you want them included in the next release.
you definitely want to get your PRs merged in to _something_ as soon as they are complete, to avoid bitrot. but just because they're complete doesn't mean you want them included in the next release.
> but just because they're complete doesn't mean you want them included in the next release.
That's the crux. Continous Delivery means exactly that. You want to release early, small and often. Trunk-based development and having every PR that is merged to master be deployed to production in a very short timeframe fits that very well. That's very possible with a solid CI/CD pipeline, good tests, and something that can be deployed often like a web app.
If you don't do continuous delivery, trunk-based development might be a less natural fit.
That's the crux. Continous Delivery means exactly that. You want to release early, small and often. Trunk-based development and having every PR that is merged to master be deployed to production in a very short timeframe fits that very well. That's very possible with a solid CI/CD pipeline, good tests, and something that can be deployed often like a web app.
If you don't do continuous delivery, trunk-based development might be a less natural fit.
You can merge master/develop into your PR multiple times, always keeping it up to date with the latest until until it can be merged.
This is the magic of git, it allows multiple merges something that wasn't possible in older systems.
This is the magic of git, it allows multiple merges something that wasn't possible in older systems.
That is what I do too. Combined with restriction on the pull request so you can't merge it if it's behind the branch you are merging onto.
If you merge your PRs into something that isn't master when they're complete, they're bitrotting anyway, compared to master.
Why is admin bottleneck? Are they magick?
Is this related to the "Please stop recommending Gitflow" [1] article posted yesterday?
[1] https://news.ycombinator.com/item?id=22485489
[1] https://news.ycombinator.com/item?id=22485489
[deleted]
Kind of, that article recommends making things less complex if you do continuous delivery to a SaaS.
The update article mentions it was meant for versioned software.
I tried to articulate the various levels of branching you need in GitLab Flow https://docs.gitlab.com/ee/topics/gitlab_flow.html The various levels are: feature branches, production branches, environment branches, and release branches.
The update article mentions it was meant for versioned software.
I tried to articulate the various levels of branching you need in GitLab Flow https://docs.gitlab.com/ee/topics/gitlab_flow.html The various levels are: feature branches, production branches, environment branches, and release branches.
Sounds like gitlab tried to fix githubflow instead of realising its fundamentally broken and going with the gerrit pattern that works fine for both trunk based and release style applications.
We offer multiple options that depend on your need. GitHub flow and Git flow are at two ends but there are also intermediate steps.
We started with gitflow, but our team ended up going with a Release based branching system as the constant integration of various features ended up giving less control on what to deploy and when. Most common scenario...
Devs create feature branches from master
Features will be targeted to a release at some point, Release branch gets taken from master
Devs PR feature branches to the release branch
Release branch merges back to master on deployment.
Devs create feature branches from master
Features will be targeted to a release at some point, Release branch gets taken from master
Devs PR feature branches to the release branch
Release branch merges back to master on deployment.
that workflow has always been my preference. QA can review features in batches, you can schedule releases to minimize impact, and several eyes are on the release to catch possible bugs.
A problem with GitFlow is it's painful to follow PR model.
Especially on the Github repository where pushing to master is a lock and can only done by PR and merge(or squash or whatever).
Let's take a look at a release:
1. Branch of release/1.0 from develop 2. Fix some bug, then at the time of deploymennt. Merge it back to master and develop
But once we merge, github close the PR... If we attempt to create another PR for other branch. The Github now see different history between master/develop...
Especially on the Github repository where pushing to master is a lock and can only done by PR and merge(or squash or whatever).
Let's take a look at a release:
1. Branch of release/1.0 from develop 2. Fix some bug, then at the time of deploymennt. Merge it back to master and develop
But once we merge, github close the PR... If we attempt to create another PR for other branch. The Github now see different history between master/develop...
There exist scripts and workflows to perform auto-merges from all release branches back to master. This way, you only have to care about branching off releases, and hotfixes will automatically land on develop once released.
(Assuming master is your "next release", and develop is your main development branch)
(Assuming master is your "next release", and develop is your main development branch)
Like I said, I cannot merge because we cannot push from our computer to Github(It's locked).
The only way to get code into branch is by doing a PR, and click merge button on Github UI. But a PR cannot be merged twice.
And If I open two PRs two master/develop then I will see this kind of error on Github: "This branch is 1 commit behind. 1 commit ahead of develop"
The only way to get code into branch is by doing a PR, and click merge button on Github UI. But a PR cannot be merged twice.
And If I open two PRs two master/develop then I will see this kind of error on Github: "This branch is 1 commit behind. 1 commit ahead of develop"
the way to go here is probably creating an "administrator" account which is used by those merging scripts, and allow push access to protected branches for administrators.
So, on my team, juniors can create feature branches and easily deploy a feature branch to a ‘staging’ server.
If the feature is deemed good, an ‘admin’ will ‘finish’ the feature and it is merged into develop branch.
Then, when we are ready for a release, the ‘admin’ will create a release and both develop and master are updated.
Updating master triggers a webhook that deploys the release.
For a hotfix, the release step is skipped, but both develop and master are updated, again causing the webhook to trigger a deployment.
If the feature is deemed good, an ‘admin’ will ‘finish’ the feature and it is merged into develop branch.
Then, when we are ready for a release, the ‘admin’ will create a release and both develop and master are updated.
Updating master triggers a webhook that deploys the release.
For a hotfix, the release step is skipped, but both develop and master are updated, again causing the webhook to trigger a deployment.
Is Gitflow meant to be used with pull requests?
I do not use it that way, but I am open-minded to its advantages.
I do not use it that way, but I am open-minded to its advantages.
I guess it isn't well suit for pull requests model. At least, with Github Pull Request.
I've yet to work on a codebase that was so congested it required such strong delineation between feature/hotfix/release but I can see how it might be required at a certain size. I wonder what that threshold is.
It depends what you're working on. We have a fairly small team (7 engineers) working on a reasonably complex app (many features).
Our "features" take anything from 1 day to 3 weeks of effort, and generally need to roll out as a whole. They are often really upgrades of existing features, and the work to continuously merge w/ feature flags, supporting both flows/designs, is just not something we've felt was worth the extra time.
We release on a 2 week cycle, the same day each time. We merge our features into the develop branch as they are ready and safe for production.
Our scheduled release day is Wednesday. The Thursday beforehand, we branch from develop into a release/x.y.0. This builds release candidates. We have a sanity check on Friday, then our remote QA team runs through them over the weekend. Any issues they find, we try to have squared away by Monday, Tuesday at the latest.
Meanwhile other team members are still merging new things into develop. It's useful to have that release branch because it lets the rest of the team keep moving - we maybe only put out 1 or 2 developer's features each cycle, the rest are still trickling in.
Once we release, we tag, merge into master, and start over again. If any issues show up on production, we use the hotfix branch, hotfix/x.y.z. That let's us resolve issues with risking a ship of any of the changes that have come into develop that haven't had a few days of stability behind it.
For our team scale, we have a lot of users (100k's DAU), and we find this method is a nice balance between the pace of output you might get from true CI but with the reliability of a more traditional cycle.
Backend and web projects, however... Straight CI, into master, and we roll out test > staging > production when we're happy.
Our "features" take anything from 1 day to 3 weeks of effort, and generally need to roll out as a whole. They are often really upgrades of existing features, and the work to continuously merge w/ feature flags, supporting both flows/designs, is just not something we've felt was worth the extra time.
We release on a 2 week cycle, the same day each time. We merge our features into the develop branch as they are ready and safe for production.
Our scheduled release day is Wednesday. The Thursday beforehand, we branch from develop into a release/x.y.0. This builds release candidates. We have a sanity check on Friday, then our remote QA team runs through them over the weekend. Any issues they find, we try to have squared away by Monday, Tuesday at the latest.
Meanwhile other team members are still merging new things into develop. It's useful to have that release branch because it lets the rest of the team keep moving - we maybe only put out 1 or 2 developer's features each cycle, the rest are still trickling in.
Once we release, we tag, merge into master, and start over again. If any issues show up on production, we use the hotfix branch, hotfix/x.y.z. That let's us resolve issues with risking a ship of any of the changes that have come into develop that haven't had a few days of stability behind it.
For our team scale, we have a lot of users (100k's DAU), and we find this method is a nice balance between the pace of output you might get from true CI but with the reliability of a more traditional cycle.
Backend and web projects, however... Straight CI, into master, and we roll out test > staging > production when we're happy.
I basically follow the same protocol as you up to this point:
> Once we release, we tag, merge into master, and start over again. If any issues show up on production, we use the hotfix branch, hotfix/x.y.z. That let's us resolve issues with risking a ship of any of the changes that have come into develop that haven't had a few days of stability behind it.
In our case, there's no point in a hotfix branch because master is always considered stable. You don't merge to master unless you're ready for that code to hit production. Feature gates are used for hiding live code when necessary for marketing announcements or if the feature simply isn't ready for actual customer usage yet (sometimes we leave the endpoints live but turn off all the navigation to the endpoint so we can send out the link as a "beta" test for specific customers).
This does occasionally cause problems with feature branches lagging behind, but it's on the developer to ensure that their code merges into master smoothly (usually best done by syncing your feature branch back up to master frequently)
> Once we release, we tag, merge into master, and start over again. If any issues show up on production, we use the hotfix branch, hotfix/x.y.z. That let's us resolve issues with risking a ship of any of the changes that have come into develop that haven't had a few days of stability behind it.
In our case, there's no point in a hotfix branch because master is always considered stable. You don't merge to master unless you're ready for that code to hit production. Feature gates are used for hiding live code when necessary for marketing announcements or if the feature simply isn't ready for actual customer usage yet (sometimes we leave the endpoints live but turn off all the navigation to the endpoint so we can send out the link as a "beta" test for specific customers).
This does occasionally cause problems with feature branches lagging behind, but it's on the developer to ensure that their code merges into master smoothly (usually best done by syncing your feature branch back up to master frequently)
It all depends on the situation. The simplest possible situation of mostly just having a master branch also can work quite well. When you have specialized testers that test a candidate release you start to need release branches. Feature branches start being necessary if changes are temoprarily incompatible with current master. This should not be too common, though. In many cases one can avoid feature branches by having feature switches that get removed as soon as a feature is done. If one wants to or needs to use feature branches they should be short-lived because one will be creating a merge hell if this is not the case. The git flow seems just too complicated for most cases. One comment already notices that it is quite unclear why a separate development and master branch are necessary.
I love this branching model. I have being used in all projects and teams I can. I use it even in projects where I work alone. For me it is the ultimate branching model.
AFAICT Gitflow just has one more branch than it needs. If you got rid of the master branch and supported production releases off release branches it would be just as useful and significantly less complicated.
Can anyone explain to me what real value the master branch adds in this model? And without using fluffy meaningless words like, "source of truth," please.
I fully agree. Removing the master branch which is just fluff that noone uses you essentially come to trunkbased development.
The main advantage I can think of is you can just configure your nightly expensive CI jobs (running absolutely all e2e tests) to run against a static branch.
Another might be that git clone gets the "correct" branch by default. Might be important if your git repo gets cloned by sales, support or whatever for external demo purposes.
Anyway I don't think it makes a huge difference in complexity - it's rarely used for developers' workflows and doesn't even need to be updated that often.
Another might be that git clone gets the "correct" branch by default. Might be important if your git repo gets cloned by sales, support or whatever for external demo purposes.
Anyway I don't think it makes a huge difference in complexity - it's rarely used for developers' workflows and doesn't even need to be updated that often.
I don't really understand the distinction between develop and master either. You're supposed to branch off develop and merge forward back into develop, then merge into a release branch, then merge into master. So everything in develop is supposed to end up in master.
So... why not branch off master and merge back into master?
So... why not branch off master and merge back into master?
There is a small trade secret, that's hard to visualize. If you merge two changes into master you can break it.
The fact that a change passed a CI and another change passed a CI doesn't mean they can merge and the result is guaranteed to pass CI. It will often break and if you think for a minute you will come up with really rudimentary examples of this.
The really primitive solution is to call the merge-recipient branch the "develop" and only merge to "master" when you are sure what you are merging is good. You'll be ok if you do it sequentially (no parallel merges to master).
A more complicated solution is to actually test (in CI) "what happens when I merge this" but in hiding, without showing it to others. Even more complicated is to do it in parallel. Gitlab calls it Merge Trains (Premium only). Zuul-CI calls it gate/gating and has a pretty good description here: https://zuul-ci.org/docs/zuul/discussion/gating.html#testing...
The fact that a change passed a CI and another change passed a CI doesn't mean they can merge and the result is guaranteed to pass CI. It will often break and if you think for a minute you will come up with really rudimentary examples of this.
The really primitive solution is to call the merge-recipient branch the "develop" and only merge to "master" when you are sure what you are merging is good. You'll be ok if you do it sequentially (no parallel merges to master).
A more complicated solution is to actually test (in CI) "what happens when I merge this" but in hiding, without showing it to others. Even more complicated is to do it in parallel. Gitlab calls it Merge Trains (Premium only). Zuul-CI calls it gate/gating and has a pretty good description here: https://zuul-ci.org/docs/zuul/discussion/gating.html#testing...
Also known as the "Not Rocket Science" rule: https://graydon2.dreamwidth.org/1597.html
When you're ready to merge, have CI test the result of the merge, and only push the merge commit if things pass. Don't let anyone other than CI push to master.
You can set up a bot with a queue to do this, of course, but you can also define "CI" as a shell script. GitHub, GitLab, etc. will automatically close a pull request if you push to master; you don't have to push the button on the website. Train people to not hit "merge" and instead to run a script, something like this which I just threw together:
When you're ready to merge, have CI test the result of the merge, and only push the merge commit if things pass. Don't let anyone other than CI push to master.
You can set up a bot with a queue to do this, of course, but you can also define "CI" as a shell script. GitHub, GitLab, etc. will automatically close a pull request if you push to master; you don't have to push the button on the website. Train people to not hit "merge" and instead to run a script, something like this which I just threw together:
#!/bin/sh -e
repo="$(mktemp -d)"
git clone https://... "$repo"
cd "$repo"
git merge "$1"
make test
git push
cd
rm -rf "$repo"
I don't like the article - too many words, too many digressions. Also the chosen catch-phrase is poor.
As long as GitHub does it wrong, you won't be able to "train people" not to hit the big juicy green button, that should be really a big red "Emergency Merge - Can Break The Receiving Branch" button.
As long as GitHub does it wrong, you won't be able to "train people" not to hit the big juicy green button, that should be really a big red "Emergency Merge - Can Break The Receiving Branch" button.
So always rebase onto master and test your rebased changes before you can advance (sigh, "fast-forward merge", in the parlance of our gits) the master ref to your changes. I still don't see the need for two separate branches.
This is true. Another explanation could be a two stage testing process. The CI test for develop might run quickly in a few minutes, but the test suite for master might run for a day or require manual testing.
Manual testing is an important thing to consider, for sure.
Who does QA on the feature branches, and where? Does each feature branch have its own corresponding deployment? Are they sharing databases with other branches?
Who does QA on the feature branches, and where? Does each feature branch have its own corresponding deployment? Are they sharing databases with other branches?
Master is current production version. Develop is current development version. You branch off develop to work on your feature, then you merge it back to develop so you have integration point with your co-workers (you fix conflicts here) which can be deployed to test server or just run through automated tests. When you have develop branch in correct state (whatever that means for your team) then you merge it into master creating new release on production.
This way in case of some unforeseen bugs you can fix master which is production like and then merge that fix also into develop so it is present in next release. If you would develop on master you don't have obvious "production like" place. You could do tags, but somehow it fits better with CI setup and Jenkins.
In our team we have acceptance branch instead of "release branch".
Feature branches are short lived, develop, acceptance, master are long lived branches. You also don't mix stuff because what you really want is one way direction of promoting changes develop->acceptance->master.
This way in case of some unforeseen bugs you can fix master which is production like and then merge that fix also into develop so it is present in next release. If you would develop on master you don't have obvious "production like" place. You could do tags, but somehow it fits better with CI setup and Jenkins.
In our team we have acceptance branch instead of "release branch".
Feature branches are short lived, develop, acceptance, master are long lived branches. You also don't mix stuff because what you really want is one way direction of promoting changes develop->acceptance->master.
[deleted]
It's named "master" and people have become very superstitious about that name.
That's it, that's the reason.
That's it, that's the reason.
Yeah, I usually just have master(gitflow-dev) and release(gitflow-master). Works fine for me. Has the added benefit of git noobies not defaulting to the release branch they shouldn't just push to.
I have never succeeded in working in a branch and having the real owners accept my state.
I am of course talking about 'outsider' changes to code, but the effect is real: the master owner can elect to say "sorry, we decided to re-architect significantly and what you did is now superfluous, or at least too hard for us to re-integrate" and almost everything I have been involved with, this is what happened.
Submitting small patches to their head state is more effective for me than trying to maintain state against a moving target with a realistic chance of making significant change, and having it accepted into the mainline.
I am of course talking about 'outsider' changes to code, but the effect is real: the master owner can elect to say "sorry, we decided to re-architect significantly and what you did is now superfluous, or at least too hard for us to re-integrate" and almost everything I have been involved with, this is what happened.
Submitting small patches to their head state is more effective for me than trying to maintain state against a moving target with a realistic chance of making significant change, and having it accepted into the mainline.
Strongly agreed with the new author note. I used the same ideas with the teams I've been coaching:
"Consider your context...." as the need or not of support for multiple versions and "...Decide for yourself.
[deleted]
I accept your apology.
History is for reading, not writing. Do we make decisions that give us readable and informative histories? The people who give up and rebase absolutely everything into one linear trunk are not working optimally, but at least they're trying.
History is for reading, not writing. Do we make decisions that give us readable and informative histories? The people who give up and rebase absolutely everything into one linear trunk are not working optimally, but at least they're trying.
So cool that something Linus Torvalds made to solve his own problems, spread all around the world so quickly. I guess great dev tools can only be made by great developers, who have been through a lot of scenarios/problems themselves.
What's interesting to me is that git does not seem like a "great dev tool" when you first start using it, especially if you're already used to another VCS. Right now I'm using SVN, Perforce and git for different projects and I definitely like git the most, but it took a while to get there. And it's not the centralized vs. decentralized per se, like 99.9% of git users I'm almost always within milliseconds of a central server. The big change was to treat my development "as-if" I was doing my own thing, decentralized, when in fact I'm not.
[1] https://trunkbaseddevelopment.com/