Benefits of a Monorepo(pspdfkit.com)
pspdfkit.com
Benefits of a Monorepo
https://pspdfkit.com/blog/2019/benefits-of-a-monorepo/
18 comments
There is available tooling to help you with that, not only has to be discipline.
A monorepo oriented build tool such as Bazel will make sure that you don't compile more than needed. This is extremely efficient. In fact, in my company with a few hundred projects we tried it and it was 10 times faster than our previous Maven based workflow. These tools also help you with dependency management so you keep your code organized. The killer feature is that you finally have true Continuous Integration because the build tool will figure out any dependencies of the code that you change, rebuilding it and testing to make sure that nothing is broken.
A monorepo oriented build tool such as Bazel will make sure that you don't compile more than needed. This is extremely efficient. In fact, in my company with a few hundred projects we tried it and it was 10 times faster than our previous Maven based workflow. These tools also help you with dependency management so you keep your code organized. The killer feature is that you finally have true Continuous Integration because the build tool will figure out any dependencies of the code that you change, rebuilding it and testing to make sure that nothing is broken.
I don't trust incremental builds (whether via Bazel or any other tool) for an actual deployable build - I've seen them go wrong too many times.
For the local edit-test cycle incremental builds are great, but any decent IDE will do them in the multirepo case just as in the monorepo case.
For the local edit-test cycle incremental builds are great, but any decent IDE will do them in the multirepo case just as in the monorepo case.
Wait what? As long as the build process is reproducible, why do you care if it's incremental or not? If you don't trust bazel, people have been using distcc+ccache since forever.
Just as an anecdote, in all my time as a professional software developer, the number of projects I have worked on that have enjoyed reliable, reproducible, incremental builds is zero. The effects of global optimisations change. The output code winds up varying based on the order of incremental changes. Updated dependencies get overlooked because of race conditions or networking glitches or filesystem bugs or a hundred other unlikely but not impossible failures.
Incremental builds are great for quick day-to-day development, but in my experience they are invariably unsuitable for robust production builds. For me, those should use a git clone or equivalent into an empty directory and then a completely clean build of everything, always.
Incremental builds are great for quick day-to-day development, but in my experience they are invariably unsuitable for robust production builds. For me, those should use a git clone or equivalent into an empty directory and then a completely clean build of everything, always.
> As long as the build process is reproducible, why do you care if it's incremental or not?
I wouldn't care if it was reliable. My experience is that it isn't.
> If you don't trust bazel, people have been using distcc+ccache since forever.
And it's been causing inconsistent builds all the while.
I wouldn't care if it was reliable. My experience is that it isn't.
> If you don't trust bazel, people have been using distcc+ccache since forever.
And it's been causing inconsistent builds all the while.
I've been using bazel (and the Google-internal version, blaze) for years now. The builds are extremely reliable and are used to deploy loads of projects across Google every day. I no longer work there, but my new company is also using bazel and we never run into issues with unreliable builds.
I think you're falling into the trap of "past solutions didn't work, therefore the problem is intractable", which is a special case of the "throwing the baby out with the bathwater" fallacy. Sometimes a new tool comes out that fixes the problems of past tools.
I think you're falling into the trap of "past solutions didn't work, therefore the problem is intractable", which is a special case of the "throwing the baby out with the bathwater" fallacy. Sometimes a new tool comes out that fixes the problems of past tools.
> I think you're falling into the trap of "past solutions didn't work, therefore the problem is intractable", which is a special case of the "throwing the baby out with the bathwater" fallacy. Sometimes a new tool comes out that fixes the problems of past tools.
Where's the breakthrough then? What's the magic that allows bazel to succeed where dozens of similar efforts have failed? Sometimes a new tool does fix the problems of past tools, but that's almost always due to some new insight.
Where's the breakthrough then? What's the magic that allows bazel to succeed where dozens of similar efforts have failed? Sometimes a new tool does fix the problems of past tools, but that's almost always due to some new insight.
Not sure exactly, since I haven't used any of the dozens of similar efforts. It sounds like you're a lot more familiar with the area, so here's some resources:
* Bazel's FAQ: https://bazel.build/faq.html
* A description on how Blaze works: https://google-engtools.blogspot.com/2011/08/build-in-cloud-...
Unfortunately the design doc for Blaze is internal and AFAIK not available publicly. I imagine they took a look at a number of the similar efforts and made decisions based on what those approaches did wrong.
* Bazel's FAQ: https://bazel.build/faq.html
* A description on how Blaze works: https://google-engtools.blogspot.com/2011/08/build-in-cloud-...
Unfortunately the design doc for Blaze is internal and AFAIK not available publicly. I imagine they took a look at a number of the similar efforts and made decisions based on what those approaches did wrong.
While I can relate to your point, if you actually experience Blaze at Google (and Bazel too, but it has other ecosystem immaturity issues not related to reproducibility) you will quickly change your mind. I can totally see why prior tools you used may have put bad taste in your mouth.
The concern is that an incremental build is not reproducible. In theory, it is, but in practice, sometimes it is not.
Doing a clean build for the final artifact that is going to be released is pretty standard practice, IME.
Doing a clean build for the final artifact that is going to be released is pretty standard practice, IME.
Have you used bazel? The number of times I've even considered using `clean` (the drop some caches make everything mostly clean though still use the global cache command) in my 2+ years with it I can count one one hand, and most of those were unnecessary.
My point was about the the fact that by declaring your dependencies explicitly, Bazel allows you to compile only the things you depend on, not everything in the monorepo. And once you have built your dependencies once, you don't have to build them again.
If you think about it, a package manager workflow (like maven, nuget, etc) is also a form of incremental compilation. You compile your libraries only once and publish them in binary form in some sort of repository and then import them. The difference is that if you want to make a change in both a library and an application this becomes hard to do with packages, while very easy with a monorepo and Bazel.
If you think about it, a package manager workflow (like maven, nuget, etc) is also a form of incremental compilation. You compile your libraries only once and publish them in binary form in some sort of repository and then import them. The difference is that if you want to make a change in both a library and an application this becomes hard to do with packages, while very easy with a monorepo and Bazel.
> If you think about it, a package manager workflow (like maven, nuget, etc) is also a form of incremental compilation. You compile your libraries only once and publish them in binary form in some sort of repository and then import them.
The difference is that that's explicit and visible, rather than being "magically" figured out by the build system. And the compiled binaries are 100% reproducible everywhere since it's literally the same file from the repository everywhere. So it's much more reliable, understandable, and debuggable.
> The difference is that if you want to make a change in both a library and an application this becomes hard to do with packages, while very easy with a monorepo and Bazel.
Having a library and application so tightly coupled that changes have to be simultaneous is poor practice, and will likely get you into trouble when the time comes to deploy them. Better to enforce that you test old application + new library (or vice versa) up front, than hit it for the first time in production.
The difference is that that's explicit and visible, rather than being "magically" figured out by the build system. And the compiled binaries are 100% reproducible everywhere since it's literally the same file from the repository everywhere. So it's much more reliable, understandable, and debuggable.
> The difference is that if you want to make a change in both a library and an application this becomes hard to do with packages, while very easy with a monorepo and Bazel.
Having a library and application so tightly coupled that changes have to be simultaneous is poor practice, and will likely get you into trouble when the time comes to deploy them. Better to enforce that you test old application + new library (or vice versa) up front, than hit it for the first time in production.
> A monorepo oriented build tool such as Bazel will make sure that you don't compile more than needed.
Not at all true in my experience. I use Buck, which is very similar. It constantly recompiles stuff it doesn't need to by any standard. I've seriously considered avoiding Mercurial bookmarks in favor of manual patch management to see if I can avoid triggering the cases where it gets stupid. Maybe it's a problem specific to Buck, but my conversations with peers at other companies suggest not (and only a tiny percentage of developers at any of these companies get a choice of which tool to use).
The other issue is that "more than needed" is subject to more than one interpretation. There's "more than needed" as code was actually written, and there's "more than needed" as code should have been written. Tooling can help with the first but not the second. If developers are sloppy about how their header files are structured, then a change to one that's widely used will make it "necessary" to recompile a lot of code - including code that didn't actually use the part that changed. Oddly enough, though, that "necessary" recompilation wouldn't occur if those same header files were split up. (BTW some languages make good design more difficult by requiring private implementation details in public header files, but that's a separate conversation.) Tooling can optimize what it can see, but what it sees might already be a tangle of dependencies added only because a monorepo made it easy.
Not at all true in my experience. I use Buck, which is very similar. It constantly recompiles stuff it doesn't need to by any standard. I've seriously considered avoiding Mercurial bookmarks in favor of manual patch management to see if I can avoid triggering the cases where it gets stupid. Maybe it's a problem specific to Buck, but my conversations with peers at other companies suggest not (and only a tiny percentage of developers at any of these companies get a choice of which tool to use).
The other issue is that "more than needed" is subject to more than one interpretation. There's "more than needed" as code was actually written, and there's "more than needed" as code should have been written. Tooling can help with the first but not the second. If developers are sloppy about how their header files are structured, then a change to one that's widely used will make it "necessary" to recompile a lot of code - including code that didn't actually use the part that changed. Oddly enough, though, that "necessary" recompilation wouldn't occur if those same header files were split up. (BTW some languages make good design more difficult by requiring private implementation details in public header files, but that's a separate conversation.) Tooling can optimize what it can see, but what it sees might already be a tangle of dependencies added only because a monorepo made it easy.
You are describing modifying someone else's unrelated code will cause the rebuild of your code right? This looks like your targets unintentionally depend on the other code.
To find out how/where the dependency was introduced, you can use bazel query 'somepath(your_target, some_other_file)'. Not sure if Buck have an equivalent, but bazel should work with Buck repo.
To find out how/where the dependency was introduced, you can use bazel query 'somepath(your_target, some_other_file)'. Not sure if Buck have an equivalent, but bazel should work with Buck repo.
> This looks like your targets unintentionally depend on the other code.
Sometimes it's unintentional, for example an include left in after it's no longer needed. Other times it's intentional but lazy. For example, I often see commits that pull in one dependency just to use one tiny little utility function, but that dependency pulls in ten others, which each pull in ten others, and before you know it any change in a hundred files forces yours to be recompiled even when that tiny utility function wasn't touched.
A responsible developer would refactor the header file to keep the number of spurious rebuilds down, but the world is full of irresponsible developers. With multiple repos, the public immutable part of an API and the private fluid part tend to find their way into separate files pretty quickly. With a monorepo they don't, which is kind of ironic because one of the purported benefits of a monorepo is that it's easy to reach into "someone else's code" and refactor it. Unfortunately, people just don't seem to use that ability for good often enough unless review culture forces them to.
Sometimes it's unintentional, for example an include left in after it's no longer needed. Other times it's intentional but lazy. For example, I often see commits that pull in one dependency just to use one tiny little utility function, but that dependency pulls in ten others, which each pull in ten others, and before you know it any change in a hundred files forces yours to be recompiled even when that tiny utility function wasn't touched.
A responsible developer would refactor the header file to keep the number of spurious rebuilds down, but the world is full of irresponsible developers. With multiple repos, the public immutable part of an API and the private fluid part tend to find their way into separate files pretty quickly. With a monorepo they don't, which is kind of ironic because one of the purported benefits of a monorepo is that it's easy to reach into "someone else's code" and refactor it. Unfortunately, people just don't seem to use that ability for good often enough unless review culture forces them to.
It sounds like it might be worth trying Bazel (I haven't used either). The parent made a suggestion of a specific tool with one anecdatum about performance, and your argument is that a different tool doesn't do the same thing.
My real argument, as represented by the majority of the text I wrote, was that tooling only solves one part of the problem. The lesser part, at that. But I guess that went over the downvote brigade's heads. Ditto for the fact that I did mention conversations with peers who used other tools (including Bazel BTW) and had similar experiences. Seriously, folks, read before you nitpick.
> Tooling can optimize what it can see, but what it sees might already be a tangle of dependencies added only because a monorepo made it easy.
In practice this is almost never because of "a monorepo" and more because people don't like being good about including what they use (see: any Titus Winters talk ever). If, for example you have :test_foo and :foo_lib, things are fine and dandy. When you modify foo_lib you'll run :test_foo and nothing else. But often people are lazy and you have :foobarbaz_lib which is built from three source files, foo, bar, and baz. Each of those has a test, but all the tests depend on :foobarbaz_lib, because that's all that exists. So when you change foo, you run :baz_test, even though in practice, baz might depend on foo, and not the reverse. You fix this by making :foo_lib, :bar_lib, and :baz_lib each with their associated :target_test and explicit dependencies, and then :foobarbaz_lib depends on all three libs (or doesn't exist at all, and everyone who needs :foo_lib uses :foo_lib directly.
So yes, absolutely, when/if you start (essentially) globbing your imports, bazel/buck/pants stop being awesome. So don't do that.
In practice this is almost never because of "a monorepo" and more because people don't like being good about including what they use (see: any Titus Winters talk ever). If, for example you have :test_foo and :foo_lib, things are fine and dandy. When you modify foo_lib you'll run :test_foo and nothing else. But often people are lazy and you have :foobarbaz_lib which is built from three source files, foo, bar, and baz. Each of those has a test, but all the tests depend on :foobarbaz_lib, because that's all that exists. So when you change foo, you run :baz_test, even though in practice, baz might depend on foo, and not the reverse. You fix this by making :foo_lib, :bar_lib, and :baz_lib each with their associated :target_test and explicit dependencies, and then :foobarbaz_lib depends on all three libs (or doesn't exist at all, and everyone who needs :foo_lib uses :foo_lib directly.
So yes, absolutely, when/if you start (essentially) globbing your imports, bazel/buck/pants stop being awesome. So don't do that.
> In practice this is almost never because of "a monorepo"
I think you mean in the practice you've seen. I've seen some practice too, and it seems much more prevalent in monorepos. Yeah, still a small sample, correlation does not equal causation, yadda yadda, but I still think it's because monorepos make it so damn easy to reach out and add a huge tree of dependencies without having to think about it. My whole point is that it's possible to do the right thing, but requires more discipline.
I think you mean in the practice you've seen. I've seen some practice too, and it seems much more prevalent in monorepos. Yeah, still a small sample, correlation does not equal causation, yadda yadda, but I still think it's because monorepos make it so damn easy to reach out and add a huge tree of dependencies without having to think about it. My whole point is that it's possible to do the right thing, but requires more discipline.
> add a huge tree of dependencies without having to think about it.
Keep in mind that the alternative is "I have to build (or pull a prebuilt version of) the entirety of every repo that I depend on". An entire repo is always going to be larger than a single target. Yes you're right that pulling in the entirely of {{some large tool}} because you want `tool.strings.util.mangle_name` is bad.
The alternatives are
1. (multi-repo) You stick mangle name in its own repo (think `left-pad`)
2. (monorepo) Depend on `tool` in its entirety
3. (either, though more common in multi-repo) Everyone re-implements mangle_name. If the way the name is mangled is important, and it changes in the future, you have major issues and incompatibilities. (think IPV4 -> IPV6 for an example where a "global" `parse_addr` function would have made everyone's lives easier). You can bet on the assumption that mangle_name will never change, but then you aren't doing software engineering[1]
All of these are the wrong solution
4. Core utilities like this get factored out into their own files. In a multi-repo environment, everyone depends on the entire core/util repo. In a monorepo environment, you depend on the granular files you need.
4 is the only correct solution, and note that in a monorepo environment it results in you depending on fewer things than in a multi-repo environment. Yes there are a lot of ways to do things wrong, But that's true no matter what tooling you use. When used "correctly", monorepos have the best outcome, and the bad outcomes aren't actually that bad (sometimes longer builds and binary bloat if you don't tree-shake). You always need discipline. Monorepos don't need more.
[1]: https://www.youtube.com/watch?v=tISy7EJQPzI
Keep in mind that the alternative is "I have to build (or pull a prebuilt version of) the entirety of every repo that I depend on". An entire repo is always going to be larger than a single target. Yes you're right that pulling in the entirely of {{some large tool}} because you want `tool.strings.util.mangle_name` is bad.
The alternatives are
1. (multi-repo) You stick mangle name in its own repo (think `left-pad`)
2. (monorepo) Depend on `tool` in its entirety
3. (either, though more common in multi-repo) Everyone re-implements mangle_name. If the way the name is mangled is important, and it changes in the future, you have major issues and incompatibilities. (think IPV4 -> IPV6 for an example where a "global" `parse_addr` function would have made everyone's lives easier). You can bet on the assumption that mangle_name will never change, but then you aren't doing software engineering[1]
All of these are the wrong solution
4. Core utilities like this get factored out into their own files. In a multi-repo environment, everyone depends on the entire core/util repo. In a monorepo environment, you depend on the granular files you need.
4 is the only correct solution, and note that in a monorepo environment it results in you depending on fewer things than in a multi-repo environment. Yes there are a lot of ways to do things wrong, But that's true no matter what tooling you use. When used "correctly", monorepos have the best outcome, and the bad outcomes aren't actually that bad (sometimes longer builds and binary bloat if you don't tree-shake). You always need discipline. Monorepos don't need more.
[1]: https://www.youtube.com/watch?v=tISy7EJQPzI
Your language and/or build system should provide visibility controls to limit what can be coupled. Package-privacy exists for a reason.
Yes you can post a PR that relaxes someone else’s control but they can block it.
Yes you can post a PR that relaxes someone else’s control but they can block it.
This is a concern with monorepo but you can still enforce the boundaries programmatically. For example you can add some simple, custom lint rules that prevent cross-boundary imports.
Another thing you can do is have your CI only check out specific directories when running tests using something like git’s sparse-checkout.
Another thing you can do is have your CI only check out specific directories when running tests using something like git’s sparse-checkout.
I like your comment about commits spanning components. Having many repos forces my team to think about how to stage changes that span many repos.
TBH, I'm not convinced that a great effort to think about "how to stage changes that span many repos" is always (or maybe even often) a benefit. It seems like the simpler approach of just making it easy to make multimodule changes is a benefit of the monorepo.
That depends on your deployment model. Google is live at head, with no support for past releases. In the web world this is reasonable: they control the servers and thus can force releases. I don't work in that world though, there are times where I have to support customers on older releases who refuse to upgrade (when the customer is spending millions of dollars per year you can't ignore their rules no matter who stupid you think they are). Thus I need to think about those factors anyway.
Google still has the notion of release branches. They're point-in-time snapshots of the monorepo (copy on write), plus the occasional cherry-pick from HEAD.
Development happens at HEAD is the important part.
Development happens at HEAD is the important part.
Having to stage changes across multiple repos, and minimize dependencies between them, can definitely create more work up front. The point is that it lessens the recurring cost (not just across time but across many developers) of having those spurious dependencies in the code forever. Code spends more time being maintained than being written, so it's the recurring cost that dominates long-term productivity.
In my experience, the first time the team tries to separate concerns, they don't necessarily fully understand the problem, and build out the wrong separations.
Multirepo teams live with the wrong modularity boundaries longer than they should, because fixing the design ("violating the rules," when they're bad rules) causes immediate pain. In monorepo it's easier to do the refactor in an atomic commit.
It's all part of the sandboxing cycle. https://xkcd.com/2044/
Multirepo teams live with the wrong modularity boundaries longer than they should, because fixing the design ("violating the rules," when they're bad rules) causes immediate pain. In monorepo it's easier to do the refactor in an atomic commit.
It's all part of the sandboxing cycle. https://xkcd.com/2044/
When I was a contractor at Google I was surprised by the mono repo, but really liked it. Scaling issues aside, it makes sense having all dependencies in one repo.
I thought of having a mono repo at home because many of my projects are inter related but instead I spent some time with a library setup. For Common Lisp, I used a pattern in one of Zach’s blogs to have one central repo for all of my personal Common Lisp code that is configured as a root for Quicklisp. Any of my code can be quick loaded no matter where I am working. Really simple but it makes it possible to write small utilities/libraries and reuse them for whatever I am doing.
I sort of do the same for Python, writing a setup.py for each small project or library to install globally, including appropriate executable scripts. This also allows me to combine small things easily. I am not yet set up to do this yet for my Haskell projects but it is on my todo list.
I thought of having a mono repo at home because many of my projects are inter related but instead I spent some time with a library setup. For Common Lisp, I used a pattern in one of Zach’s blogs to have one central repo for all of my personal Common Lisp code that is configured as a root for Quicklisp. Any of my code can be quick loaded no matter where I am working. Really simple but it makes it possible to write small utilities/libraries and reuse them for whatever I am doing.
I sort of do the same for Python, writing a setup.py for each small project or library to install globally, including appropriate executable scripts. This also allows me to combine small things easily. I am not yet set up to do this yet for my Haskell projects but it is on my todo list.
I merged all my personal C++ projects into one monorepo and I really liked it. Especially useful with C++ since setting up the build system is so cumbersome.
That's what I do too. I have a git repo with submodules for each library and service that's related to the overall thing. So, while I'm working on it, locally, it almost looks like a monorepo, but the way things are built, run, deployed and stored in remote git is as separate repos and artifacts.
If changes regularly involve multiple submodules, then I know that the boundaries are wrong and refactoring is needed.
If changes regularly involve multiple submodules, then I know that the boundaries are wrong and refactoring is needed.
Hmm, if you're working on a single product (like the author is), then of course it makes sense to stick them in the same repository. But isn't this just common sense?
I mean, if you have a single product split over many repositories then you will run into the issues mentioned by the author: A change will invariably involve multiple repositories. Again, to me this seems to be just an indication that you are, in fact, working on a single product.
I mean, if you have a single product split over many repositories then you will run into the issues mentioned by the author: A change will invariably involve multiple repositories. Again, to me this seems to be just an indication that you are, in fact, working on a single product.
This is obvious to me (a monorepo advocate) but most people these days tend to organise their repos based on module/implementation lines rather than on product boundaries.
My motto is: "What you release together, you should version-control together".
My motto is: "What you release together, you should version-control together".
I've learnt that for "What you release together, you should version-control together", the phrase "released together" needs to be defined as "what gets globally updated at once". If you provide anything for customers to implement, you cannot update those globally at once (because you need the customer to update), and should think about using a multi-repo set-up.
My experience is that testing the interactions of historic versions of different components in a mono-repo against each other is difficult. With a multi-repo set-up, one can checkout to whatever historic version one needs to test. That makes it trivially easy to set-up testing that allows one to test historic versions of components against each.
My experience is that testing the interactions of historic versions of different components in a mono-repo against each other is difficult. With a multi-repo set-up, one can checkout to whatever historic version one needs to test. That makes it trivially easy to set-up testing that allows one to test historic versions of components against each.
I'm a multirepo advocate, but I suggest splits similarly.
Maybe it's my old-timey way of releasing software, but because I do releases as RPMs, it bothers me less if you do things like have your base library, your backend services, and your frontend in the same repo. If they need to be able to be deployed independently, I just split them up as subpackages.
I guess most language package managers don't really have such a concept, so people are optimizing for that...
Maybe it's my old-timey way of releasing software, but because I do releases as RPMs, it bothers me less if you do things like have your base library, your backend services, and your frontend in the same repo. If they need to be able to be deployed independently, I just split them up as subpackages.
I guess most language package managers don't really have such a concept, so people are optimizing for that...
I like having one repo for the project ("What you release together") that tracks subprojects as submodules.
It's a common sense, but it's really hard to argue against super-fine splitting. Having repo per module seems like stronger separations of concerns from a distance.
Of course, what in the end really decides about the strength of your concern separation is the shape of your CI, but alas, repository granuality is just much more visible then CI checks.
Of course, what in the end really decides about the strength of your concern separation is the shape of your CI, but alas, repository granuality is just much more visible then CI checks.
I would argue that if you have clear separation of concerns you wouldn't need to make changes in multiple repositories all the time.
Well let's take frontend/backend separation in a classic webapp -- let's not think about React for now. This makes good concern separation as rendering HTML is something else than computing prices.
If you put frontend and backend into different repositories, you need to make two PRs for every new feature and -- for much worse -- you make integration testing much harder as you don't have atomic product merges to run automation against.
Don't laugh about this example, my corporation is doing stuff like this (backend only repo tightly coupled to frontend repo) all the time.
So maybe we can steelman the argument into "good vertical concern separation doesn't force you into doing multiple PR for single feature, as opposed to horizontal frontend/backend/repository splits."?
But I'm still not sure. Wide integration testing is necesity for reliable microservices. It's difficult to do that with dependencies being out of the scope of the change.
If you put frontend and backend into different repositories, you need to make two PRs for every new feature and -- for much worse -- you make integration testing much harder as you don't have atomic product merges to run automation against.
Don't laugh about this example, my corporation is doing stuff like this (backend only repo tightly coupled to frontend repo) all the time.
So maybe we can steelman the argument into "good vertical concern separation doesn't force you into doing multiple PR for single feature, as opposed to horizontal frontend/backend/repository splits."?
But I'm still not sure. Wide integration testing is necesity for reliable microservices. It's difficult to do that with dependencies being out of the scope of the change.
Integrating with other products is something that goes beyond the scope of SCM repository layout, though. If you change your API then clearly someone on the other side also needs to adjust, but that's not simply sending a pull request. That's also making sure you deploy your products in the right order, ensure backwards compatibility and so on.
Re integration testing: That's why things like contract testing are becoming more popular. You make sure that you do as much testing as you can without relying on an entire ecosystem of depdendencies being available, so once you DO release something you can be fairly confident that it's already working as expected.
Re integration testing: That's why things like contract testing are becoming more popular. You make sure that you do as much testing as you can without relying on an entire ecosystem of depdendencies being available, so once you DO release something you can be fairly confident that it's already working as expected.
I mostly agree. Just notes:
Classic example in favor of monorepos is shared internal library. You should be able to change its API in one step, because that's the way to not having to care about compatibility at all. Compatibility concerns only happen if there's a time difference and with atomic company-wide update, this just doesn't need to happen.
In my example, backend and frontend can be deployed together. And we can discuss whether they should -- I would say there is little benefit of maintaining such single-purpose, wide interface just for getting the ability deploy frontend changes only, but this would depend on care the team put into configuration of cluster orchestration.
Contract testing is good, but not as good as testing against the real thing. Very much depend on the circumstances though.
Classic example in favor of monorepos is shared internal library. You should be able to change its API in one step, because that's the way to not having to care about compatibility at all. Compatibility concerns only happen if there's a time difference and with atomic company-wide update, this just doesn't need to happen.
In my example, backend and frontend can be deployed together. And we can discuss whether they should -- I would say there is little benefit of maintaining such single-purpose, wide interface just for getting the ability deploy frontend changes only, but this would depend on care the team put into configuration of cluster orchestration.
Contract testing is good, but not as good as testing against the real thing. Very much depend on the circumstances though.
> then of course it makes sense to stick them in the same repository. But isn't this just common sense?
It depends. Are you planning more projects which may share functionality? Are you going to release within a company with other teams that may want to integrate? Are some components managed by dedicated teams who may have their own cycles of work? Are you planning to split off a specific service soon?
Or to quote risky business: what is the problem we're trying to solve?
It depends. Are you planning more projects which may share functionality? Are you going to release within a company with other teams that may want to integrate? Are some components managed by dedicated teams who may have their own cycles of work? Are you planning to split off a specific service soon?
Or to quote risky business: what is the problem we're trying to solve?
> ... may ...
If something only may be the case, then I'd say its premature effort to plan for something that may or may not happen. Better to do the simplest thing now and then refactor when the need arises.
If something only may be the case, then I'd say its premature effort to plan for something that may or may not happen. Better to do the simplest thing now and then refactor when the need arises.
What about a single product with multiple components though? Let's say you have an API and a browser client (SPA). Should you keep them in the same repo, or separate them?
So far the only good reason I've had for separating them has been related to deployment concerns (e.g. your API is deployed to AWS and your browser client is on Cloudflare).
So far the only good reason I've had for separating them has been related to deployment concerns (e.g. your API is deployed to AWS and your browser client is on Cloudflare).
If you have an API and a SPA that's tightly coupled then, yes, same repo.
If your API is a separate product with a separate release cycle, proper testing and backwards compatibility checks, then different repo.
If your API is a separate product with a separate release cycle, proper testing and backwards compatibility checks, then different repo.
Sounds like you need/could split it up into core/engine functionality, and then implementation of that core in different environments (i.e. a ios and android one).
That makes sense to me, to have 3 repositories. Common code, perhaps even in your own dsl. And then implementations done in the various platforms.
That makes sense to me, to have 3 repositories. Common code, perhaps even in your own dsl. And then implementations done in the various platforms.
For my part, I've worked with two long-lived monorepos so far. Both were quite small, all told, but, nonetheless, in both cases I saw the monorepo largely serving as an incubator for bit rot.
All those direct interdependencies, combined with the impossibility of just using package versioning to allow a library's consumers to upgrade at their own pace, meant that developers had a strong disincentive to do any cleanup that might result in a breaking change. They'd tend to prefer patches instead. So the long-term trend was for everything to eventually become a tangled mass of patchwork.
Google has entire teams devoted to combatting this problem. I'm not sure most companies can (or want to) afford that.
This is one of those spots where I think it's wise to be cautious about blog-driven development. By analogy, I've seen experienced woodworkers have fun with mocking the DIY furniture projects that come up in places like Pinterest. One common criticism was that, since wood expands and contracts with changing humidity, but only in certain directions, if you put it together in certain ways, you end up with a product that will literally tear itself apart over time. Of course, this isn't apparent from the Instructable article, because it's a slow process, and all those pictures were taken when the piece was still brand spanking new. It's also not something you're likely to see written in the comments section or anything like that - the people I've known who know this sort of thing aren't generally the type to write Instructables articles or maintain their own DIY blogs.
I suspect something like this also happens with any decent sized codebase: Code is a living thing that expands and contracts with changing business requirements. Over time, it will twist itself out of shape. Weak coupling and fragmented codebases are two different (though related) ways of trying to limit the damage from this effect. They both bring their own challenges, though, and aren't perfect. Monorepos don't have the same drawbacks, but that doesn't mean they don't have any. One, I think, is that you're left directly exposed to the full strength of the forces that cause bit rot.
If you're disciplined enough to recognize this and treat it as another incentive to attack bit rot head-on, then you'll probably end up in a better place for it. If you think monorepos are a panacea, though, you're setting yourself up for the kinds of failure that people generally don't like to blog about.
All those direct interdependencies, combined with the impossibility of just using package versioning to allow a library's consumers to upgrade at their own pace, meant that developers had a strong disincentive to do any cleanup that might result in a breaking change. They'd tend to prefer patches instead. So the long-term trend was for everything to eventually become a tangled mass of patchwork.
Google has entire teams devoted to combatting this problem. I'm not sure most companies can (or want to) afford that.
This is one of those spots where I think it's wise to be cautious about blog-driven development. By analogy, I've seen experienced woodworkers have fun with mocking the DIY furniture projects that come up in places like Pinterest. One common criticism was that, since wood expands and contracts with changing humidity, but only in certain directions, if you put it together in certain ways, you end up with a product that will literally tear itself apart over time. Of course, this isn't apparent from the Instructable article, because it's a slow process, and all those pictures were taken when the piece was still brand spanking new. It's also not something you're likely to see written in the comments section or anything like that - the people I've known who know this sort of thing aren't generally the type to write Instructables articles or maintain their own DIY blogs.
I suspect something like this also happens with any decent sized codebase: Code is a living thing that expands and contracts with changing business requirements. Over time, it will twist itself out of shape. Weak coupling and fragmented codebases are two different (though related) ways of trying to limit the damage from this effect. They both bring their own challenges, though, and aren't perfect. Monorepos don't have the same drawbacks, but that doesn't mean they don't have any. One, I think, is that you're left directly exposed to the full strength of the forces that cause bit rot.
If you're disciplined enough to recognize this and treat it as another incentive to attack bit rot head-on, then you'll probably end up in a better place for it. If you think monorepos are a panacea, though, you're setting yourself up for the kinds of failure that people generally don't like to blog about.
> an incubator for bit rot
Genius phrase. Overall, that might be one of the best comments I've ever read on HN. The DIY furniture analogy is a good one. There's so much focus on what seems to work today, and so little awareness of what affects long-term maintainability. I see this at work too, everyone chasing "impact" for the half even when it screws us over any longer time scale. Understanding how code evolves over time is an essential part of being a competent developer. It's much more important than any in-the-moment coding test.
Genius phrase. Overall, that might be one of the best comments I've ever read on HN. The DIY furniture analogy is a good one. There's so much focus on what seems to work today, and so little awareness of what affects long-term maintainability. I see this at work too, everyone chasing "impact" for the half even when it screws us over any longer time scale. Understanding how code evolves over time is an essential part of being a competent developer. It's much more important than any in-the-moment coding test.
Can you help me understand what bit rot means as you are using it, and why it is different in a mono repo vs an equally large multi-repo system?
Just cherry-picking one rather extreme example: There was a method in a core library that had an argument that was not used. Few people knew this, and it was one that was called at least once by most services.
When I discovered this, I asked the module's maintainer about it. He explained that it used to be used, but requirements changed so that it wasn't needed anymore. He decided to leave it anyway, though, because the alternative would have been an immense effort to track down all the call sites, remove the argument, then trace all the resulting dead code, and remove that as well. It would have eaten up a week, I'm sure. Everyone had bigger fish to fry, and always would have bigger fish to fry.
As I understand it, Google gets around that problem by having a group of people where frying fish like that is their primary responsibility.
When I discovered this, I asked the module's maintainer about it. He explained that it used to be used, but requirements changed so that it wasn't needed anymore. He decided to leave it anyway, though, because the alternative would have been an immense effort to track down all the call sites, remove the argument, then trace all the resulting dead code, and remove that as well. It would have eaten up a week, I'm sure. Everyone had bigger fish to fry, and always would have bigger fish to fry.
As I understand it, Google gets around that problem by having a group of people where frying fish like that is their primary responsibility.
I don't understand how avoiding bitrot is better in a multi-repo setup. With a multi-repo, if you're changing a core-library you can make the change quickly but you can't upgrade clients until they upgrade on their own. You still have bitrot. The difference is that the bitrot isn't visible because it's compartmentalized into separate repos.
So, to me, different things using different versions of the same library isn't inherently bitrot. For one, if you're actively maintaining your code, then everything's going to eventually end up upgraded. It's just a question of whether you want to boil the ocean all at once, in one big pot, or do it in a series of smaller batches.
There are legitimate arguments for both options. The thing I'm trying to say is, you should beware the idea that going monorepo means the ocean's going to just boil itself somehow.
There are legitimate arguments for both options. The thing I'm trying to say is, you should beware the idea that going monorepo means the ocean's going to just boil itself somehow.
The reason that a monorepo facilitates this kind of bit-rot is because of one of the advantages of the monorepo setup: that it also facilitates easily making connections between different parts of the codebase. In a multi-repo setup, you need to be intentional about when and how you pull in another repo, because the bar to doing so is so high. In a monorepo, every piece of every part of the code is available to you and no extra action is required to "pull in" that other repo, which means in practice that without tooling or discipline you end up with a codebase with lots of tight interconnections.
So the reason it happens in a monorepo more is that those connections are easier to make and therefore a lot more common, not because those connections are impossible in a multi-repo setup.
So the reason it happens in a monorepo more is that those connections are easier to make and therefore a lot more common, not because those connections are impossible in a multi-repo setup.
I see, the argument is that monorepo facilitates tight-coupling because it's easy to add dependencies. The multi-repo doesn't have the same problem because it's more difficult to have cross module dependencies.
There's a number of solutions to prevent tight-coupling.
1. Language-level visibility modifiers. 2. Build system visibility. Bazel offers fine-grained visibility. I'm not sure about others. 3. Using RPCs as the primary interconnect between services.
My view is that disadvantages of tight-coupling in a mono-repo is a much better problem to have than the disadvantages of having logic spread across different repos.
There's a number of solutions to prevent tight-coupling.
1. Language-level visibility modifiers. 2. Build system visibility. Bazel offers fine-grained visibility. I'm not sure about others. 3. Using RPCs as the primary interconnect between services.
My view is that disadvantages of tight-coupling in a mono-repo is a much better problem to have than the disadvantages of having logic spread across different repos.
> As I understand it, Google gets around that problem by having a group of people where frying fish like that is their primary responsibility.
I'd rephrase this: Google gets around this problem by having tooling (one such tool is the monorepo) that makes fixing issues like these fairly simple. A straightforward example like this one could be fixed with sed/clangtidy/refaster (depending on the language) + Rosie[1] in a few days, even if there were thousands (or 10s of thousands) of uses. But the work of upgrading is borne by the person doing the upgrade, not client teams who suddenly encounter compile errors when they try to version bump. This is much "fairer".
[1]: https://cacm.acm.org/magazines/2016/7/204032-why-google-stor...
I'd rephrase this: Google gets around this problem by having tooling (one such tool is the monorepo) that makes fixing issues like these fairly simple. A straightforward example like this one could be fixed with sed/clangtidy/refaster (depending on the language) + Rosie[1] in a few days, even if there were thousands (or 10s of thousands) of uses. But the work of upgrading is borne by the person doing the upgrade, not client teams who suddenly encounter compile errors when they try to version bump. This is much "fairer".
[1]: https://cacm.acm.org/magazines/2016/7/204032-why-google-stor...
Funny you mention this as a drawback, while it's actually a strength of mono repo that is perfectly handled.
For a widely used library. Leave the argument. This argument could be defaulted to NULL, the function can be marked @deprecated and it can log a warning message that the argument has no effect (depending on the language). That's how to handle updates with backward and forward compatibility. No need to break user code.
That's a strength of a mono repo. It only lets you do the right thing. You can't ignore other developers in the company and break all their software.
In a typical multi repo, there is the luxury to change anything anytime, as often as a developer feels like renaming a variable or a parameter. Creating a ton of unnecessary work for other developers who have the pain to keep up. That is, if they ever attempt to upgrade any library, there is no benefit to upgrading and it's impossible to keep up with all changes going on in all libraries thus it's rarely done in practice. The more it's behind, the more work there is to upgrade, the less likely it is to be done.
For a widely used library. Leave the argument. This argument could be defaulted to NULL, the function can be marked @deprecated and it can log a warning message that the argument has no effect (depending on the language). That's how to handle updates with backward and forward compatibility. No need to break user code.
That's a strength of a mono repo. It only lets you do the right thing. You can't ignore other developers in the company and break all their software.
In a typical multi repo, there is the luxury to change anything anytime, as often as a developer feels like renaming a variable or a parameter. Creating a ton of unnecessary work for other developers who have the pain to keep up. That is, if they ever attempt to upgrade any library, there is no benefit to upgrading and it's impossible to keep up with all changes going on in all libraries thus it's rarely done in practice. The more it's behind, the more work there is to upgrade, the less likely it is to be done.
I don't get how people can live like this. In any decent language with a primitive static type system this would take you 15 minutes at most and you could be sure that it works afterwards.
This is exactly why we are adopting a monorepo: it makes tasks like this tractable. Across thousands of microrepos, they are not.
I've been an avid user of docker-compose for web app type projects. In this case, I've found putting everything in one repo perfectly manageable. Usually this means the backend, frontend, nginx and any other random microservices are just in separate folders in the base repo along with a compose yaml file. I gather that if you are using more complex CI/CD tools this becomes more complicated...
Side note: blog posts and news articles that don't display the date are very irritating. There has been a lot of discussions about monorepos, I want to know if this is something new adding to the debate or something old. If it's a news article, I want to know how up to date it is. All I get from this is the slug saying '2019'.
I decided to read this expecting to see an explanation as to why my DRF backend should be in the same repository as my React Native frontend, and instead found a discussion on why my React Native code shouldn't be broken up into iOS and Android.
Not sure this is what people mean by a monorepo ...
Not sure this is what people mean by a monorepo ...
I tried deploying my DRF backend[1] + Nuxt.js[2] demo-app on Heroku[3] as one repo but it was a NIGHTMARE. Yes, it sucks to have to open both projects in my IDE in development, but it's nowhere near the difficulty I had with the mono-repo. Plus, I can find things much faster within each project when they are split.
[1]: backend → https://github.com/SHxKM/django-vue-ssr
[2]: frontend → https://github.com/SHxKM/django-nuxt-ssr-front
[3]: live-app → https://django-nuxt-ssr.herokuapp.com/ (on a free dyno, could crash on first load)
[1]: backend → https://github.com/SHxKM/django-vue-ssr
[2]: frontend → https://github.com/SHxKM/django-nuxt-ssr-front
[3]: live-app → https://django-nuxt-ssr.herokuapp.com/ (on a free dyno, could crash on first load)
I've settled on a many-repos-in-same-parent-folder pattern, which means that I can open them all in the same editor window if I want to.
I've also forgone proper IDE "project" in favour of opening the directory in my editor on an ad-hoc basis from the command line (e.g. `cd frontend && subl .`). It's not perfect, but it avoids the issue of having too many windows, or having to fumble through IDE project switching dialogs.
I've also forgone proper IDE "project" in favour of opening the directory in my editor on an ad-hoc basis from the command line (e.g. `cd frontend && subl .`). It's not perfect, but it avoids the issue of having too many windows, or having to fumble through IDE project switching dialogs.
[deleted]
monorepo means that the whole product is inside a single repo. Like google and fb do it.
The CI story has been my biggest gripe with monorepos. I'm surprised that no separate tool has emerged, like the "custom Jenkins scripts" mentioned in the article.
Maybe I'm missing something, but couldn't you get pretty far with something like:
Maybe I'm missing something, but couldn't you get pretty far with something like:
# This tells the tool where to look for changes
packages_folder: ./packages
# List dependencies. In this instance, if any files in `shared_api` change, frontend and backend need to build.
dependencies:
- frontend:
- shared_api
- backend:
- shared_api
# Definite build order
build_order:
- shared_api
- backend
- frontend
# Somehow find these scripts in each package subfolder and try to run them.
scripts:
- build
- test
- deploy
One advantage that isn't mentioned in the article: code examples. As a monorepo grows, it's more likely that someone else using the repo somewhere has attempted to do the same thing that you want to do, and you can use tools like searchcode to easily find usages and see how they did it. Often this works better than documentation, which for internal projects tends to be non-existent or out-of-date.
One disadvantage: a lot of IDEs don't cope well with monorepos as they try to index the entire thing.
One disadvantage: a lot of IDEs don't cope well with monorepos as they try to index the entire thing.
One of the projects I'm on at work just switched several repos into a mono repo. The issue was that there were often related changes to the Database, UI, Configuration and API projects... Those four in particular were almost always released for 2-3 of them for a given feature. Coordinating releases to dev/qa servers even was a bit of a pain. Merging them removed most of this.
There are several smaller/micro services that are still separate, however. They're generally less effected by the coordination of db/api/ui changes.
There are several smaller/micro services that are still separate, however. They're generally less effected by the coordination of db/api/ui changes.
I read this, and I still don't know what "many benefits" they're referring to. I basically read one in the article -- a monorepo avoids the pain associated with making changes to multiple repo at once, which was happening frequently. Does the author make reference to some other benefits that I just completely missed...?
> This makes CI test runs slightly longer, as they first need to do a fresh checkout.
-depth=1 ameliorates this somewhat (though plays poorly with tag-heavy workflows).
-depth=1 ameliorates this somewhat (though plays poorly with tag-heavy workflows).
Summary for those who can't be bothered to read: we had two repos and copy/pasted code that needed to be in both instead of adding more repos. That copy/paste had pain and we solved it by going to one repo.
They could have solved the pain by adding more repos as well. They never considered this as far as I can tell.
They could have solved the pain by adding more repos as well. They never considered this as far as I can tell.
They specifically mentioned breaking code outbid to more repos:
> We had three choices: ... Extract Core out of PSPDFKit-Android and again deal with fragmented pull requests.
> We had three choices: ... Extract Core out of PSPDFKit-Android and again deal with fragmented pull requests.
That is a very uncharitable summary. The article discusses codebases being developed together or growing closer together over time, causing feature changes in one to need changes in another. It doesn't mention copy/pasting anything. They solved it by moving everything into a single repo.
Adding more repos would once again split up pull requests for features that touch multiple codebases, which is the reason they merged two codebases into one repo in the first place.
Adding more repos would once again split up pull requests for features that touch multiple codebases, which is the reason they merged two codebases into one repo in the first place.
I will admit to being uncharitable.
I am assuming they did copy/paste coding to get features into both. It is possible that they actually did write everything twice, but that is generally such a stupid idea I can't believe it (except for UI - it makes perfect sense to write a the UI for android and iOS completely separate - the logic behind the UI should still be common, but the UI elements/APIs are different enough that common probably isn't worth it)
Adding more repos should not split up pull requests. If it does that is a function of bad architecture.
I am assuming they did copy/paste coding to get features into both. It is possible that they actually did write everything twice, but that is generally such a stupid idea I can't believe it (except for UI - it makes perfect sense to write a the UI for android and iOS completely separate - the logic behind the UI should still be common, but the UI elements/APIs are different enough that common probably isn't worth it)
Adding more repos should not split up pull requests. If it does that is a function of bad architecture.
It sounds like this person would have benefited from using Subversion or Perforce. Subversion 1.12 has alpha support for a limited local branch (called Checkpoints).
I really wish there existed more tools, it seems like there is a gap for some tooling.
It would be a simple one that could check the git log and see what files has changed since last time.
Or check the hash of each folder and compare it to previous commit.
If you use Docker, you can leverage that docker will take a hash of the context and if the images already exist on the machine use the cache if nothing has changed for those files. That is also quite effective +1
It would be a simple one that could check the git log and see what files has changed since last time.
Or check the hash of each folder and compare it to previous commit.
If you use Docker, you can leverage that docker will take a hash of the context and if the images already exist on the machine use the cache if nothing has changed for those files. That is also quite effective +1
It could well read like "The Many Benefits of Using Repo Modules"
To me it feels like a misguided attempts to work around the need of having a package manager
Why not simply build your SRPM or DEB package repo for code? Worked like magic for me on four of my past workplaces.
This also works towards building your culture of stable releases instead of non-stop development and hotfixes.
To me it feels like a misguided attempts to work around the need of having a package manager
Why not simply build your SRPM or DEB package repo for code? Worked like magic for me on four of my past workplaces.
This also works towards building your culture of stable releases instead of non-stop development and hotfixes.
Approaches such as Fan In supported by GoCD are of great help when working with monorepos.
See https://docs.gocd.org/current/advanced_usage/fan_in.html
What tools are people using to manage multiple repos? A search reveals many, but what is the latest and greatest?
NuGET, Maven, whatever is the package manager for the language in question.
The outcome of multiple repos should be modular libraries.
The outcome of multiple repos should be modular libraries.
Unless you're a library vendor, the outcome of multiple repos is most likely a single final application. Or a few applications.
That is not a reason not to write modular code.
Modular code works just fine in subdirectories.
It can work either way. There are pros can cons to both ways, code organization needs to be done correctly either way. Nothing about either ensures your code is a good modular structure vs a haphazard mess. In most cases code in fact has a modular structure behind it that is followed - but the structure is not always good (something that is easy to see in hindsight, but the wrong rules were put in place early and now it is hard to change)
Mono repo makes violation of modular rules easier, just link it in - in a complex project odds are nobody will notice. However this doesn't mean monorepo is bad, it just needs that you need to be careful to ensure the rules are written down.
Multi-repo makes violation of those same rules harder - you have to make a bunch of changes to ensure that the bad dependency is in the environment. This hints at the downside of multi-repo: you need to have some sort of dependency management system to ensure that the right version is there when you build.
Mono repo makes violation of modular rules easier, just link it in - in a complex project odds are nobody will notice. However this doesn't mean monorepo is bad, it just needs that you need to be careful to ensure the rules are written down.
Multi-repo makes violation of those same rules harder - you have to make a bunch of changes to ensure that the bad dependency is in the environment. This hints at the downside of multi-repo: you need to have some sort of dependency management system to ensure that the right version is there when you build.
Dependency management systems are now a standard thing and they are integrated with CI servers very well - I would not say that the need to use it is a downside.
Depends on your target environment. Some targets/languages have great eco systems, some do not.
Not when it gets compiled from scratch all the time, or is open to spaghetti code modifications.
Modular code in subdirectories is what gave microservices an excuse against the "monolith".
Modular code in subdirectories is what gave microservices an excuse against the "monolith".
Subdirectories can belong to different repos. In modern IDEs like Intellij or Eclipse it's possible to work with multiple project roots in the same window (e.g. if you have a common library for microservices and want to redesign its API and make changes everywhere at once). The good thing here is that in most cases you work with just a fraction of your code and with a single project root, switching to multi-project view only in rare big reengineering tasks.
> The outcome of multiple repos should be modular libraries.
Yes; also they should all be treated as separate, independent products. Where multi-repos go wrong is when a whole product spans multiple repos, and a feature of the product depends on features being added or altered in multiple projects.
Ideally, the split should be so that nobody has to work across multiple repos to finish a feature.
The github / NPM ecosystem is IMO a good way to look at it; libraries that are all independent of one another who have a (usually pretty good) adherence to semver.
Yes; also they should all be treated as separate, independent products. Where multi-repos go wrong is when a whole product spans multiple repos, and a feature of the product depends on features being added or altered in multiple projects.
Ideally, the split should be so that nobody has to work across multiple repos to finish a feature.
The github / NPM ecosystem is IMO a good way to look at it; libraries that are all independent of one another who have a (usually pretty good) adherence to semver.
> Ideally, the split should be so that nobody has to work across multiple repos to finish a feature.
That does not sound realistic. Often I have to add a new function, and I think: where (in what library) does this function belong, and very often it's not the main library I'm working in at that moment.
That does not sound realistic. Often I have to add a new function, and I think: where (in what library) does this function belong, and very often it's not the main library I'm working in at that moment.
Adding a new function anywhere other than the leaf project where it's used is premature. Organize around user-facing functionality. Pull out common libraries as and when you have something common enough to factor out (and, crucially, test independently), not before.
So the answer to needing to open multiple PRs is to open multiple PRs.
The answer is to adapt your working patterns so that you never have multiple PRs on the critical path. Long-term codebase quality may require multiple PRs, but delivering user-facing functionality doesn't.
Usually libraries are owned by separate teams.
Sounds good in theory, but I'm wondering: your edit-test cycle now becomes edit-install-test?
Depends on the environment you're working in, but generally: interpreted languages typically have no problem loading a package from a temporary dev-location, compiled languages have build systems that handle that.
Not necessarily. With Java IDEs, if you have the sources you can usually modify those and the IDE knows to ignore the package manager dependencies.
To the downvoter: https://stackoverflow.com/questions/10688344/in-eclipse-m2e-...
Surely, on the QA server as CI/CD pipeline.
conda is not bad as a generic package manager for this sort of stuff (packages don't have to be Python although that's what it was initially aimed at).
Zuul (https://zuul-ci.org/) is an open source CI/CD system that does speculative execution of CI on changes across multiple repos. So if you have a change in one repo that depends on a change in another, you can test both together before merging either (and it enforces merging in the correct order).
IME it solves the same problem that monorepos are intended to solve, but without any of the downsides.
IME it solves the same problem that monorepos are intended to solve, but without any of the downsides.
What's more, same result can be achieved with practically every modern CI server with minor admin effort.
We wrote one in house, it works well and has useful features that nothing else has. Today I'd use conan.io, and add those useful features to that project. Our internal system works well so it is hard to find justification to tear it out, but the downside is no tool/IDE understands it, and so you can't use your favorite IDE unless your favorite IDE is the one obsolete version that we have modified to understand our tool.
The thing I don't see discussed often enough, in all of the "monorepos are good/bad" debates, is that they require a certain discipline. When one commit can span multiple components, it's easy to add or ignore bad coupling between those components. That contributes to the rebuild treadmill for everyone, and is bad design in a bunch of other ways as well. IDEs make it worse, because they obscure the boundaries between components, but everyone ends up using an IDE because their predecessors made it very difficult to follow the flow of control across components otherwise. So you end up with one big ball of mud. With separate repos this is discouraged, because violating the rules of good design causes immediate pain. With a monorepo the pain must come in the form of stronger reviews, to discourage bad coupling even though it's easy. Unfortunately, the very same companies that have embraced monorepos have also abandoned separation of concerns as a design principle. :(
ETA: please read bunderbunder's comment (https://news.ycombinator.com/item?id=19796960) as well. It's an insightful take on some of the same issues.