[MakersF]

Like some people, I was expecting to find a way to have the advantages of a monorepo while having projects in separate repos. This is something Bloomberg is doing, and it's very cool. Each project is a separate repo, but they have a central integrated "repo" with all the repos, which is the "source of truth", and were code is built and deployed from. You can commit changes in your repo, and then you "release" the code into the integrated repo, which will rebuild all the transitive dependencies and run their tests to make sure everything still works. If anything fails, your release of the code is not merged in the repo. I'm now working with a monorepo, and I much prefer the Bloomberg approach. Cross repo changes can be made atomically (you update the reference in the integration repo for multiple individual repositories at once), and that is usually the big sell point of monorepo. And it doesn't have the downsides of the monorepo. The only issue is that it's not very ergonomic, and there isn't a tool to make that easy. But building such a tool is definitely easier than implementing a virtual FS as it has been done in multiple companies.

I'd love if someone still working there were to write a nice post about that system, it was the first of such a kind I saw.

[rwarmka]

> Each project is a separate repo, but they have a central integrated "repo" with all the repos, which is the "source of truth", and were code is built and deployed from.

That's exactly one of the things josh can already do for you :) Josh's concept of workspaces is precisely this: define your dependencies (no matter where else they originate from in the monorepo) and then check out only those dependencies, along with any code that solely exists in the workspace. Your workspace checkout is effectively the "bloomberg"-style repo setup you described, as you only see your code and the code of your dependencies, but when you push, your changes get added back to the hidden, backing monorepo (the source of truth) where all related and pertinent tests are run, and your change can only be committed if those tests all pass.

Thus, your commit is your "release". Sure, you don't have exactly the same workflow, as there's no difference then between a commit and a release, as by your definition you don't release after every single commit, but the whole "release this change to everything else in the monorepo" is touted as one of the benefits: there's no massive integration headache if you have multiple breaking changes which you then need to work on resolving for everyone else.

source: I work directly with "chrschilling" - who wrote josh

[choeger]

But that only works in one direction, no? So it works if you can develop your single repo, but it doesn't help you when you depend on other projects.

I think the best approach would be to have bidirectional links between the projects (if A needs B, then A has the stable version of B and vice versa). The point in that setup would be that "upstream" projects can notice when they are about to break tests in "downstream" repos and act accordingly.

[MakersF]

They do, the repositories define dependencies, so when you change something everything that depends on you is rebuilt and tested. This prevents breaking changes, both for your dependencies and your reverse dependencies, identically to a monorepo.

It's a bit complicated to explain, but it works. That's why I hope they'll make a blog post :)

[throwaway315724]

> I'd love if someone still working there were to write a nice post about that system, it was the first of such a kind I saw.

I don't know how far back you saw the Bloomberg system, but at this point it's basically the same as the Debian system (as in, debian/ subdirectories, .deb files, etc.). Versions of git projects are published as tarballs (source packages). Then sets of published projects are "promoted" and all projects that transitively depend on them are rebuilt and unit tested in a sandbox environment. If that process fails, the promotion fails.

Each source package can use any number of build systems, implementation languages, or project structures.

There's also a legacy subversion monorepo with a monolithic build system that builds on top of that, but it's slowly being phased out.

All that is an integration build including thousands of discrete projects. Those projects typically have additional CI/CD enrollments outside of the integration build system too.

[MakersF]

This is what I saw. But I think it would be good to share the experience of a big, mature company with it to show that it's a system that can work, and can be easily added to companies which currently have a multi repo approach.

Also, there are quite a few tools to manage the distributions, and it would be great if they were open sourced. Basically Bloomberg championing their approach, to gain the usual advantages of open source (developer familiarity, cooperating across companies for improvements, and so on)

> Those projects typically have additional CI/CD enrollments outside of the integration build system too.

Another thing to call out is that you can simulate the "promotion", so you can check in your PRs whether your change is going to break any dependency or dependant.