There's more details 2 messages later on what the cost is for:
> Some clarification I got from Daniel in a private conversation, since I was confused about what the money was paying for exactly:
> We're paying 75K USD for the bandwidth to transfer data from the GitLab cloud instance. i.e., for viewing the https site, for cloning/updating git repos, and for downloading CI artifacts/images to the testing machines (AFAIU).
gitlab.freedesktop.org is hosted in GCE us-east1. One runner for CI jobs is also in us-east1, but there are also (donated) runners at Hetzner and Packet, and a lot of bandwidth goes toward communicating with those.
Apparently even if they disabled CI, it would still be $30K/year, I guess for hosting and web/git traffic. Not a good situation.
This is laughable. Just renting another box at Hetzner with the specs necessary for their Gitlab would be around $100-$150 per month. That is maybe 1-2k per year. With 10tb/month traffic included already, IIRC, additional at a very reasonable price, and traffic to their already existing Hetzner runner box being non-metered to begin with.
> All the other services are run on a small fleet of machines hosted at Portland State University.
Would the university host the GitLab instance? (Or a VM for it, or whatever.)
The bandwidth cost to them should be very small. I run infrastructure hosted at a university, and they change around $500/year for a 10Gb/s uplink. I think this is just the cost price for them.
Doing like Netflix did by paying AWS millions is wasteful and insane when managed colo is cheaper and self-managed colo is cheaper still. ByteMark.co.uk and pair.com are both great vendors that could save them money.
Colocation would shift a good portion of the annual expense to an up-front investment in hardware (as well as setup, shipping, install fees). It would also increase the required admin time.
It's quite possible that they could save some money by going this way, but it's by no means going to just fix their problem. In a way it would exacerbate it due to the capex for hardware.
What we do at my company: everything is in the cloud except build farm because renting dedicated servers is much cheaper, we need lots of them and we don't care if one drops out. So we have 32c/256G boxes for running tests (a full suite finishes in 2 hrs on a box like that), they cost something like $300/mo.
> With reasonable estimates for continued
growth we're expecting hosting expenses totalling 75k USD this year,
How does that happen? My understanding here is the servers are doing something ephemeral and so... uh, why this isn't on Hetzner SB? I do not know of course whom they are using for hosting but 35 USD gets you a decent quad core 32GB RAM server there which is very hard to beat.
Would switching from GitLab to something like Gitea/Gogs cut down on those costs? The GitLab system requirements start at multiple gigabytes of RAM and CPU cores for almost idle installations whereas a small Gitea I'm running takes <100MB RAM and next to zero CPU. Most people I know that self-host (i.e. privately, not commercially) a git server with issue tracker don't run GitLab due to the resource hogging. But I don't know how it scales when you put more load on Gitea vs the same load on GitLab -- does anyone have experience with that?
It says in the message it's storing and serving build artifacts that is driving costs. This means the expense is in disk and bandwidth, not CPU and RAM. Switching to Gogs would cut down on the resources which are not identified as the problem.
[+] [-] mrpippy|6 years ago|reply
> Some clarification I got from Daniel in a private conversation, since I was confused about what the money was paying for exactly:
> We're paying 75K USD for the bandwidth to transfer data from the GitLab cloud instance. i.e., for viewing the https site, for cloning/updating git repos, and for downloading CI artifacts/images to the testing machines (AFAIU).
And more details on their infrastructure in general is at https://www.freedesktop.org/wiki/Infrastructure/
gitlab.freedesktop.org is hosted in GCE us-east1. One runner for CI jobs is also in us-east1, but there are also (donated) runners at Hetzner and Packet, and a lot of bandwidth goes toward communicating with those. Apparently even if they disabled CI, it would still be $30K/year, I guess for hosting and web/git traffic. Not a good situation.
[+] [-] Slartie|6 years ago|reply
But woooh, that's probably not cloudey enough.
[+] [-] Symbiote|6 years ago|reply
Would the university host the GitLab instance? (Or a VM for it, or whatever.)
The bandwidth cost to them should be very small. I run infrastructure hosted at a university, and they change around $500/year for a 10Gb/s uplink. I think this is just the cost price for them.
[+] [-] anonsivalley652|6 years ago|reply
Doing like Netflix did by paying AWS millions is wasteful and insane when managed colo is cheaper and self-managed colo is cheaper still. ByteMark.co.uk and pair.com are both great vendors that could save them money.
[+] [-] jcrawfordor|6 years ago|reply
It's quite possible that they could save some money by going this way, but it's by no means going to just fix their problem. In a way it would exacerbate it due to the capex for hardware.
[+] [-] sz4kerto|6 years ago|reply
[+] [-] tixe|6 years ago|reply
[+] [-] stonogo|6 years ago|reply
[+] [-] astrodust|6 years ago|reply
[+] [-] CameronNemo|6 years ago|reply
[+] [-] chx|6 years ago|reply
How does that happen? My understanding here is the servers are doing something ephemeral and so... uh, why this isn't on Hetzner SB? I do not know of course whom they are using for hosting but 35 USD gets you a decent quad core 32GB RAM server there which is very hard to beat.
[+] [-] earenndil|6 years ago|reply
[+] [-] lucb1e|6 years ago|reply
[+] [-] stonogo|6 years ago|reply
[+] [-] earenndil|6 years ago|reply
[+] [-] drenginian|6 years ago|reply
[+] [-] unknown|6 years ago|reply
[deleted]