How to set up continuous deployment using free hosted tools

I can't believe the author doesn't mention gitlab. Setting up full CI on gitlab is amazing. They have the easily integratable gitlab runners i.e. travis-ci functionality + different build stages for automatic deployment via ansible/chef/whatever and unlimited free private projects to boot. If you're working on anything that can't be public I highly recommend gitlab. Its an amazing service.

FYI: I am in no way affiliated with Gitlab.

GitLab is great and has come a very long way over the last several years. However, I still gladly pay GitHub $7 a month for my private repos because I feel that GitHub truly does offer a better code review and repository experience. GitLab in certain places is still very rough around the edges, not to mention performance has always been a concern.

To me, GitHub still is the gold standard and will likely be for quite some time.

I think I saw that GitHub now supports non-gh_pages branches, but Gitlab pages are also much easier to setup.

I've moved all my sites to Gitlab pages generated via Hugo or Jekyll.

Uptime (via Uptime Robot) just doesn't seem as good as GitHub, but it's not bad.

I haven't tried it :) I'll definitely give it a go.

> can't believe the author doesn't mention gitlab.

I absolutely agree. We use Gitlab in our company for continuous deployment and it works like a charm.

Apart from the other tools I miss some performance monitoring. Sentry is great, but if your application suffers from bad performance after the latest deployment, you won't notice that soon enough.

A great self-hosted CI alternative is Drone.io (https://github.com/drone/drone).

To set it up is simply a matter of starting a Docker image and creating a .drone.yml file (very similar to .travis.yml).

Note: I'm not affiliated with the project in any way. I'm just a happy user who think the project deserves way more buzz.

Once you need something a bit more heavyweight than Travis (e.g. more CPU) I would highly suggest Buildkite (https://www.buildkite.com). It’s a “bring your own machines” solution, which means you can have beefy machines (e.g. via buildkite on EC2 spot autoscaling), or even machines that run inside production without handing buildkite itself any secrets. We’ve been using it at Ladder and it’s been phenomenal.

Also it has a lot of emojis.

Another very happy buildkite user here. It's extremely flexible and the crew just keep adding feature after feature.

+1 for mentioning Buildkite, also works great for machine learning

Heroku does have a free tier, but it's intended for experiments. It's very limited and wouldn't be acceptable for any serious website, certainly not a 'world class' setup. Unless you're very careful to have one free dyno per account, and avoid using features like staging apps, they will power down your server for several hours a day.

Yeah, the “world-class” hook was more about the quality of these tools than their applicability for use at scale for free - I encourage people to pay real money at various points in the article.

I’m currently running my blog on a single $7/month Heroku Dyno (plus $9/month for Heroku PostgreSQL) behind a free Cloudflare account - with the right caching headers I can easily handle huge amounts of read traffic. I was going to mention Cloudflare in this piece but I decided it didn’t fit the theme of continuous deployment as well as the other tools.

You probably should not use the free tier for any serious website or 'world class' setup...

> Once you’ve set up Travis, it will automatically test every commit to every branch

Small correction: It will automatically test the latest commit on every push to every branch

Thanks - I updated that sentence to say push instead of branch.

You can manually test all commits in push though. Or automatically run bisect on failure, etc.

I'm the developer advocate for Codeship.

Codeship (https://codeship.com/features) allows forever free builds for public repos. It has support for both Continuous Integration and Delivery/Deployment. Maybe you can give it a spin as well?

I can second codeship - even though its a bit more clunky than Travis and with less integrations and command-line tools, its has way beefier machines, easily several times faster than what travis offers even to paid customers. And their tools are improving. Keep up the good work.

We did not pursue Codeship due to lack of a "codeship.yml"-style configuration so that we could track config in source control.

>The integration of Travis with GitHub runs deep

At the moment there are 10+ companies that do the exact same thing.

http://blog.codepipes.com/hosted-ci-comparison/hosted-ci-com...

If you don't have time to click the link:

  - Buddy Works
  - CircleCI
  - Codefresh
  - Codeship
  - DeployBot
  - Distelli
  - SemaphoreCI
  - Shippable
  - SolanoLabs
  - Vexor
  - Wercker

https://buddy.works/ is great, fast builds, slick UI, and a free self-hosted enterprise version.

Only comment I would make is that you can use CircleCI instead of Travis to get for free running tests on private repos. The configuration is similar and the benefits as well.

Thanks for the tip - I just added a note about Circle CI to the article.

Doesn't show any automated verification of code in deployed fashion, no load tests or verifications which would be part of a world class pipeline. Also no canary or red-blue tree type rollout hooked to sentry. At least it has a staging but don't know what they're using it for other than to test migrations before promoting to prod. Nice little write up but far from world class.

Exactly - continuous deployment is about moving some/most testing to production, not eliminating it altogether.

Acceptance testing the system once deployed is the cornerstone of organisational acceptance of the CD doctrine - errors will always happen, it's just that they can now be fixed much faster in production.

Using an acceptance test suite as a deployment safety harness should dramatically increase the quality of tests, and hopefully the application too.

> Need more? Pay for it!

or be willing to move vendor and re-implement the stuff.

For comparison my magic combination(and not all free or realistic for everyone):

  - gitlab
  - jenkins/looking into gl ci
  - linode/cloudafrica(ah-hah hosting is not free)
  - sentry is a tough one. We rolling a custom mqttwarn/zabbix solution
  - irc(fine I will take your slack and use the irc-bridge pfftt)

> - jenkins/looking into gl ci

Yup, definitely look into it. We just killed our last Jenkins instance last week after we found out that everyone eventually moved to the GitLab CI.

> - sentry is a tough one. We rolling a custom mqttwarn/zabbix solution

Why a tough one? I found Sentry easy to self-host. They have pretty good docs: https://docs.sentry.io/server/installation/

> - irc(fine I will take your slack and use the irc-bridge pfftt)

We ditched Slack for Mattermost. It's not entirely up to par (yet), but works for most use-cases. And there's an IRC bridge for it, too (https://github.com/42wim/matterbridge). Oh, and Mattermost comes bundled with GitLab in their omnibus.

Heroku is awesome for this use case, but there are also great alternatives based on Cloud Foundry. Cloud Foundry is an open source PaaS with support for Heroku buildpacks + docker containers. At a median of ~5€/GB RAM per month they will also be a bit cheaper than Heroku.

Some providers that offer Cloud Foundry including a free tier for small Apps:

- Meshcloud (hosted in Germany, 2 Locations) https://panel.meshcloud.io/#/register (Disclaimer: I'm a founder there)

- Pivotal Web Services (hosted on AWS US): https://run.pivotal.io/

- IBM Bluemix https://www.ibm.com/cloud-computing/bluemix/ (various locations, a bit more expensive than the other two though)

Shameless plug: I wrote an open source incoming webhook server that can run your build & redeployment scripts: https://github.com/adnanh/webhook

Thank you very much for this!

We use this to listen for Github webhooks and execute jobs with Saltstack.

I have a similar setup, but using Docker images instead of Heroku buildpacks. Buildpacks are horrible if you want to do anything custom.

To make this work, I use Codeship instead of Travis. Codeship builds a Docker image, runs the tests against the image, then pushes the image to Heroku if the tests pass: https://documentation.codeship.com/pro/continuous-deployment...

You can also do every step listed, for free using a single platform: VSTS (Visual Studio Team Services).

Online, free for small teams, Git source control, CI/build/release, supports every major build stack, not just MS. It even has a top quality agile work tracker, that I've found better (or at least more balanced) than Jira/Trello equivs.

EDIT: link: https://www.visualstudio.com/team-services/

Not sure bitbucket pipelines are mentioned here. But for free I manage to commit Hugo source files to a private git repository, generate the static files by spinning up a docker container and generating them using Hugo and finally rsyncing the static output to my Webserver. All for free. You only get 50 minutes of time now (for free) in the docker instance but for simple continuous deployment cases like mine it’s very effective.

You might want to look at Gitlab if you want to drop the need for a web server. Their pages support supports Hugo much better than GitHub (which leans toward Jekyll), and is pretty well documented.

Add Netlify and it will continue to be free. I use Hugo as well, Netlify compiles Hugo on it's own, so I don't need workflows, but it is cool to have them.

The last time I tried using pipelines, it wouldn't let me use any arbitrary Docker image, I had to use one of the Atlassian "blessed" images. :/

Why is that better than generating static locally? Seems like a lot of fiddling for no benefit.

I just set up CI tests for some of my Python libraries on GitHub, and another tool to note is Appveyor -- very useful for running tests on Windows (which Travis doesn't do yet). Example: https://github.com/benhoyt/scandir/blob/master/appveyor.yml and https://ci.appveyor.com/project/benhoyt/scandir

I have started using visual studio team services, it is really good and it's free. https://www.visualstudio.com/team-services/

I used to use team city, octopus and jenkins in previous jobs. Personally, it is really easy to use and you don't have to use the microsoft stack. It supports Python, Ruby, node etc... It feels to me like they waited to see all the good bits people have done and then just implemented them.

I didn't want to manage a server so it is perfect for me.

GitHub and Travis are not self-hosted tools.

It would be nice to see a setup with something like Gitea[0] and/or Jenkins[1] edit and some other slack alternative like mattermost[2]

[0] https://try.gitea.io/

[1] https://jenkins.io/

[2] https://about.mattermost.com/download/

The title says 'hosted', which is the opposite of 'self-hosted'.

"Free hosted", not "self-hosted".

> Travis are not self-hosted tools.

Pretty sure you can self host travis.

A bit that's missing: bors-ng https://bors.tech to only merge PRs which passed testing.

Does anyone have a solution for a situation where I have a Django backend and a VueJS front end and want to run the Vue build step before deploying? Currently doing this with Git hooks and committing the resultant build, but that is bad for a number of reasons. This is running on Heroku. Would their release phase allow for this without significantly slowing things down? Would it deal with npm installs?

You might be able to solve this using a bin/post_compile script - most standard Heroku buildpacks will run these as part of the build step, and they can be used to run commands that generate files which will then be included in the compiled app that gets deployed to Heroku.

Here's an example that might be useful: https://github.com/nigma/heroku-django-cookbook/blob/master/...

I've done this for several apps. We have a large Jenkins 2 cluster and use declarative Jenkinsfile's. For Jenkins, you can accomplish this by simply setting up a Jenkinsfile that has an agent defined in each stage. Build your VueJS app in the first stage with a NodeJS container, and build the second stage with a Python container. Bonus points for building an actual Docker image as your artifact that has Django inside of it with Apache or nginx serving the VueJS bits too.

You do not need to commit your build to git if you are using heroku.

If you add the nodejs buildpack to your heroku project, it should detect and build your package.json automatically on heroku. Works with both yarn and npm.

You just need to make sure to set NODE_ENV=production in your heroku config.

https://devcenter.heroku.com/articles/nodejs-support#build-b...

We have this with GitLab CI, although with React and not Vue (but I don't think this matters). We have two stages, `build-assets` and `build-app`, first one runs Webpack etc, and the second one copies the build artifacts into a Docker image with Django and finalizes the image with collectstatic.

We're using Docker, but I'm sure the same approach can be used with simple tarballs, OS packages (.debs, etc) or deploy-specific git repo (for Heroku)

88 comments