Re #3: I'm confident that they won't offer that OotB, but they'll very likely have SNS integration, which could be translated into a Slack or HipChat message via Lambda. Depends on how much work you want to put in.
The Github PR integration is the most important part for me. It's a necessity for us to be able to use it. We use our own install of Drone on a fairly sizable EC2 instance and not having to manage that anymore would be wonderful.
How many different CI workflow tools are on AWS these days? How do I know which one to pick? I really wish Amazon would spend some time building comparison guides for their services. Each one feels very silo'd off from each other and the crossover in functionality seems very high.
Cloudformation is for when you have a team of ops or a workflow where you have a lot of repeated resource recreation; Beanstalk is for when you have no ops, and you're happy for someone else to handle the environment; CodeDeploy is just yuck; and don't know about this one yet.
Cloudformation: infrastructure-as-code (but has sharp edges) (doesn't touch your app/code directly)
Opsworks: wizard-style 'drop your app here' kind of thing (less flexibility and control)
Beanstalk: a simpler version of OpsWorks? (never tried it)
CodeDeploy: install an agent, it pulls code/artifacts (janky workflow)
CodeBuild: no idea, just been released
Just Using The Web Console: convenient, but manual process (labour-intensive, prone to manual errors)
On the CI thing - from my experience at one of the places I work, there are a thousand CI systems out there, but very few CD systems. Pretty much anything can schedule and track builds, but few things schedule and track deploys (which gets suprisingly tricky suprisingly quickly). CD is the 'last mile'...
EDIT: missed some that I've never looked at. It is getting crazy...
CodeCommit: looks like it might be a 'github'?
CodePipeline: No idea. Perhaps a spruced-up version of CodeDeploy?
This was my thought as well. I've seen them offer a few different of these "workflow engines" that seemed to be quite clearly oriented towards build services, although this is the first that's very explicitly focused on that.
I'm excited about what this means for Lambda. Building Lambda packages has always been a pain because much of the time it needs to be built on Amazon Linux to work, meaning most (all?) CI SaaS solutions are out the window. I assume CodeBuild will run Amazon Linux and CodePipeline integrates with Lambda, so this should make setting up continuous deployment with Lambda much easier.
When I saw this I got super excited because I thought it was going to be "per-minute low-latency distcc hosts" which was probably set up by having a ton of common toolchains ready to go at all times, and my big questions were "did they bother to support MinGW (easy)" and "do they have a reasonable story for iOS (though I can't imagine they have armv6 working right)", but while they got my hopes up when they said one could even upload custom toolchains I realized that didn't make much sense and a few paragraphs later I got the disappointing news that this is just some extremely thin wrapper over ECS that is limited to doing builds on single computers which max out at 8 vCPU (why?!). I mean, I guess this makes CI slightly more accessible to some people, but it isn't anything terribly exciting and is mostly going to help people with extremely small projects: this isn't going to scale up to the kinds of builds where you'd expect a service billing itself on scalability to be most valuable.
...Except for this part: CloudBuild has per-minute billing!! This is one of the major complaints people have about EC2 (and all the services Amazon builds over it), and is one of the major downsides of using it over Google's Compute Engine. If you have any kind of task that can possibly be thought of as a "build"--one which can be expressed as a container of software configured to access some external asset as input and which generates a concrete output "artifact" (and maybe even not, right? to support some silly things people do in their builds like "check out code from npm", you likely get network access, and your build output could always be an empty file)--this now seems like a depressingly hilarious way to trick Amazon's infrastructure into giving you per-minute billing for random tasks which take less than 20 minutes to run (important limit, as they are charging a 3x overhead vs the on demand price for an equivalent instance: for 8 vCPU / 15 GB instance, a c4.2xlarge costs $0.419 per hour and a build.general1.large costs $.02 per minute, which would be $1.20 per hour).
In other words: I will argue that this service really can and maybe should just be looked at as a different pricing model for ECS, to support any "small" task (not just building code): if it takes less than 20 minutes and doesn't require a massive computer, CodeBuild is not only cheaper but probably easier to use (as it already models the problem in terms of a task queue, so you don't have to do that part either).
As someone who has had to shop around and try out a bunch of continuous integration services (Travis, CircleCI, Snap, Solano, to name a few), this looks pretty interesting! We've stayed away from managing our own CI infrastructure but this could be a good (and cheaper) solution. It doesn't seem to actually be available yet, but it'll be worth a look.
As it sounds like it's running inside Docker itself, I'd like to know if this supports "Docker in Docker". My requirement is not strictly DinD however I run multiple containers during CI (Postgres, node, test containers, etc). Possible via different approaches in CircleCI, Shippable and SemaphoreCI. I don't actually build any containers to save.
This is actually how Codeship's Docker infrastructure works by default, builds up containers and executes all commands in them natively. DinD still possibly but somewhat yet useful with that approach.
What's frustrating is if you go to the aws reinvent page they tell you a lot except the dates of the conference!!! https://reinvent.awsevents.com/ I was wondering why all these aws announcements were here.
At any time someone can show up to eat your lunch but if you're developing software for operations or developers it seems like a scary space to be in. AWS has a huge advantage by being able to simply add a service to their existing catalog.
Gave it a try, seems to be kinda slow building docker images. An image that take 2 minutes in my machine is taking around 10 minutes in CodeBuild.
The build seems to freeze a little between docker build steps :
This sounds a bit like BuildKite which has worked out pretty well for us! They provide an elastic AWS CI environment that you run in your own AWS account, and scales up/down as builds are queued up.
I really wish AWS CodeBuild supported .net\C# as a preconfigured environment. Hopefully .net is not too far down on their priority list...right now we are using Jenkins.
> 2. We need be able to access resources inside a VPC.
From my initial testing, this looks like an annoying "no". There doesn't appear to be any way to set the VPC or security group in which the build executes, only an IAM role.
(which already makes it a non-starter for my use case with a private npm-enterprise server)
FAQ says this. Is there any examples on setting it up? It isn't obvious from inside CodePipeline.
Yes. The CodePipeline Plugin for Jenkins can be used to integrate CodeBuild into Jenkins jobs. The build jobs are sent to CodeBuild, eliminating the need for provisioning and managing the Jenkins worker nodes.
Using CodePipelines and CodeCommit you can create a workflow where a git commit to a CodeCommit repo can get picked up by pipelines and sent to the build service (i.e. CodeBuild or Jenkins). Then CodeBuild will push the resulting artifact to S3. CodeDeploy (and Elastic Beanstalk, CloudFormation and OpsWorks) can be configured to deploy the built artifacts to your application fleet.
It's the last piece in AWS's solution for continuous deployment.
[+] [-] St-Clock|9 years ago|reply
1. Caching. CircleCI and Travis cache intermediate build artifacts (e.g., virtualenv in python) to reduce build time.
2. Github pull request integration (red cross on pull requests if the build fails).
3. Chat integration. Sending a message to slack or hipchat when the build fails.
4. SSH into build container. Very handy for rare but difficult to locally reproduce build bugs.
Interesting offer though. We found that we would pay less than 5$ a month for our build needs and they would run concurrently.
[+] [-] St-Clock|9 years ago|reply
1. No cache. You need to build your own.
2. No obvious github integration. You can pull code from github but that's it.
3. Notifications are managed with SNS. So you need to use lambda or something like that for Slack Notifications.
4. No obvious ssh support. Would need to check the dockers image they use to be 100% sure.
[+] [-] luhn|9 years ago|reply
[+] [-] donatj|9 years ago|reply
[+] [-] debaserab2|9 years ago|reply
[+] [-] vacri|9 years ago|reply
EDIT: missed some that I've never looked at. It is getting crazy...
[+] [-] venantius|9 years ago|reply
[+] [-] luhn|9 years ago|reply
[+] [-] timoth|9 years ago|reply
[+] [-] saurik|9 years ago|reply
...Except for this part: CloudBuild has per-minute billing!! This is one of the major complaints people have about EC2 (and all the services Amazon builds over it), and is one of the major downsides of using it over Google's Compute Engine. If you have any kind of task that can possibly be thought of as a "build"--one which can be expressed as a container of software configured to access some external asset as input and which generates a concrete output "artifact" (and maybe even not, right? to support some silly things people do in their builds like "check out code from npm", you likely get network access, and your build output could always be an empty file)--this now seems like a depressingly hilarious way to trick Amazon's infrastructure into giving you per-minute billing for random tasks which take less than 20 minutes to run (important limit, as they are charging a 3x overhead vs the on demand price for an equivalent instance: for 8 vCPU / 15 GB instance, a c4.2xlarge costs $0.419 per hour and a build.general1.large costs $.02 per minute, which would be $1.20 per hour).
In other words: I will argue that this service really can and maybe should just be looked at as a different pricing model for ECS, to support any "small" task (not just building code): if it takes less than 20 minutes and doesn't require a massive computer, CodeBuild is not only cheaper but probably easier to use (as it already models the problem in terms of a task queue, so you don't have to do that part either).
[+] [-] nzoschke|9 years ago|reply
I wrote up how we plan to use this in the Convox platform here: https://convox.com/blog/codebuild/
Practically speaking, we're working through PCI compliance. Getting builds off of production services and root-enabled docker daemons is a huge win.
[+] [-] AtticusTheGreat|9 years ago|reply
[+] [-] redgc|9 years ago|reply
[+] [-] ethangj|9 years ago|reply
[+] [-] msie|9 years ago|reply
[+] [-] brazzledazzle|9 years ago|reply
[+] [-] brazzledazzle|9 years ago|reply
[+] [-] TheRealWatson|9 years ago|reply
Kind of like a lot of Microsoft tools, they are all lacking but, hey, the have deep integration with each other, so easier gluing.
[+] [-] bloomark|9 years ago|reply
[+] [-] AtticusTheGreat|9 years ago|reply
[+] [-] Malkav|9 years ago|reply
--- Example 1 ---
[Container] 2016/12/01 22:39:59 Step 9 : EXPOSE 3000
[Container] 2016/12/01 22:40:31 ---> Running in 1c6e3a4dbec8
[Container] 2016/12/01 22:40:45 ---> 602aa4bc97ac
-----------------
--- Example 2 ---
[Container] 2016/12/01 22:36:00 Step 4 : WORKDIR /src
[Container] 2016/12/01 22:36:32 ---> Running in 98800352e6c2
[Container] 2016/12/01 22:36:45 ---> b437afe2a1c5
-----------------
Not sure if this caused by the fact its a docker inside docker implementation.
[+] [-] davecap1|9 years ago|reply
[+] [-] ianceicys|9 years ago|reply
[+] [-] gtsteve|9 years ago|reply
[+] [-] STRML|9 years ago|reply
1. We should be able to configure this for a few/all branches (including PRs) and have conditional build tasks based on branch.
2. We need be able to access resources inside a VPC.
3. Turnkey chat integrations would be nice, but it's not a big deal to just curl.
4. We need a way to execute actions on failure.
[+] [-] mbailey|9 years ago|reply
From my initial testing, this looks like an annoying "no". There doesn't appear to be any way to set the VPC or security group in which the build executes, only an IAM role.
(which already makes it a non-starter for my use case with a private npm-enterprise server)
[+] [-] bueoko|9 years ago|reply
CodePipeline has codebuild and lambda integrations, you could use the lambda functions to do what you want after success/failure.
[+] [-] gsharma|9 years ago|reply
I am curious to know what kind of actions would they be other than notifying via chat/email?
[+] [-] mesozoic|9 years ago|reply
Yes. The CodePipeline Plugin for Jenkins can be used to integrate CodeBuild into Jenkins jobs. The build jobs are sent to CodeBuild, eliminating the need for provisioning and managing the Jenkins worker nodes.
[+] [-] unknown|9 years ago|reply
[deleted]
[+] [-] btashton|9 years ago|reply
[+] [-] kt9|9 years ago|reply
disclaimer: I'm the founder at distelli
[+] [-] viktorbenei|9 years ago|reply
[+] [-] gtsteve|9 years ago|reply
[+] [-] crb002|9 years ago|reply
[+] [-] callumjones|9 years ago|reply
[+] [-] bkendzior|9 years ago|reply
[+] [-] sidcool|9 years ago|reply
[+] [-] Timmy_C|9 years ago|reply
Using CodePipelines and CodeCommit you can create a workflow where a git commit to a CodeCommit repo can get picked up by pipelines and sent to the build service (i.e. CodeBuild or Jenkins). Then CodeBuild will push the resulting artifact to S3. CodeDeploy (and Elastic Beanstalk, CloudFormation and OpsWorks) can be configured to deploy the built artifacts to your application fleet.
It's the last piece in AWS's solution for continuous deployment.
[+] [-] moritzplassnig|9 years ago|reply