I'm really excited to finally open-source Chromeless. We've used NightmareJS and similar tools before to run integration tests but these basically added ~20min to each build. With Chromeless we were able to reduce this time to under a minute!
How did you manage to get Chrome headless compiled and running on AWS?
I spent a while looking at this a few months ago to do a similar project (run chrome in headless mode on AWS Lambda) but couldn't get the binary to work.
How difficult would it be to also support google cloud functions and the azure offerings? This seems like a really useful standard tool that lots of people might want to use. CI jobs on pull requests that take seconds instead of minutes = big win!
shamless plug: i've also written a high level api on top of the chrome remote debugger chrominator [1]
similar idea. chrominator use promises instead of a fluent api. it also follows the selenium w3c spec where possible. it does cool stuff with evaluate and evaluateAsync where it resolves the remote object to something usable.
to be fair there are a few other projects i know about that wrap chrome remote debugger with a high level api:
Shameless plug: I've been hacking on headless chrome in AWS Lambda but with selenium webdriver support [1], also using the binaries from the serverless-chrome [2] project.
To echo another comment on this thread, headless chrome seems well-positioned to shake up the automated browser testing market. The price—especially with the AWS Lambda free tier—is very, very compelling for a number of projects.
[1] Chrominator looks interesting. jesg/gowan is the maintainer for PhantomJS GhostDriver, so the API looks extensive. Another project that I heard recently is Navalia [4], which uses GraphQL type of queries to run headless Chrome. Also worth noting is Chromy [5] by a Japanese developer; it's one of the earliest to market (released in early May). BackstopJS maintainer also looks to be in active development using Chromy.
This looks great! As the developer of an automated screenshot solution (https://urlbox.io), one of the major pain points when taking screenshots is font-rendering. I wonder how you could install/configure fonts on lambda?
This would have been awesome to have back when I was heavy in to the UI test automation game. Our best option at that point was a spot instance EC2 fleet and analyzing commits to determine which tests would be the most valuable to run. It's awesome being able to easily run hundreds or thousands of tests in parallel, completely segmented, and pay only on demand. A fantastic use of AWS Lambda! It suddenly becomes reasonable to do full integration tests on every merge request, or even commit, and get feedback to the developer in seconds.
Do the math. If you're suggesting running all the tests, drastically increasing your compute volume, it's probably cheaper to use a dedicated instance running at max CPU utilization.
This is a really cool project, but looking closer at the API and issues raised it seems that the features are being over-promised.
- "Do pretty much everything you've used PhantomJS, NightmareJS or Selenium for before".
The main features of those tools plus their ability to handle a large range of edge cases are built up over the years in production use and do not seem to be already in Chromeless. Also, Lambda costs can be a significant point of consideration for professional test automation with large volume.
Nevertheless, there's no turning back as flood gates have been opened and many developers are noticing Chromeless. I believe, with enough dedication from Chromeless maintainers, they may be able to channel the attention and contributions to shape Chromeless to be the main challenger to existing test automation approaches. That will really be a blessing to the open-source community!
The only catch I believe, is it may be easier for those existing tools to be made working in Lambda or implement a similar form of parallelism while still having their mature API, than for Chromeless to catch up to the state of maturity of those tools. But as they say, growth solves almost every problem, so issues like these may be ironed out through collaborative efforts from contributors/maintainers.
I totally agree with you! It took years for these tools to mature and so it will be the case for Chromeless. There are probably a range of edge-cases that yet have to be solved but like you said, I'm very optimistic that together with our great community, we'll be able to handle all of these cases.
The big incentive for us to create Chromeless instead of using Nightmare or similar (which I've done for years) was the fact that you can now use headless Chrome (which provides a way more stable foundation) + the ability to execute the code on AWS Lambda which solves the parallelisation question. I hope this makes sense to you :)
I assume the number one use of this would be test automation for one's own sites so blocking would not be an issue.
What are sites' motivations for blocking AWS IPs? I bet there are some reasons I would agree with even though the somewhat crude method of blocking ip range would have some unintended consequences (e.g. blocking people running a personal VPN).
Indeed, so many of AWS’s IP ranges are used for DDoS and malicious behaviour they end up getting blacklisted due to their poor reputation. It’s a bit like using one third party resellers shared IP ranges in your mail relay - you’re asking to end up on reputation based lists. There’s a lot to be said for your IP reputation on the internet and when you outsource that - you outsource your freedom to maintain your reputation risk.
How is this different from using a container/vm image which has chrome pre-installed and on request launch it in headless mode, accessing the instance via chrome-launcher and manipulating the browser with chrome-remote-interface?
You can then use the vm/container as a function to match AWS lambda.
Is it the that the api is more-user friendly or selenium w3c complaint?
Genuinely curious, don't know much about this project.
I've been super excited about Chrome headless but haven't had a chance to dig into using it yet. The api here looks amazing for getting started without getting lost in the weeds. It'd be fairly trivial to hook this up to a Slackbot and to get on-demand screenshots of various pages on my websites, etc.
A month ago I was trying to migrate some tests from phantomjs to chrome headless. The big blockers I ran into were that chromedriver automation has certain functionality that only works with a special chrome extension that it installs- but headless chrome doesn't support extensions.
There's work being done to fix this, but it's still in progress, last I checked. Until then, resizing windows, taking screenshots and a handful of other things simply don't work.
Other than taking a screenshot and evaluating JS code in the context of Chrome and returning JSON, we haven't yet implemented any file-download features. But it might be possible for us to implement something. Would you mind creating an issue here describing your use case so we can discuss it further? https://github.com/graphcool/chromeless/issues/new
Looks like Google releasing headless Chrome is really shaking up the test automation domain. This is a really cool project :) The other day I saw another interesting and refreshing implementation using GraphQL - https://github.com/joelgriffith/navalia (not affiliated with the project, just got to know from a PhantomJS issue).
(Shameless promoting): for an API version of the prerendering functionality, with no warm-up latency, we're running a large cluster of Chrome headless instances here: https://www.prerender.cloud/
Sounds like a great service and very useful for frontend developers.
We've built something similar internally and will shortly migrate it to Chromeless. We basically use it to pre-render our websites and docs: https://github.com/graphcool/prep
We've been using chrome-remote-interface for test automation in a project that makes heavy use of Lambda for a distributed event processing infrastructure. I'm looking forward to seeing whether we can implement this for running our test automation suite!
Would love to hear how that goes. Please reach out if you have any problems or questions. The easiest way is to ping me on Slack: https://slack.graph.cool
Yes exactly + this can run (in parallel) on AWS Lambda, so you don't need to worry about provisioning & running servers. That's actually the part I'm most excited about :)
[+] [-] schickling|8 years ago|reply
Here is btw a demo playground to try it out: https://chromeless.netlify.com/
Let me know if you have any questions :)
[+] [-] djhworld|8 years ago|reply
I spent a while looking at this a few months ago to do a similar project (run chrome in headless mode on AWS Lambda) but couldn't get the binary to work.
[+] [-] ivansavz|8 years ago|reply
[+] [-] gowan|8 years ago|reply
similar idea. chrominator use promises instead of a fluent api. it also follows the selenium w3c spec where possible. it does cool stuff with evaluate and evaluateAsync where it resolves the remote object to something usable.
to be fair there are a few other projects i know about that wrap chrome remote debugger with a high level api:
* autogcd [2]
* ghostjs [3]
[1] https://github.com/jesg/chrominator
[2] https://github.com/wirepair/autogcd
[3] https://github.com/KevinGrandon/ghostjs
[+] [-] smithclay|8 years ago|reply
To echo another comment on this thread, headless chrome seems well-positioned to shake up the automated browser testing market. The price—especially with the AWS Lambda free tier—is very, very compelling for a number of projects.
[1] https://github.com/smithclay/lambdium [2] https://github.com/adieuadieu/serverless-chrome
[+] [-] kensoh|8 years ago|reply
[4] https://github.com/joelgriffith/navalia
[5] https://github.com/OnetapInc/chromy
[+] [-] kensoh|8 years ago|reply
https://medium.com/@kensoh/chromeless-chrominator-chromy-nav...
[+] [-] cjr|8 years ago|reply
[+] [-] titel|8 years ago|reply
[+] [-] schickling|8 years ago|reply
Please feel more than welcome to take a stab at it yourself and create a PR for this feature! :)
[+] [-] pkilgore|8 years ago|reply
Btw, does the .viewport() option not work in the demo? I'm seeing a `TypeError: Failed to fetch` when I set one.
[+] [-] 33degrees|8 years ago|reply
[+] [-] shortj|8 years ago|reply
[+] [-] toomuchtodo|8 years ago|reply
[+] [-] schickling|8 years ago|reply
[+] [-] kensoh|8 years ago|reply
- "Do pretty much everything you've used PhantomJS, NightmareJS or Selenium for before".
The main features of those tools plus their ability to handle a large range of edge cases are built up over the years in production use and do not seem to be already in Chromeless. Also, Lambda costs can be a significant point of consideration for professional test automation with large volume.
Nevertheless, there's no turning back as flood gates have been opened and many developers are noticing Chromeless. I believe, with enough dedication from Chromeless maintainers, they may be able to channel the attention and contributions to shape Chromeless to be the main challenger to existing test automation approaches. That will really be a blessing to the open-source community!
The only catch I believe, is it may be easier for those existing tools to be made working in Lambda or implement a similar form of parallelism while still having their mature API, than for Chromeless to catch up to the state of maturity of those tools. But as they say, growth solves almost every problem, so issues like these may be ironed out through collaborative efforts from contributors/maintainers.
[+] [-] schickling|8 years ago|reply
I totally agree with you! It took years for these tools to mature and so it will be the case for Chromeless. There are probably a range of edge-cases that yet have to be solved but like you said, I'm very optimistic that together with our great community, we'll be able to handle all of these cases.
The big incentive for us to create Chromeless instead of using Nightmare or similar (which I've done for years) was the fact that you can now use headless Chrome (which provides a way more stable foundation) + the ability to execute the code on AWS Lambda which solves the parallelisation question. I hope this makes sense to you :)
[+] [-] yeldarb|8 years ago|reply
https://github.com/adieuadieu/serverless-chrome
[+] [-] schickling|8 years ago|reply
[+] [-] afandian|8 years ago|reply
I'm doing something similar, and this concern was one motivation for running in our datacentre vs EC2.
Does anyone have concrete info on rates of bots blocked from AWS IPs?
[+] [-] extra88|8 years ago|reply
What are sites' motivations for blocking AWS IPs? I bet there are some reasons I would agree with even though the somewhat crude method of blocking ip range would have some unintended consequences (e.g. blocking people running a personal VPN).
[+] [-] mrmondo|8 years ago|reply
[+] [-] ingenuous2|8 years ago|reply
[+] [-] bdcravens|8 years ago|reply
[+] [-] 1024core|8 years ago|reply
[+] [-] pedrocls|8 years ago|reply
You can then use the vm/container as a function to match AWS lambda.
Is it the that the api is more-user friendly or selenium w3c complaint?
Genuinely curious, don't know much about this project.
[+] [-] biscarch|8 years ago|reply
[+] [-] mabbo|8 years ago|reply
There's work being done to fix this, but it's still in progress, last I checked. Until then, resizing windows, taking screenshots and a handful of other things simply don't work.
[+] [-] coredog64|8 years ago|reply
[+] [-] adieuadieu|8 years ago|reply
[+] [-] inertial|8 years ago|reply
The promise of fast execution time in parallel is tempting with Chromeless. Thanks for sharing.
- https://github.com/webdriverio/webdriverio
- https://github.com/nightwatchjs/nightwatch
- https://github.com/assaf/zombie
- https://github.com/dhamaniasad/HeadlessBrowsers
[+] [-] schickling|8 years ago|reply
Ultimately it was the combination of using headless Chrome and the ability to execute code in parallel on Lambda, which made us invest in Chromeless.
[+] [-] shimon_e|8 years ago|reply
The error report given by this are some of the best.
[+] [-] Vuneu|8 years ago|reply
[+] [-] kensoh|8 years ago|reply
[+] [-] yamafaktory|8 years ago|reply
[+] [-] jotto|8 years ago|reply
[+] [-] schickling|8 years ago|reply
We've built something similar internally and will shortly migrate it to Chromeless. We basically use it to pre-render our websites and docs: https://github.com/graphcool/prep
[+] [-] languagehacker|8 years ago|reply
[+] [-] schickling|8 years ago|reply
[+] [-] iokevins|8 years ago|reply
The first two examples seem to return 404, for me:
https://github.com/graphcool/chromeless#examples
[+] [-] schickling|8 years ago|reply
[+] [-] victorhooi|8 years ago|reply
Seems pretty awesome.
My use case is to take screenshots of various pages - the docs don't mention the default viewport dimensions, btw.
[+] [-] schickling|8 years ago|reply
[+] [-] roughcoder|8 years ago|reply