Huxley, an app that helps fight CSS and UI regression

[taitems]

For anyone looking for a non-Python way of doing this, PhantomCSS will achieve nearly all of this. The main difference between the two is that Huxley seems to listen and re-perform actions, instead of needing them to be scripted.

We've had PhantomCSS running on a living style-guide/style tile on check in and can even fail the build on TeamCity. It's resemble.js under the hood to do the pixel-by-pixel comparison and will spit out a percentage difference, fail screenshots etc.

EDIT: Link added https://github.com/Huddle/PhantomCSS

[peterhunt]

Just to clarify, it's written in Python but you don't have to write any Python code to use it. Just a teeny config file that contains a test name and a URL.

[nirvdrum]

Nifty project. It'd be nice to add in automated analysis and make it a full Web Consistency Testing tool (disclaimer: a project I work on a lot): http://webconsistencytesting.com/

The three major problems with screenshot based tools are: 1) hard to adapt to acceptable levels of element shifting; 2) hard to extract semantic understanding of what broke; and 3) if it's not automated and you test a lot of pages in a lot of browsers, they simply won't get compared. But adapting this shouldn't be too hard.

[aphexairlines]

Why Resemble.js instead of http://pdiff.sourceforge.net/ ? Here's a presentation from a Google team using pdiff for front-end continuous deployment: http://www.youtube.com/watch?v=UMnZiTL0tUc

[bslatkin]

The project for that is here: https://github.com/bslatkin/dpxdt

[sync]

Looks quite awesome! It would be helpful to include some installation instructions for those of us unfamiliar with python libraries... do I use pip? do I use easy_install? do I git clone and then do something with setup.py?

[peterhunt]

I fixed this and actually put it on PyPI :) pip install huxley should do the trick.

[ds_]

Would anyone pay for this as a service? i.e. you enter a bunch of urls and screen sizes, then get sent nice emails with request stats, screenshots and pdiffs with easy integration into your deployment workflow.

[andreasklinger]

You could try to address "non-dev people" with this kind of tool.

"Alerts when your shop breaks" Basically CI but on the live system (with a testuser) and daily report emails and instant alerts.

[abritishguy]

I'd just integrate it into travis.

[hkmurakami]

Totally thought this would be named after Aldous Huxley, the author of Brave New World, but the answer was quite cool anyways:

So we named it after the guy who coined the term "agnostic", Thomas Henry Huxley.

[achalv]

Pretty awesome. I could definitely see myself using this.

[cburgmer]

Here is a project that gathers all available solutions around CSS testing: http://csste.st

[abritishguy]

Is there a way to test responsive designs with this? Can you set the device metrics?

[peterhunt]

Like window size? You can from the Python API, I just didn't include the option in the Huxleyfile. I've opened an issue here https://github.com/facebook/huxley/issues/2

Pixel ratio, I'm afraid, would be quite difficult.

[faramarz]

Is there a visual output? I'm not sure i understand this project

[peterhunt]

It writes a .huxley dir that contains screenshots and a log of user actions. It can replay these user actions programmatically and if the screen shots have changed it will save new ones to disk. You can use a tool like Kaleidoscope to compare the old ones with the new ones (we use git to keep the old ones around).

When you commit, the tests will be updated and the new screenshots will be part of the code review.

Additionally, you can run it in continuous integration mode which will break the build if screenshots are out of date.