Requestdiff – Send two HTTP requests and visualize any differences

[jalfresi]

For command-line junkies:

diff <(curl -sS -L https://httpbin.org/get) <(curl -sS -L https://httpbin.org/get?show_env=1)

[jalfresi]

Actually this is closer still to the output. Requires colordiff:

diff -u <(curl -sS -D - -L https://httpbin.org/get) <(curl -sS -D - -L https://httpbin.org/get?show_env=1) | colordiff

[tacone]

The website does show differences in the HTTP headers though.

[therein]

Good work but frankly this would be useful if it was:

(1) Terminal based

(2) Supported other types of HTTP requests

(3) Supported request body

(4) Allowed editing request headers

(5) Wasn't so easily exploitable to be used as a proxy or a DDoS relay (server-side bummer):

http://requestdiff.com/proxy?url1=https://www.amazon.com&url...

[JackuB]

You might want to take a look at Gavel https://github.com/apiaryio/gavel (and its JS implementation Gavel.js https://github.com/apiaryio/gavel.js)

(disclosure: I work for Apiary.io)

[odabaxok]

(6) Support encodings other than utf-8.

[diggan]

This is very cool! Looking forward to it being open source (also, things from Barcelona rocks ;) )

For people wanting to have a CLI tool instead, John Graham-Cumming's httpdiff might be worth looking at. https://github.com/jgrahamc/httpdiff

[victordg]

Thanks a lot :)

Httpdiff is a really good tool and way more powerful than mine.

I've used it multiple times and wanted to try making something that focused more on the request body (which in Httpdiff are saved to temporary files) and that was also a bit more visual than a CLI.

Btw, here is the code! https://github.com/vdel26/requestdiff

[gregingelmo]

Use Vim, it's the best diff tool by a long shot. Syntax highlighting, side-by-side diffs, and code folding (when large contiguous sections are equal)

vimdiff <(curl -s "https://httpbin.org/get") <(curl -s "https://httpbin.org/get?show_env=1")

It takes some getting used to but once you go vimdiff you'll never go back.

[smarx]

Seems like a useful tool; thanks for sharing it!

Two questions:

1) What's the diff logic? At first glance, it looks like JSON is reformatted (maybe canonicalized in some way) and then a line-by-line diff is applied. Is there more to it? Since the tool seems JSON-aware, I was surprised to see an added trailing comma up as a difference.

2) Do you have plans to expand the kind of HTTP requests users can make? It would be nice to use different verbs, headers, and request bodies. Runscope has a similar tool[0] built in that I believe (haven't tried it yet) allows a bit more flexibility, but it would be nice to have a standalone tool available.

[0] http://blog.runscope.com/posts/comparing-http-requests-using...

[victordg]

Great questions:

1) the diff is really simple, line by line and text based. I'm just using this really awesome library to do it: https://github.com/kpdecker/jsdiff The JSON is just being pretty printed to be able to do a text diff in a meaningful way.

2) Yes, I do, and thanks for the suggestions! But I just wanted to go with the minimal version first and see if people found it useful.

I didn't know about Runscope's diffing feature. Btw, I'm using a HTTPBin for the sample requests, which is a Runscope project as far as I know :)

[no_gravity]

Comparing URLs comes up from time to time here on Hacker News and at one point inspired me to write a tool that batch compares pages visually:

http://www.productchart.com/blog/2015-07-19-urldiff

It renders a set of pages in a headless browser, compares them visually and alerts you if something changed.

Just a few lines of bash as you can see. But it turned out to be pretty useful. UrlDiff is a regular part of our regression testing at Product Chart now.

[kozhevnikov]

Reminds me of Diffy, a tool Twitter is using to algorithmically diff test their HTTP-based services.

https://blog.twitter.com/2015/diffy-testing-services-without...

[pixelbeat]

Personally I use these from the command line:

http://www.pixelbeat.org/scripts/urldiff http://www.pixelbeat.org/scripts/idiff

See also mergely which supports diffing URLs: http://pixelbeat/programming/diffs/#mergely

[andybak]

Wonderful.

Just the other day I needed something similar and was disappointed that I could find it.

I wanted to discuss something with a remote colleague and to illustrate it I wanted a visual diff of two files. I was hoping there was a nice little web app offering this but I was forced to screenshare (I could have terminal-shared but it was more hassle).

I was hoping for something like Etherpad but with a live visual diff.

[dannybtran]

Not working on Safari Version 8.0.6 (10600.6.3) Mac OS X 10.10.3

From JS Console:

[Error] TypeError: undefined is not a function (evaluating 'Array.from(e)') _toConsumableArray2 (app.min.js.pagespeed.ce.ozGaCBt6Kj.js, line 1) s (app.min.js.pagespeed.ce.ozGaCBt6Kj.js, line 1) f (app.min.js.pagespeed.ce.ozGaCBt6Kj.js, line 1) onload (app.min.js.pagespeed.ce.ozGaCBt6Kj.js, line 1)

[vortico]

It's pretty easy to write a recursive diff function that compares JSON strings, in order to avoid the JSON -> diff by line hack that you're doing. But it's a clever hack that easily translates to the command-line.

[skastel]

If you're looking for something similar, but with more features that natively understand HTTP, check out www.runscope.com. Full disclosure, I'm an employee and proud of what we've built!

[krakensden]

Another solution: https://github.com/xthexder/httptee

A daemon that sends to two backends, and diffs the results.

[zwischenzug]

This would be useful behind a firewall. Is the source available?

[victordg]

Not yet, but I'm planning to open source it later today. I will reply here with the Github link. Glad that you found it useful!

[amelius]

I'd like to have a bookmarklet that lets me compare the states of the DOM trees of a website taken at two different times.

[blemasle]

Requests are made server side, which make this unusable for local/dev environnement purpose. Too bad !

[victordg]

Right. I was aiming to do it client-side only but I found out that web browsers restrict the headers you have access to through cross-origin XHR requests [1][2].

So it was a trade-off and I preferred showing all the headers at the expense of having to proxy the calls through a server.

[1] http://www.w3.org/TR/XMLHttpRequest/#the-getallresponseheade...

[2] http://www.w3.org/TR/cors/#cross-origin-request

I will make it available on Github later, so that you at least have the option of running it yourself.

[JackuB]

For dev or CI you might want to take a look at Gavel https://github.com/apiaryio/gavel (or even Dredd for that matter https://github.com/apiaryio/dredd)

(disclosure: I work for Apiary.io)

[gdc]

Doesn't seem to work at all in IE11 / RTM Win10.

[SimeVidas]

Use cases?