Substitute "XML" for "JSON" and we've now come full circle.
The point about REST is that it is self-describing. And ideally should be using the same URIs as the version people clicking around in Firefox or Chrome see. The API is just the XML or JSON or whatever is flavour of the week version of the HTML version.
(Or we could use embedded data—microformats, microdata, RDFa—and get rid of that distinction.)
Agreed. I came here to post something similar but I was going to mention working with SOAP[1] in addition to what you mentioned. It sounds like the OP is trying to do something which sounded very much like using SOAP and XML to me.
To paraphrase the OP, "Since so many APIs can be described in similar terms, why don't we have some sort of standard that one can look at to identify how to use the API instead of letting the API speak for itself?"
When you start going down this track, you're not only making things complicated on the client's end of things. On the server side, you're having to maintain two things for the API now. First: the ruleset, ensuring it's 100% to spec lest a client fail. Second: the code generating the response in the first place.
I've built clients and servers for both RESTful and SOAPy APIs and I can say I would take REST any day.
Yes, URIs in the response sounds amazingly cool, but it won't change anything.
The inline URLs of the web work because the consumers are humans who can deal with changes (more than just trivial URL changes, like added, removed features) and now click on this button or that button.
Software isn't that flexible, so it will be just as coupled as it is today--you're just moving the coupling around.
So this idea of a "robust client-server relationship" is a pipe dream IMO.
Can you give an example of an acceptable implementation of a HATEOAS REST API (wow, that's a lot of letters) with an associated client that actually uses it?
My experience has been that you can't communicate much through HATEOAS that's actually beneficial to a human programmer writing a client. Sure, you can add all the hypermedia links you want in your API responses, but how does that make writing a client easier? Wouldn't it just be helpful to crawlers?
Not trying to put down the idea - I want to believe, but I just haven't seen any obvious examples using it in the real world yet.
I have been puzzling over this API discovery issue on my current project (building out a reporting API).
I'm starting with self documentation for developers built in, not this (admittedly admirable) goal of a machine generated API mapping layer. I think the main issue is, you're trying to generate a generic interface to something that isn't, itself, generic.
How many versions of "RESTful" have you encountered?
Building a generic interface to non-generic interfaces is the domain of software engineers. Until we have machines building both sides of this equation, there will always be a need for human intervention.
I don't see how writing JSON REST API descriptions isn't practically the same as writing REST API clients anyway: they're still clients, just written declaratively rather than procedurally.
If the point is "stop writing procedural REST API clients and write them declaratively instead" then that advice is by no means restricted to REST API clients.
If the point is "hey, I noticed that REST API clients are another thing that we can now comfortably write declaratively" then OK.
Writing using authoritative language is very common, and widely considered a best practice.
I highly recommend you simply accept it as what it is: the way some people communicate, especially online. It's not worth your time/attention/care to think about this.
I was just thinking that, I hate article titles that are phrased exactly as a command and it's merely a blog post that wants to change an entire body of thought that's well founded.
I take all direct orders as suggestions. Sometimes this gets me into trouble; The same way it does my 2 year old. Most of the time though, it's the way to go.
I was thinking about this yesterday, but its seems like HN likes that kinda thing. Lots of frontpage article are direct orders from blogs with who knows what credibility.
You aren't, but it's a mistaken thing to dislike: there's no value to rephrasing it as "I believe you should [do X]" or "I am arguing that you should [do X]" because that's necessarily always true anyway, i.e. no article can ever be anything other than what the author believes and argues for. So 'softening' the language would be inefficient - it would use more words while adding nothing of substance.
I read it as (I'd like to show you something that may help you) "Stop writing REST API clients". Imagine trying to visually scan HN article titles having to filter through useless pleasantries like that.
This article is not at all about REST, it is about RPC and its shortcomings. These shortcoming were fixed by REST, and the author of the article rediscovers these fixes.
A key features of a REST API is that is self describable, in a sense, that it has a single entry point from which a generic client can automatically discover available resources and actions. APIs in the given examples are not like this, they require custom clients that are strongly coupled with an application. These are not REST APIs but RPC APIs.
> A key features of a REST API is that is self describable
How practical is that, in reality?
I know I've added the whole HATEOAS thing to my API and I am not sure if it just makes my IDs longer. Customers seem to hard-code the API entry points anyway. Everyone of course says "Oh, yeah this is cool" but when it comes to doing it given performance constraints, they don't want to start generating 10s of extra GET requests on startup to rediscover the API.
Now I can say "well not my problem" and that is what I say, except that looking back and I just don't see the practice match with the supposed theoretical advantages of the REST-ful interface.
Another issue I see happening, is the return of message bus like interface brought about by Websockets and server push optimizations it makes possible. I think REST and Websocket's channels/message bus architectures will have a battle at some point -- and one will end up dominating.
Just like AMQP is becoming a standard for message brokers, I think at some point that will be extended to the browser. Kind of like RabbitMQ has the web-STOMP plugins. I can see future hotness being just that -- message broker architecture all the way to the web client and everyone will laugh at REST-ful hype just like we are laughing at SOAP now.
In what way is REST self-describable? REST is not a standard, but rather a widely accepted convention.
I have seen a few RESTful servers self-describe, (ie. GET /api/v1/ returns ['/users', '/posts']). However you can't claim this is a key feature of REST clients because there is no agreed-upon standard to have services describe themselves. HTTP is not sufficient.
If there were a real standard here, we would not have this problem. Like it or not, everybody is calling their custom API a 'REST' API nowadays, and without a real standard, nobody is wrong.
All that these sorts of description produce is a low-level API. That can be useful, but what's really needed are high-level APIs that provide meaningful semnatics:
my $me = Facebook->new( username => 'autarch' );
$me->post_status("I'm on Hacker News writing this comment");
my $friend = Facebook->new( username => 'imaginary' );
$me->post_on_wall( $friend, "Hey buddy, I am on Hacker News writing this comment" );
Really? Seems to me like Hypermedia APIs should move the problem from the wire protocol to the application protocol?
Hypermedia says "oh yeah, here's some markup, look there are URIs in it". For a human user, we're like "cool, I'll try and click these, see what they do".
But software is going to want "um...okay, how to I parse this markup, and how do I generate the submissions you want? And, okay, you can change URIs, but please don't change anything else about that operation, or I will break completely. That's right, we're not really decoupled."
So, even with hypermedia APIs, AFAIK you're still going to want some marshaling to/from host language. ...and so you're back to having a spec, and coupling, you've just moved it around.
(Rant on coupling, people seem to think it's always bad and you can make it go away. Reality: you can't make it go away, and sometimes just accepting it directly is a whole lot simpler than deceiving ourselves about it's existence by over-applying abstractions.)
We've been experimenting with this at my office. We use yaml descriptions of all of our routes to generate test coverage. We plan to later generate our documentation and client libraries with the same docs.
Document-generated server behavior is something we're researching as well, to possibly represent business logic. We're hoping that patterns can be found and condensed into notations, like Regular Expressions do for string-parsing. We'll post about anything that we come up with.
One of my side projects us an Ajax library which allows javascript to respond to requests (LinkJS [1]). It has a helper object called the Navigator, which is like a miniature Web Agent. It retains a context, and uses the Link header from the response to populate the navigator with relations. It works out like this:
var nav = Link.navigator('http://mysite.com');
nav.collection('users').item('pfraze').getJson()
.then(function(res) {
console.log(res.body); // => { name:'pfraze', role:'admin' ...}
})
.except(function(err) {
console.log(err.message); // => 404: not found
console.log(err.response.status); // => 404
});
The advantage is that the link header is a relatively condensed representation of the resource graph. As a result, it's not a problem to send it and process it every time. You do gain latency, but the internet is only getting faster, and caching can be used. Meanwhile, the server can rewire links without interrupting their clients.
Json Schema is pretty good at describing data coming through JSON. But describing (REST) APIs requires more. For example a standard way to describe API endpoints with parameters and response types, errors, related models, default/allowable values etc. This was what the OP was referring to and this is what Swagger is trying to do. The Swagger Spec is here with some more details on whats required in addition to JSON Schema to document APIs: https://github.com/wordnik/swagger-core/wiki/API-Declaration Incidentally model/data specifications in swagger spec does map closely with json schema.
While I agree with the title, I am not so sure about the solution presented. HATEOAS, whether encoded in JSON or XML, can only give you so much information about the semantics of links.
IMHO, what's needed is better support for "generic" REST in programming languages and/or libraries. Objective-Smalltalk (http://objective.st) features "Polymorphic Identifiers", which make it possible to both interact directly and abstract over web interfaces.
For abstraction, you can build your own schemes, either directly in code or by composing/modifying other schemes. For example, if I want to look up RFCs, I can define the rfc scheme:
I actually have no idea why the hal people aren't writing hal specifications for existing services right now--it's not quite as nice as "native" support, but the format supports it, and it would be useful to see what a hal version of the Twitter API looked like, for example.
This is essentially trying to solve the same problem as Swagger. Swagger is "a specification and complete framework implementation for describing, producing, consuming, and visualizing RESTful web services" [1]. Check out the spec on GitHub here [2].
There is a lot of talk about this idea being SOAP-like, but I disagree.
SOAP was insane and its counterpart, WSDL (which is really the part that is most comparable to this idea), was even more insane.
But, the basic premise was not bad. It was the execution which sucked by trying to account for every situation, adding namespaces, etc. And if you ever worked with language libs designed to interface with SOAP/WSDL, it would make you slap a bunny.
With this idea, however, adding an optional JSON-based descriptor language could be helpful. Key would be to keep it simple, allowing the bare mnimum number of data types, with one simple array structure for collections. Allow object definitions with an infinite number of nesting levels, and that would be it. I wouldn't even get into optional vs required stuff, validation, etc. That stuff should stay at the application level. Why stuff it into the interface layer?
From there, it would be easy to develop libraries to generate clients in any language for any API just by feeding it the JSON descriptor. Or (as I think the author intended) just use one universal client that any app can use. For languages that aren't strongly typed anyway, the latter would be fine.
Someone mentioned that it would require the server side devs to keep the descriptor in sync with the code. No biggie for apps that already offer client libs in different languages and must keep them up to date anyway. Not to mention there should be some doc that needs to be kept in sync (REST is not typically self documenting in reality).
In any event it wouldn't be required. What would be the harm in creating a standard for those apps that choose to use it?
Can't we just stop writing clients for specific REST API:s period and rather just build one good API client that can easily be extended and adapted to any API?
That's the path I've been using in all projects lately - because frankly - I don't want to deal with a bunch of different API clients for Twitter, Facebook, Soundcloud, Instagram or whatever sites it is that I integrate with - all those different syntaxes and all that duplicated code etc doesn't help me - I want all of their individual differences hidden away for me and colleagues behind a single well known syntax which I myself can extend to expose the resources and methods that I need - like if I need it a method for posting a photo for the API:s that support that and so on.
My advice today would be: Pick a good HTTP client, preferably with good OAuth support, and then build your own extendable API-client on top of that and integrate all the different API-resources you need with that client whenever you need them.
A client template library for REST API need to be Turing complete, otherwise it will be too weak to be able to handle complex services or complex client-side tasks, such as caching, dependency relations, data that span multiple services etc. Even if you make the template library simple, you'll need to wrap that with a layer of complex code. All you've done will be adding another layer on your code. You could re-design your code to fit a manageable design, but server side of the REST APIs are usually design by others whose priority is the code on the server side. So, by definition of the very task, the REST API client code have to be a complex soup where the client considerations mix up with those of the servers'.
[+] [-] tommorris|13 years ago|reply
The point about REST is that it is self-describing. And ideally should be using the same URIs as the version people clicking around in Firefox or Chrome see. The API is just the XML or JSON or whatever is flavour of the week version of the HTML version.
(Or we could use embedded data—microformats, microdata, RDFa—and get rid of that distinction.)
[+] [-] randomdrake|13 years ago|reply
To paraphrase the OP, "Since so many APIs can be described in similar terms, why don't we have some sort of standard that one can look at to identify how to use the API instead of letting the API speak for itself?"
When you start going down this track, you're not only making things complicated on the client's end of things. On the server side, you're having to maintain two things for the API now. First: the ruleset, ensuring it's 100% to spec lest a client fail. Second: the code generating the response in the first place.
I've built clients and servers for both RESTful and SOAPy APIs and I can say I would take REST any day.
[1] - http://en.wikipedia.org/wiki/SOAP
[+] [-] michaelw|13 years ago|reply
Stop creating REST APIs that are only level 1 or 2 (see http://martinfowler.com/articles/richardsonMaturityModel.htm... ).
Start writing HATEOS systems where the client is coupled to the semantic rather than the syntax.
Machine parseable interface descriptions might get rid of some boilerplate but it doesn't make for a more robust client-server relationship.
[+] [-] stephen|13 years ago|reply
The inline URLs of the web work because the consumers are humans who can deal with changes (more than just trivial URL changes, like added, removed features) and now click on this button or that button.
Software isn't that flexible, so it will be just as coupled as it is today--you're just moving the coupling around.
So this idea of a "robust client-server relationship" is a pipe dream IMO.
[+] [-] steveklabnik|13 years ago|reply
[+] [-] algorithmmonkey|13 years ago|reply
[+] [-] robbles|13 years ago|reply
My experience has been that you can't communicate much through HATEOAS that's actually beneficial to a human programmer writing a client. Sure, you can add all the hypermedia links you want in your API responses, but how does that make writing a client easier? Wouldn't it just be helpful to crawlers?
Not trying to put down the idea - I want to believe, but I just haven't seen any obvious examples using it in the real world yet.
[+] [-] unknown|13 years ago|reply
[deleted]
[+] [-] petercooper|13 years ago|reply
Flippancy aside, maybe there's a need for a next generation of this that skips all the XML headaches after all.
[+] [-] pfarrell|13 years ago|reply
I have been puzzling over this API discovery issue on my current project (building out a reporting API).
I'm starting with self documentation for developers built in, not this (admittedly admirable) goal of a machine generated API mapping layer. I think the main issue is, you're trying to generate a generic interface to something that isn't, itself, generic.
How many versions of "RESTful" have you encountered?
Building a generic interface to non-generic interfaces is the domain of software engineers. Until we have machines building both sides of this equation, there will always be a need for human intervention.
[+] [-] stephen|13 years ago|reply
JSON has just been lucky that its user base hasn't been the same enterprise architects that ruined Java.
[+] [-] gluegadget|13 years ago|reply
[+] [-] ubermammal|13 years ago|reply
If the point is "stop writing procedural REST API clients and write them declaratively instead" then that advice is by no means restricted to REST API clients.
If the point is "hey, I noticed that REST API clients are another thing that we can now comfortably write declaratively" then OK.
[+] [-] TelmoMenezes|13 years ago|reply
[+] [-] edanm|13 years ago|reply
I highly recommend you simply accept it as what it is: the way some people communicate, especially online. It's not worth your time/attention/care to think about this.
[+] [-] TallboyOne|13 years ago|reply
[+] [-] steve918|13 years ago|reply
[+] [-] INTPenis|13 years ago|reply
[+] [-] shurcooL|13 years ago|reply
[+] [-] Goopplesoft|13 years ago|reply
[+] [-] ubermammal|13 years ago|reply
[+] [-] jasonkostempski|13 years ago|reply
[+] [-] onlyup|13 years ago|reply
[+] [-] njy|13 years ago|reply
[+] [-] mixedbit|13 years ago|reply
A key features of a REST API is that is self describable, in a sense, that it has a single entry point from which a generic client can automatically discover available resources and actions. APIs in the given examples are not like this, they require custom clients that are strongly coupled with an application. These are not REST APIs but RPC APIs.
[+] [-] rdtsc|13 years ago|reply
How practical is that, in reality?
I know I've added the whole HATEOAS thing to my API and I am not sure if it just makes my IDs longer. Customers seem to hard-code the API entry points anyway. Everyone of course says "Oh, yeah this is cool" but when it comes to doing it given performance constraints, they don't want to start generating 10s of extra GET requests on startup to rediscover the API.
Now I can say "well not my problem" and that is what I say, except that looking back and I just don't see the practice match with the supposed theoretical advantages of the REST-ful interface.
Another issue I see happening, is the return of message bus like interface brought about by Websockets and server push optimizations it makes possible. I think REST and Websocket's channels/message bus architectures will have a battle at some point -- and one will end up dominating.
Just like AMQP is becoming a standard for message brokers, I think at some point that will be extended to the browser. Kind of like RabbitMQ has the web-STOMP plugins. I can see future hotness being just that -- message broker architecture all the way to the web client and everyone will laugh at REST-ful hype just like we are laughing at SOAP now.
[+] [-] evv|13 years ago|reply
I have seen a few RESTful servers self-describe, (ie. GET /api/v1/ returns ['/users', '/posts']). However you can't claim this is a key feature of REST clients because there is no agreed-upon standard to have services describe themselves. HTTP is not sufficient.
If there were a real standard here, we would not have this problem. Like it or not, everybody is calling their custom API a 'REST' API nowadays, and without a real standard, nobody is wrong.
[+] [-] autarch|13 years ago|reply
[+] [-] ahallock|13 years ago|reply
[+] [-] ayushgta|13 years ago|reply
[+] [-] hoov|13 years ago|reply
[+] [-] St-Clock|13 years ago|reply
And then, we'll need a central location to store all of these API descriptors and UDDI [2] will be back with a vengeance.
[1] http://www.w3.org/TR/wsdl
[2] http://en.wikipedia.org/wiki/Universal_Description_Discovery...
[+] [-] onlyup|13 years ago|reply
[+] [-] taude|13 years ago|reply
I think too many people consume REST APIs in different manners, utilizing different data in unique relations. This is the beauty of it.
[+] [-] taude|13 years ago|reply
[+] [-] mtts|13 years ago|reply
[+] [-] ms123|13 years ago|reply
[+] [-] stephen|13 years ago|reply
Hypermedia says "oh yeah, here's some markup, look there are URIs in it". For a human user, we're like "cool, I'll try and click these, see what they do".
But software is going to want "um...okay, how to I parse this markup, and how do I generate the submissions you want? And, okay, you can change URIs, but please don't change anything else about that operation, or I will break completely. That's right, we're not really decoupled."
So, even with hypermedia APIs, AFAIK you're still going to want some marshaling to/from host language. ...and so you're back to having a spec, and coupling, you've just moved it around.
(Rant on coupling, people seem to think it's always bad and you can make it go away. Reality: you can't make it go away, and sometimes just accepting it directly is a whole lot simpler than deceiving ourselves about it's existence by over-applying abstractions.)
[+] [-] pfraze|13 years ago|reply
Document-generated server behavior is something we're researching as well, to possibly represent business logic. We're hoping that patterns can be found and condensed into notations, like Regular Expressions do for string-parsing. We'll post about anything that we come up with.
One of my side projects us an Ajax library which allows javascript to respond to requests (LinkJS [1]). It has a helper object called the Navigator, which is like a miniature Web Agent. It retains a context, and uses the Link header from the response to populate the navigator with relations. It works out like this:
The advantage is that the link header is a relatively condensed representation of the resource graph. As a result, it's not a problem to send it and process it every time. You do gain latency, but the internet is only getting faster, and caching can be used. Meanwhile, the server can rewire links without interrupting their clients.1 https://github.com/pfraze/linkjs
[+] [-] id_heaven|13 years ago|reply
Most overused phrase right there.
[+] [-] feralmoan|13 years ago|reply
[+] [-] ayushgta|13 years ago|reply
[+] [-] stephen|13 years ago|reply
My naive impression is that JSON-Schema is trying to be just like XML Schema, but in JSON. Which doesn't seem like a good thing.
[+] [-] mpweiher|13 years ago|reply
IMHO, what's needed is better support for "generic" REST in programming languages and/or libraries. Objective-Smalltalk (http://objective.st) features "Polymorphic Identifiers", which make it possible to both interact directly and abstract over web interfaces.
To reference a URL, just write it down:
Arguments can be added without string processing: This is a file downloader, similar to curl: For abstraction, you can build your own schemes, either directly in code or by composing/modifying other schemes. For example, if I want to look up RFCs, I can define the rfc scheme: Or I can compose schemes so the rfc scheme looks in a bunch of different places (memory, local directoy, several http/ftp servers).[+] [-] jconnolly|13 years ago|reply
[+] [-] mjs|13 years ago|reply
[+] [-] dgritsko|13 years ago|reply
[1]: http://developers.helloreverb.com/swagger/
[2]: https://github.com/wordnik/swagger-core/wiki
[+] [-] unclebucknasty|13 years ago|reply
SOAP was insane and its counterpart, WSDL (which is really the part that is most comparable to this idea), was even more insane.
But, the basic premise was not bad. It was the execution which sucked by trying to account for every situation, adding namespaces, etc. And if you ever worked with language libs designed to interface with SOAP/WSDL, it would make you slap a bunny.
With this idea, however, adding an optional JSON-based descriptor language could be helpful. Key would be to keep it simple, allowing the bare mnimum number of data types, with one simple array structure for collections. Allow object definitions with an infinite number of nesting levels, and that would be it. I wouldn't even get into optional vs required stuff, validation, etc. That stuff should stay at the application level. Why stuff it into the interface layer?
From there, it would be easy to develop libraries to generate clients in any language for any API just by feeding it the JSON descriptor. Or (as I think the author intended) just use one universal client that any app can use. For languages that aren't strongly typed anyway, the latter would be fine.
Someone mentioned that it would require the server side devs to keep the descriptor in sync with the code. No biggie for apps that already offer client libs in different languages and must keep them up to date anyway. Not to mention there should be some doc that needs to be kept in sync (REST is not typically self documenting in reality).
In any event it wouldn't be required. What would be the harm in creating a standard for those apps that choose to use it?
[+] [-] VoxPelli|13 years ago|reply
That's the path I've been using in all projects lately - because frankly - I don't want to deal with a bunch of different API clients for Twitter, Facebook, Soundcloud, Instagram or whatever sites it is that I integrate with - all those different syntaxes and all that duplicated code etc doesn't help me - I want all of their individual differences hidden away for me and colleagues behind a single well known syntax which I myself can extend to expose the resources and methods that I need - like if I need it a method for posting a photo for the API:s that support that and so on.
My advice today would be: Pick a good HTTP client, preferably with good OAuth support, and then build your own extendable API-client on top of that and integrate all the different API-resources you need with that client whenever you need them.
[+] [-] utopkara|13 years ago|reply