Hypermedia types and connection negotiation

September 25, 2012 Uncategorized 3 comments ,

I was fortunate enough to be able to travel to RESTFest in Greenville SC over the past week to spend time mixing with people who are highly influential in the world of REST and Hypermedia. Many interesting things where discussed there and it got me thinking about connection negotiation a little more (one of many many things I have taken away from the conference – subjects for later articles!)

Before attending the conference I had always advocated taking an existing hypermedia type (for the examples here I will use application/hal+json) and trying to represent a resource using that. More often than not, I would require some affordances that are not defined in the spec – so extending it and creating my own vender specific mime type (which defines what base type it extends) made a lot of sense to me.

Now that various media types are become ubiquitous amongst API designers (Hal and collection+json amongst others), it’s made me see the advantage of sticking to the base type whilst negotiating with the server on what content you receive. The reason? Because other clients that understand these hypermedia types are staring to emerge – much like the web browsers in which we browse the text/html hypermedia type today.

Like HTML, hypermedia types are created with the understanding that if an element exists that the client is not able to assign meaning to (an affordance that is not part of the spec, or not implemented by the client), then it can simply ignore it as if it was never there. This is quite powerful – it allows you to define optional extensions to a hypermedia type whilst maintaining compatibility with clients and intermediaries which already understand your base type.

Collection+json supports this idea quite well, and the author (Mike Amundsen) accepts additional patches to define extensions as part of the specification – extensions that one day could form part of a revised version of the specification for everyone to use, but exist as extensions that can be understood by any third party client that support them.

It is now my opinion that the creation of custom mime types (without registering them with IANA) actually harm the interoperability of the web – and close down a hypermedia API to clients which have specifically been created for your own hypermedia type.

Perhaps a missing piece of the puzzle right now is a way of being able to declare which extensions your document is actually making use of (if this is even needed once the extensions become part of the specification). Using a Link header (http://www.w3.org/wiki/LinkHeader) is potentially a good way of achieving this, though exactly how this works in practice is still being considered.

Comments encouraged!

JSON Sucks

April 26, 2012 Uncategorized 14 comments , ,

Ok, so perhaps I was a little bit OTT with the title of this blog post (but it got your attention, right?). What I actually mean is that JSON Sucks for hypermedia APIs, and I am going to tell you why. But before I do, I just want to point out that actually I think JSON is awesome for what it’s designed for – serialising an object. Objects naturally contain data structures like ints, strings, arrays and other objects – if all you need to do is represent those four things, you can’t go wrong with JSON. Seriously.

Hating XML is rather fashionable these days, so many API’s are appearing that are JSON representation only even when XML really is a better fit for purpose. When you’re designing a hypermedia API you’re not simply serialising an object. There are media types that add semantic meaning to JSON (application/hal+json, my own application/vnd.error+json (draft) and others) but really, XML is so much more expressive that using JSON actually makes your clients have to work harder to understand the meaning you’re trying to convey using an object serialisation format over a markup language.

XML gives you some stuff for free – ‘xml:lang’ for one. Put this on any element of your document and just the fact that your document is XML means that you can express elements in different (human) languages without having to update your media type.

Consider how this actually looks in XML vs JSON representation.

<error id="6">
    <message>Something bad happened</message>
    "error": {
        "id": 6,
        "message": "Something bad happened"

This looks easy, right? Consider a future version of the api where we add support for multiple languages.

<error id="6" xml:lang="en">
    <message>Something bad happened</message>
    <message xml:lang="de">Etwas schlimmes ist passiert</message>
    "id": 6,
    "messages": [
        { "lang": "en", "message": "Something bad happened" }
        { "lang": "de", "message": "Etwas schlimmes ist passiert" }

An alternative could be to key the message objects by language;

    "id": 6,
    "messages": { 
        "en": "Something bad happened",
        "de": "Etwas schlimmes ist passiert"

You can see in both scenarios that “messages” has become an different type (string to array of objects, or string to object). We’ve changed the structure of the document and introduced an almost certain backwards compatibility (BC) break in our interface – all our clients now need to be updates to follow suit. We could leave “message” as it was, and add “messages” for the multilang version of the API but then we’re duplicating data and / or making the client have to work harder.

The XML version can continue as is and the client can simply ignore the additional languages until they’re updated to support them (if they ever need to).

Then there’s just the matter of basic human readability. It’s clear to see what’s going on in XML and not so much in JSON. This is a fairly simplistic example but extrapolate that out over a fairly complex representation of an object and you can see how JSON really isn’t always the best choice for representing resources.

Imagine the scenario where we had used the second option above as our JSON representation and we wanted to add in another attribute for each of the messages. See the problem? Anything we choose to do will cause a BC break and require clients to update (or result in duplication of elements to ‘version’ the api), meaning it’s very difficult to evolve the clients and servers independently.

So i’m not saying that JSON simply ‘sucks’. I’m just saying that when considering the formats you use for representing data in an API (when it comes to selecting the media type you want to use), think hard before just going with something based on JSON by default because it’s ‘easy’. It probably isn’t.

The error hypermedia type

April 24, 2012 Uncategorized 5 comments , ,

When designing a complex hypermedia API you will inevitably come across the situation where instead of returning the required resource, you need to return an error state. Currently, there is no hypermedia type registered with IANA to deal with this scenario, so it is left up to the API designer to construct and document a mime type that can be used to express an error state in the application.

Because of this, several people have come together and put together a proposal for the application/vnd.error media type. It has been documented over on Github and the mime type has been submitted for approval by IANA.

The intention for this media type is that it should be used when a client has made a request, however the server is unable to fulfil the request to to either a server side error (a code 5xx) or a problem with the client request (a code 4xx).

Although the client does NOT have to state that it explicitly will accept the application/vnd.error+xml or json mime type (via the ‘Accept’ header in the request), it makes a lot of sense to do so if the client can understand that response type. However, RFC2616 section 10 states:

HTTP/1.1 servers are allowed to return responses which are not acceptable according to the accept headers sent in the request. In some cases, this may even be preferable to sending a 406 response. User agents are encouraged to inspect the headers of an incoming response to determine if it is acceptable.

It is up to the implementor to decide whether the api should return a 406 (Not Acceptable) response if the client states it will not accept application/vnd.error+xml or json and an error is triggere, or if it should return the application/vnd.error+xml or json response anyway.

The specification for application/vnd.error is draft – if you have comments / suggestions or enhancements to the media type, let’s talk (comments on this blog, issues on github, twitter etc).

REST and the Hypermedia Constraint

January 8, 2012 Uncategorized 1 comment , ,

The video of my talk at PHPNW ’11 has now been posted online at blip.tv


xhtml as a hypermedia format

October 2, 2011 Uncategorized No comments

XHTML is a tempting hypermedia format to use in a web service. A lot of the time, the sort of data that you serve using an API is very similar (if not the same) to what your website displays on its own. XHTML already has all the functionality built in for describing forms, where to post them back to once filled in, how to link to images, documents or any other form of data across the whole web of information.

Despite all this good stuff that you get for free, xhtml, or rather the xhtml media type (application/xhtml+xml) has its problems. And one of those problems is webkit.

Mobile devices are incredibly important in modern web development, so designing for the capabilities of those devices is just part of a standard workflow. The problem however, is that webkit on these devices prefer application/xml and application/xhtml over our friend, text/html. Take a look at the following Accept header sent by two desktop browsers (Firefox and Chrome):

Accept: text/html,application/xhtml+xml,application/xml;q=0.9, */*;q=0.8

Contrast this with what is sent from the iPhone (iOS 4.3.5):

Accept: application/xml,application/xhtml+xml,text/html;q=0.9, text/plain;q=0.8,image/png,*/*;q=0.5

And further, and Android phone (2.3.5):

Accept: text/xml, text/html, application/xhtml+xml, image/png, text/plain, */*;q=0.8

And suddenly we have a problem. If we use application/xhtml+xml then we have to account for iOS and Android devices preferring it over text/html.

There’s two things we can do to counter this. The first is to put a filter in front of your application to detect the user agents for browsers that are known to prefer the xml/xhtml representation that we want to reserve for our API clients. This is an application specific solution – you have to know that the browsers that you are filtering and then rewrite the Accept header so that text/html is served up by your application. You also have to accept that requests from those browsers will never be issuing requests against your web service as those requests will be rewritten to text/html.

The second solution is to use a vendor specific media type, and serve xhtml up on that. This means that you lose a small part of the self describing nature of using application/xhtml+xml – but means that you can be absolutely certain that no existing browser will prefer your API content over what it should be seeing. Vendor media types take the format of application/vnd.. – for example application/vnd.fdrop.xhtml+xml

Opinions on which of these two options are preferred (or if you have heard of other solutions or have any suggestions on other ways of dealing with this) would be most welcome.