development

Writing RESTful clients

July 18, 2012 Uncategorized 3 comments , ,

There’s plenty of articles out there on how to build the perfect REST API, so I am not going to go into that. This article is all about what to think about when you’re building the perfect client for a REST API. What you’re being served is clearly important – but how you use that information is 50% of the problem – and it needs to be done correctly.

API designers can spend a lot of time ensuring that their responses to you are perfectly well formed and self describing – but unless you pay attention to what it is that those messages are telling you, all that information is simply being thrown away.

Connection Negotiation

The very first thing your HTTP client will send to the server is the Request headers. Within that request, more often than not, you will send the Accept header and a list of media types that your client can support – and the order of preference within which you would like to see them. Chrome 20.0.1132.47 sends the following Accept header.

Accept:text/html,application/xhtml+xml,application/xml;q=0.9,/;q=0.8

Obviously this is a web browser, so it’s preference is going to be for text/html or application/xhtml+xml. Failing that, we can also take application/xml. If the server still can’t handle the request – then we’ll take anything at all (*/*).

The web server will make a decision based on what your client has asked for. From that point on one of 3 things can happen. 1) You get back one of the media formats that you asked for. 2) You receive a 406 response (Not Acceptable). 3) You get back something you didn’t ask for at all (this is allowed as per RFC2616 – the server SHOULD only respond with a 406).

This means that unless you’re explicitly checking what the response is after the request comes back – you could find yourself only assuming that the server is behaving as you think it is.

The Accept header becomes very useful when you separate out a normal response from an error response. In the situation where a server error is causing a problem, you are likely to receive a 5xx response code from the server with little or no meaningful body (text/plain). Where an application error has occured you should also receive a 5xx (or a 4xx) code from the server – however having declared you are able to handle a more specific media type (application/vnd.error+xml for example), you are able to present more information to the user than you would otherwise be able to.

Take the situation where you are consuming an API which serves up application/hal+xml and text/plain (hal for the resource representation, and a plain text for an error). The server is free to be able to add support for representing errors in application/vnd.error+xml without affecting any existing client – as long as the server continues to serve text/plain unless the new media type is deemed as acceptable to the client. Once the client is updated to add support for the new media type, it can take advantage of the additional information that can be provided.

Cache

A RESTful API will contain explicit directives that declare if and for how long the response body can be cached for. This information is contained across Cache-Control / Vary headers or Expires header. If the response to your request comes back from the server and declared itself as being able to be cached (only Cache-Control: no-store is excluded from user-agent caches) then you should store it locally and serve up the cached response as long as it’s valid.

There are of course already libraries you can use in your projects that will handle cacheing for you. If you usually use PHP for this stuff (I do), then I highly recommend you take a look at Guzzle to handle this for you.

URLs

Yeah i’m going to talk about it again. Hypertext as the engine of application state. Golden rule #1, never construct links in code (unless you’re following a URI Template) – follow them from the links given to you in the resource. This means that when these links change – you don’t have to worry about your app breaking. #2, use the defined link relations of the hypermedia type to navigate through the application (ie, HTML’s ‘rel’ attribute on the link tag). That’s about it. Don’t assume anything if it’s not defined by the media type rules and the representation you receive and you can’t go wrong.

Code review Karma

April 9, 2012 Uncategorized 1 comment

Implementing a good code review system into a development process is something that I have found quite difficult to do. The idea of it is conceptually quite simple – just make sure that before code is accepted into the main development stream that it’s reviewed by someone before being allowed to pass. Tools like github compliment git really well but force you down a development path that really suits open source software development, but not necessarily a co-located team.

My problem is that I trust my developers to do the right thing. Ok, so that’s not a problem – it’s actually a really nice place to be. I see code review as a learning exercise for good developers, but I don’t see is as a blocking part of the development cycle. Too many times I have seen code reviews as a blocking step in a software release cycle end up just being a formality to get through.

With that scene set i’ve been thinking of using a karma based code review system. Developers have karma (per project) and can give out positive and negative karma to other developers. At a certain level of karma, the number of reviews required on commits starts to drop off so that eventually a commit by a developer with high karma will not generate a code review (or the percentage of commits submitted for review will be very low). As a project progresses, hopefully this leads to a place where developers who will learn more from having reviews done will have the benefit, while the developers who are well established on the project are able to get code into the project repository without a blocking code review process. Improvement can be measured by karma increasing over time.

Do you have experience with code reviews on projects? (open source vs closed source / colo teams vs non-colo / blocking vs non blocking)… Interested to hear others success and failure stories!

So you think TDD isn’t worth the cost?

January 28, 2012 Uncategorized 1 comment ,

Think of the full software development life cycle. We gather requirements, create user stories, break them down into tasks, estimate, fight with your client about how long it’s all going to take, renegotiate the timings (what’s 35% of the development time on unit tests? Do we need those?), submit a revised timescale with cut down requirements and no time allocated to testing. Then we start development.

And soon the problems start coming up. At the end of the first sprint the client raises a few bugs with the software delivery… We didn’t account for those in the plan! And we can’t estimate them – we don’t know the size of the problem. That’s why it’s a bug… So we allocate some time in the next sprint to work on the top priority bugs and commit to developing fewer features.

The second sprint finishes, and more defects are raised in the software – some of them can be attributed to requirements not been defined clearly enough, but some of them are just code that we don’t understand how it even worked in development! Did someone actually test it before it went into the main development branch?! (how many people can honestly say they have never heard that one before…?)

Pretty soon you’re way behind the plan. The number of bugs that we cannot estimate and didn’t plan for are sucking up too much time and we don’t know how long it’s going to take to get back to a point where we know how long the project is going to take.

So what’s to be done? Bugs throughout the development life cycle are inevitable – but because we cannot know how many there will be and how long they will take to fix, we need to be able to minimise the number that crop up.

Some research done by microsoft (http://research.microsoft.com/en-us/groups/ese/nagappan_tdd.pdf) has shown that spending 15-35% extra development time on adopting test driven development resulted in a pre-release defect density decrease of 40-90%.

The ideal is when we can plan, develop and release software that contains no defects in a timescale that everyone agrees on. This wont happen any time soon – but an increase of 15-35% development time for up to a 90% decrease in the number of defects has *got* to be worth it by anybody’s standards.

Have you used TDD (or BDD) and experienced similar results? Comments encouraged!

Buildings a Continuous Integration Server for PHP with Hudson

October 4, 2010 Uncategorized No comments ,

An article that I wrote for techPortal has been published.

http://techportal.ibuildings.com/2010/09/20/building-a-continuous-integration-server-for-php-with-hudson/

PHPNW TestFest 2010

August 12, 2010 Uncategorized No comments ,

I am very pleased to announce this years PHPNW TestFest over at MadLab in Manchester. The event will take place on Saturday 11th September from 12pm until sometime around 5-6pm. Lunch will be provided (courtesy of our sponsor, Ibuildings), and I daresay we may wish to make an bit of an evening of it at a local bar…

More information on what the php testfest is can be found over at http://wiki.php.net/qa/testfest-2010 and you can register your interest over on upcoming, at http://upcoming.yahoo.com/event/6621123.

All you will need on the day is to bring your laptop. The venue will provide all of the connectivity that we need.

Hope to see you there!

Unit testing protected/private methods in a class

June 2, 2009 Uncategorized 4 comments ,

I think a lot of people when getting into unit testing naturally assume (and are told) that typically, a unit is a single method or function within your code, and each unit should have a unit test.

To an extent this is true – until you are presented with what you do with protected or private methods within classes.

There are two camps of thought on this issue.

One (that I disagree with) says that these methods ARE units in their own right, and should be tested as such (using introspection, or language hacks etc). In PHP this can be accomplished by abusing the __call method in your class (to allow a test suite to call protected or private methods on your class).

The second, is that a ‘unit’ is actually a single call to the public interface of your class. The protected and private methods that are called within the class are implementation details and should be allowed to be refactored entirely. Providing the behaviour and the public interface of the class does not change, refactoring does not involve ANY modification to the class itself.

Because the implementation details of a class do not constitute a unit in it’s own right, testing these methods in isolation are therefore incomplete and actually invalid (as they never have the context in which they are used, and even though your code may show 100% lines of code coverage, the branch coverage will always be far from complete).

So – if you’re tempted to create a test for a protected/private method within a class, then you’re moving away from unit tests and into protecting your implementation using a unit testing framework.