Web API’s have an Identity Problem (In response to and in support of @davewiner)

If you’ll remember, a while back I announced I was implementing RSSCloud on Windows Azure. By and large this is going well and I expect to have a demo up and running soon.

PS. what follows below is based on an email I sent to my Honours Year supervisor at University and some of this will make it into my thesis too.

The RSSCloud API relies on HTTP POST for messages in and out. And I initially thought Windows Communication Foundation was the way to go.

(bear with me, I’m using this to illustrate my point)

Up until now, WCF has been working. However, in order to actually test the RSS Cloud code, I’ve had to have it the WCF Operation Contracts written as a REST service. Its clearly HTTP POST, but its not in the RSSCloud specification. Though arguably, it should be. Why do i say this? Developers should be able to write code they are comfortable with. Whether than is REST or POST or SOAP or its against an WSDL generated API client.

Back to my little problem. So instead of:

[WebInvoke(Method = "POST", UriTemplate = "/ping?url={value}")]

        [OperationContract]

        String Ping(string value);

I had to use:

[WebInvoke(Method = "POST", UriTemplate = "/ping/url/{value}"/)]

        [OperationContract]

        String Ping(string value);

There is a subtle difference. HTTP post uses the query string, where as REST uses the url itself to transmit information.

Sending a HTTP POST to the first method (where the query string is of the form" ?url={value}&port={value}&…..") hangs the test client. The request is accepted, but it never returns. I can’t even debug the method. Using a pure REST url (the second method), things work perfectly.

In order for project as a whole to conform to the project specification (by which I mean the interoperability of the program and its compliance with the HTTP POST methods defined in the RSSCloud specification), being able to accept HTTP POST is paramount.

I spoke to one of my WCF savvy lecturers. Basically he said that there were two ways to doing this: either stick to using a REST. Or encode the url as part of the POST data. Neither of which solve the problem of sticking to the specification and using HTTP Post.

So, I was digging around ASP.Net MVC 2  yesterday. I was building the page that will actually display the posts in a feed on the page. I noticed that the Controller actions that handle the request  (i.e the feed id to get) have a [HttpPost] attribute above them. I’d never really given that much thought until yesterday.

After my little chat, I had a hunch. Using MVC, I simply added a controller action like so:

[HttpPost]

        public RedirectToRouteResult ThePing()

        {

            string url = (string) Request.QueryString["url"];

            url = url.ToLower();

            ……..

And it worked flawlessly. After all my wrestling with WCF configurations and what not, i was actually quite shocked that it worked first time. One of the problems with working with new frameworks is that you keep discovering new things, but only long after you shoud’ve known.

So, to hit the ThePing method above, the url is http://rsscloudproject/Cloud/ThePing?url=….... (obviously this isn’t deployed yet)

Why does this work?

The reason is quite simple: As I understand it, MVC exposes the Request object for you to use directly, while WCF hides this somewhere in the bowels of its inner workings. So, without getting a handle on the Request object, I can’t force WCF to process the query string differently. Hence, WCF was the wrong choice of framework for this.

So my code is now 100% in compliance with the HTTP POST methods defined in the RSSCloud Specification

Now, what does this mean for the WCF REST project?

I’m keeping it as part of the project. It gives a REST interface, and it gives WSDL that developers can use to build against my service.

Not so much the case with REST, but I personally think that the concept of a WSDL is under-represented when it comes to web based APIs. Adding these two additional interfaces to the RSSCloud specification will be one of my recommendations in the final report. I feel strongly that a web based API needs to give developers as many alternative interfaces as possible. Its no fun when you know one way of doing things, but this API is only provided in another.

For example. I wish Smugmug provided a WSDL that I could point Visual studio to and generate a client for.

Both of these situations illustrate a problem among Web API’s.

I wrote a while  back that Bill Buxton’s Mix 10 keynote about designing natural user interfaces, interfaces that respect the abilities and the skills acquired by the user also applies to designers of API’s.

Bill gives this wonderful example of a violin. The violin itself may be worth millions of dollars (if I remember correctly, Joshua Bell paid $4.5 million for his Stradivarius). The bow of the violin for any first violinist in any symphony orchestra is never less than $10,000. Remember these are musicians. They make a pittance. So as a proportion of income, it’s a fortune. But it’s worth it. Why? Because it’s worthy of the skills that those musicians been acquired over decades.

Dave Winer today published a post about Facebook not providing XML API responses. And bemoaning that Twitter is going to do the same. Dave does not want JSON. He wants XML. Why? He feels comfortable with it, and he has to tools to work with it. Clearly the new API changes do not respect Dave Winer and the abilities he has acquired of decades.

I left the following comment:

I totally understand where you are coming from.

On the other hand, tools will always be insufficent. I don’t think .Net, for example, has JSON support built in, either.

Technology moves so fast, as you say, that next week there will be something new and shiny. Developers find themselves in the curious position for having to write for today, but to prepare for next weeks new thing – they have to find a way to straddle that fence between the two. Open Source is not the complete answer to this problem ( its part of it tho).

  • So, API developers have the responsibility to provide for developers.
  • Tool developers (closed or open source) have a responsibility to provide the tools in a timely fashion.
  • And developers have the responsibility to have reasonable expectations for what they want supported.

This is a large and deep problem in the world of web API’s. They don’t have to be Just JSON or Just XML or Just HTTP POST or Just XML-RPC or Just SOAP or Just WSDL. This collection of formats and standards can co-exist.

And co-exist they should. An API should be available to the widest possible cross section of developers, to respect the individual skills that these developers have acquired of years and decades.

Because when you don’t do that, when you don’t respect that, you make people like Dave Winer stop coding against your API.