Consuming Free APIs

My day job has recently thrown up a nice new challenge, producing a public API for the company's core system to enable simpler access to the ecosystem for 3rd party developers.  Historically 3rd party developers have been able to interface with and customize the system by consuming an SDK, but that was in the days of an on premise installation of the system.  We are pushing towards a multi-tennented, single instance, cloud based offering, and this cannot be extended or customized by the use of an SDK.  How could the multiple customers using the system each have their own extensions and customizations, purchased from 3rd parties, when there is only a single instance of the service centrally hosted on the cloud?

As a result the decision was made to replace the SDK with a RESTful API which allows access to the data of the individual customer and thus multiple extensions could all coexist, simply accessing the system via the new API.

This new track of development got me thinking.  I had produced APIs in the past, non-web based APIs (using such legacy technologies as COM+ among others), and web based APIs, one using a SignalR interface performing a large amount of business logic and data persistence.  However, none of these APIs had been produced entirely as a public API for consumption by 3rd parties.  All had a target consuming application, be it a cooperating 3rd party (with the potential to resale to other customers) or an internal partner dev team in the case of the SignalR interfaced system.  These systems were developed as APIs as a way of separating concerns,  allowing the multiple teams to develop in isolation, concentrating on their own area of expertise, with an agreed interface.  The key advantage of an API.

So having not really thought in depth about the creation of an API for general public use, equally I had not thought about this from the other side, consuming a publicly accessible API produced by someone else who I have no professional relationship with.



I have for some time been toying with the idea of producing a mobile app, but have not yet come up with an idea of something I want to create for the public to see.  I am of the opinion that creating something rather than nothing is a better idea, be it imperfect or not, I should simply create something and release it, and that is what I will do.  This is a way of thinking that is a little new to me, previously I was of the opinion that if it was not a 'great' idea there was no point in trying it.  But John Sonmez's blog post on the subject of simply creating something regularly is better than sitting on an idea until a perfect idea comes along.

So where do I start? Years ago I thought that indie games would be the best approach, I still think this would be a good thing to do, potentially very profitable, but an approach that could take a lot of effort to get something out.  And more importantly something I have little experience in, particularly the design and aesthitics aspect.  If I want to create something fairly rapidly in my spare time (which is scarce) I need to find a short-cut.  This is where the use of APIs come in.

There are thousands of publicly accessible, freely accessible, APIS out there.  Web sites like https://www.mashape.com http://www.programmableweb.com/ and http://freeapi.net/ offer a range of APIs, and are just the tip of the iceberg for what is available.  Creating a mobile application that consumes one of these APIs should be fairly straightforward, and as I have no great ideas of my own, my plan is to look through the lists of readily available APIs, and try to come up with an idea of how to present its functionality in a new way within a mobile app.

This approach has the massive advantage that someone else has done all the hard work of creating the API, hopefully thought through all the logic and created something that will do what is claims to do.  So if I can present a good human interface to use such an API, then I will have a useful app, and something that the public may want to consume.

The Wrong Path?

I read an email post today regarding the benefits of doing something rather than nothing, even if it might be the wrong thing to do.  This post by John Sonmez uses the quote

Sometimes you can only discover the right path by taking the wrong path 50 times.

This struck a real chord with me on a number of levels.  I have a current desire to enter into the world of mobile apps, the problem here being that I need and idea for an app to create.  I have written at some length here about the use of genetic algorithms to solve difficult mathematical problems, an approach based entirely in exploring the 'wrong path'.  I started a SaaS project with some friends a while back that has stalled somewhat due to a number of reasons, not least a reluctance to expose the beta version to wider public scrutiny. And lastly, in my day job I am working on a public API to the companies core system.  This is a new venture, and we are finding ourselves galloping down the wrong path on a daily basis, and returning to try the left fork rather than the right very quickly.  It has been a very refreshing approach from a software house that is steeped in protracted design and analysis phases before development.  An approach that is producing some considerable discomfort in a majority of the dev team (testers included), but one that I am relishing.



A big part of the problem is that we are attempting to created an automated testing suite as we develop, a great idea in itself, however the testers who are defining the content of this test suite are struggling a little with the idea of the expectations of the system changing faster than the tests are being produced.

For example, when requesting the list of entities related to another entity using an http get request with the url /entityA/{id}/entityB to get all the instances of entityB which has an entityA with id = {id}, if there is no entityA with id ={id} the testers argue that the system should return an error, most likely a 404 not found error.  This assumption was backed up with the rationale that if the request /entityA/{id} was used then a 404 error would be returned.  In contrast if entityA with id = {id} does exist, but has no entityB children, then an empty list would be valid.

A compromise was made that for the example we looked at, it was not valid to have an entityA with no entityB children, so if an empty list was retrieved from the database, then a 404 would be returned.  The developers were happy, as we could simply return a 404 for 'routed' get all requests resulting in a zero length list, successful blanks lists for 'direct' get all requests (/entityB/) and 404 for a direct get single (/entityA/{id}) with no match.

This meant that no validation of {id} was required, so a single database query will give all the information we need.

A couple of days later this became more complex.  An example was found with entityC and entityD, where entityC could exist with no children of type entityD, so the request /entityC/{id}/entityD could produce no results for two situations, entityC with id = {id} may not exist, or it may exist with no children.  The code as it stood returned a successful result with a blank list in both circumstances.  The testers did not like this as we had made the decision to go this way with analysis of only the A,B case, without consideration of the C,D case.

The final decision of what we will do is yet to be made, however we have a system that supports the query of both A,B and C,D working right now.  If we had started analyzing all the possible combinations of entity types in the system (100s) before we decided on a way to go for any routed queries we would have nothing for a good month.

We may have taken the wrong path, the testers would say that we have, we may have chosen the right one, the architects are arguing that side based on reducing database hits to a minimum (checking if entityA with id = {id} would require another db call after we know that the full query returns nothing, and for more complex queries with multiple routings this may amount to a significant increase in DB traffic and therefore cost as its a cloud based DB).

Whether this will get us to Eldorado in the end only time will tell.  We are due to engage early with a consumer of the API, and their preference on what we return will be the deciding factor, but at least we are in a position very early to show them something.