Raw

Nothing new here...that's the problem :)

I posted this in a conversation with Nina Jeliazkova and Evan on G+, thought I'd put it here so I could find it again.

Let's say I was setting up an events service for musicians. Following +Evan Prodromou's ref, Portable Contacts would seem in scope for the musicians themselves. Events happen in a location, so one part of the API I'd be interested in is the address stuff. To work with that data it might be useful to use geonames too. It's events, so let me have the place stuff from eventful as well. Geo, geo and geo - with three completelydifferent APIs:
http://portablecontacts.net/draft-spec.html#rfc.section.7.4
http://www.geonames.org/export/web-services.html
http://api.eventful.com/docs/venues/search
The data may be exposed but it's there as a, er, kind of glass silo, it doesn't exactly lend itself to reuse.

Nina remarked:

Not that technically it is impossible to merge the APIs, there is no reason (business, whatever) for them to sit down and merge the APIs. This has happened in network engineering (and other domains) couple of decades ago; there have been many incompatible network protocol/hardware vendors then. It takes time to recognise the value of synchronisation.

She's right, but that time part is an issue. A few years ago everyone was talking about mashups - didn't the value become apparent then? We've had a good modelling language for sync'ing data since (say) 2004 when the RDF specs came out. The data-handling tooling came along with SPARQL in 2008. RESTful good practice ideas have spread widely in the past few years, with linked data I suppose being their counterpart in the semweb world. So why are APIs still so difficult?

Ok, that's glass-half-empty from a semweb perspective. Awareness of this tech has spread. The stuff around Rich Snippets, schema.org and HTML5 microdata demonstrate that the ideas are reaching a wider audience. (Incidentally I was impressed by JeniT's diplomacy about HTML5 in her excellent presentation - but I'm going to start referring to the stuff as HubrisML :)

A personal data point: last week I checked my Twitter "followers" for the first time in maybe 6 months. Around 150 new people. I'd estimate that 100 of them had reference to the Semantic Web (or some closely associated tech) in their profiles. I follow this tech, but still I hardly recognised any of these new folks.

I suspect Mike Amundsen might have a point when he says RDF will languish until it goes hyper (i.e. gain affordances as a hypermedia type). JeniT's talk of using HTML/XML/JSON/RDF for what it's best at probably applies - so how do you bring interactivity to RDF without it looking like it's got a goat's head stuck on it's back? Research needed (high on my list). Whatever, pragmatically the linked data API goes a long way.

Anyhow (once I get my bank balance back in the black) I intend to put a lot more effort into actually using this tech to build human-facing apps. I've a few ideas on how to operate as an Indie, a core one being that taking full advantage of what the Web has to offer (i.e. using linked data etc) offers a business advantage, everything else being equal.

(any comments to the thread on G+ please)

A wee rant.

Ok, I'm totally with the consensus that the future is Cloud-based, and to be a little more specific Platform-based and to be even more specific primarily HTTP-based. To back that up, cf.

• Michael Hausenblas's new report
• Mike Amundsen's recent blog post -
• Steve Yegge's awesome rant at Google

But to expand something I mentioned in passing here recently :

in one respect the emperor is stark-bollock naked. Browsers are currently a really sucky environment for client development. Sure, the HTML/CSS-based (standard!) rendering is wonderful. As shown with Node.js (and despite what Google are saying around Dart), Javascript is a reasonably pleasant, perfectly capable programming language. The growth of Ajax and JSON have shown inter-system comms is workable. There are some good dev tools and libraries. So why does working with this stuff feel like pulling your own teeth?

Here I could point to the traditional DOM API, blame the W3C for all the world's ills and an awful lot of people would nod and smile knowingly. But although that's arguably valid (heh), I reckon the problem is more systematic and can mostly be blamed on browser developers.

Ok, blame is too strong. The decisions made over the years and the directions taken have generally been perfectly rational in the context of the prevailing conditions. But there have been feedback loops at work. The flashy [sic] chrome [sic] surrounding HTML dev, from the img tag onwards, has pulled Web developers in like moths around a flame. So the browser developers act to improve that experience. Meanwhile server-side tech has developed out of the corporate legacy of silo-based systems. Let me quote Steve Yegge there: "It's a big stretch even to get most teams to offer a stubby service to get programmatic access to their data and computations.". The way services are offered over the Web, even Web 2.0 services still have a big hangover from this mentality. I'd argue that most Web APIs are only marginally better than SOAPy stubs. Largely because XML and JSON aren't particularly Web-friendly. Ok, don't bite my head off, let me qualify that.

First XML. There have been plenty of arguments over the years around XHML, and back in the day (I wonder how old that phrase is) there were arguments about the XML nature of RSS. Postel's Law, the "Robustness Principle" got cited a lot. Let me give you some deja vu:

Be liberal in what you accept, and conservative in what you send.

What a lot of people misinterpreted was the keyword robust. A robust system is one designed to be able to fail gracefully or continue working acceptably with noisy data. That's exactly what we want for the Web, right? Well not necessarily, if I was ordering a book from Amazon, and there was a partial failure, I'd rather they didn't make a best-guess when it came to taking money of my credit card (I think paraphrasing Tim Bray there). Anyhow, XML is not robust, by design. XML is designed to bail out completely at the first sniff of anything dodgy. As it happens, the way XML is often served on the Web is without proper regard for the media type, i.e. dodgy and hence broken.

Sorry, that was gratuitous deviation, the real reason I'd say XML isn't Web friendly, like JSON, is in the way people use it. Whether data is conveyed as name-value pairs or through more complex structures, the key parts are generally just simple strings. But by itself, a string on the Web is next to useless. You or I can (maybe) read it, or even paste it into Google and get a definition. But what is a poor machine client to do? What makes the Web are links. It's 101 but somehow still manages to be overlooked: the link has two facets: a universally unambiguous name (URI/IRI) and a protocol for following it (HTTP). If a client on the Web encounters a link, it can follow its nose to find out more information about it. That's what we as humans do in browsers all the time, yet when it comes to Web services for some reason a simple string is seen as adequate to identify something.

Ok, with XML, the HTML DOM and to some extent JSON there's been some justifiable resistance to the use of URIs for names, because namespaces have traditionally been uninuitive at best and agony at worst. Using URIs instead of simple strings certainly adds a burden (it doesn't have to be that great, check Turtle syntax), but its benefits far outweigh the costs.

The thing is, you'll hear talk of snowflake APIs - only one implementation of each exists - but what gets overlooked is that by their very nature, most APIs just aren't Webby. The client must have prior knowledge that the service at endpoint X uses API Y. What you end up with is effectively a series of 1:1 client-server connections. That, while the uniform interface REST may mean it's less brittle than an RPC connection, still means tight coupling.

Ok, you might argue, that for any communication to take place, some prior knowledge is required. Sure, but that can be minimised - just like the way we follow links for more information in a browser, a service client can follow links to get more information. This is only a small conceptual step, but what it enables is hugely powerful. Above everything else, it's what Linked Data and the Semantic Web gets right.

I reckon that browser developers, with their emphasis on doc-oriented HTML have a natural tendency to carry their experience in that domain across and apply it to data. Naturally namespace-less XML and JSON will seem preferable through that lens. But in practice, documents and data are apples and oranges. Browsers have been optimized over the years for the former, incidentally making the latter harder than necessary.

It's funny how you don't hear so much about service mashups these days, despite their undeniable coolness. I'll assert that it's because developing for Web data in the browser is bloody hard work, especially when there are NxN arbitrary API mappings to know.

Overall it's actually something of a miracle that the notion of cloud-based platforms has emerged.

I had planned to say more about Cloud Computing Outside of the Browser - or to put it another way, evolving old-fashioned non-browser Rich Internet Clients (as well as server-server and every other non-browser configuration). But ranting's worn me out. Anyhow, in short, I reckon that for the forseeable future, non-browser clients in many circumstance are probably preferable to browser-based equivalents, primarily because they're easier to develop (as I keep saying, I reckon the agent model of combined client/server units is a good way to go). While I personally welcome HTML5 and the APIs as a clean-up of document markup and processing, when it comes to data it isn't even a Band-Aid.

Some interesting survey results have just been published about APIs: the good, the bad and the pains. I commented about this on G+ and the discussion there got on to Atom. Some interesting points made, including the likelihood that we're stuck with snowflake APIs (every one is different) for the foreseeable future. I think it was Bill de hÓra who had a post years ago (can't find it now) about the N x N problem of diverse APIs (/models/formats). Essentially if you've got N different APIs then to connect them all you need N x N different translators. But it's also worth noting that this can be reduced to 2 x N if you have mappings to a common format/model. I reckon recent history has shown that formats are secondary, assuming certain boxes are ticked (see below). Regarding the model - there is a well-known, Web-friendly one. So here I'll simply point to ConverterToRDF and ConverterFromRDF.

In the G+ discussion Bill referred to an old blog post of his, Magnificent Seven - the value of Atom. In it he highlights the 7 'primitives' that Atom (format and protocol) uses and that he suggests should be used in any carrier format. I'm inclined to agree, if you are creating an API, tick these boxes, repeated here without Atom-specificity:

1. ID - a globally unique identifier for the chunk of data, ideally this should be a HTTP URL
2. Link - as above, it's rare that a separate ID and URL are needed
3. Updated - the most recent change, invaluable for keeping things in sync
4. Extension rules (mustIgnore, foreign markup) - anything the parser doesn't understand, it simply ignores. This allows other people to reuse and extend the format in a compatible fashion.
5. Date construct rules - using a standard date format is basic politeness
6. Content encoding rules - generally follow the rules for the media type you're using, and if there's textual content use an existing standard format (XHTML is good). Rule of thumb: UTF-8.
7. Unordered elements - insisting on order in the structure is (or at least should be) unnecessary, accessing things by name is more reliable