Blog for The Well Designed URLs Initiative

I was planning to blog something else today, but Mark Nottingham of Yahoo made a statement about URL Design [1] in his post entitled REST Issues, Real and Imagined and I simply could not let his statement without comment.

But first let me say I always appreciate Mark’s perspective on issues, and enjoy reading his writings whether on his blog or in the mailing lists. His perspective is typically insightful and prescient, and he hovers above the hovers above the muck-racking and unsupported claims that can occur on mailing lists populated by egos. All in all his involvement is very professional, and I highly respect him for that,

That said, here is the comment me made that really bothered me (2nd and subsequent emphasis mine):

Red Herring: URI Design
When somebody first “gets” REST, they often spend an inordinate amount of time agonising over the exact design of the URIs in their application; take a look over on rest-discuss, for example. In the end, though, URI design is a mostly a cosmetic issue; sure, it’s evidence that you’ve thought about good resource modelling, and it makes things more human-intelligable, but it’s seldom worth spending so much time on it.

I’d worry a lot more about cacheability, extensibility and well-defined formats before blowing out my schedule on well-designed URIs. For me, the high points are broadly exploiting the hierarchy, allowing relative references, and making sure that tools (e.g., HTML forms) can work well with my URIs; everything else is gravy.

I’ll start by saying I don’t really disagree significantly with his overall points that I believe he was making, such as the fact that there are other aspects that deserve attention in addition to URL design and also the fact some people appear to thrash when designing URLs. But to someone who does not appreciate URL design his statement could be easily misconstrued to mean that URI Design is not at all important, especially when he says it is mostly a cosmetic issue. That is false.

URI design is NOT merely a cosmetic issues! It has many tangible ramifications and those who ignore it do so at their own peril. Just scratching the surface, proper URL planning and design provides a framework for good information architecture, can facilitate spontaneous inbound linking via blogs, voting, tagging, and other social media sites, and can guard against broken URLs and subsequent traffic loss, to name but a few. They are issues that concern, or should concern everyone who publishes a site on the world wide web.

But what’s worries me most are the people who will certain latch onto Mark’s words as not only justification for ignoring patterns and best practices but also for antagonistically preaching against URL Design. This group includes web and content management system developers who would prefer not to be bothered with usability issues, system administrators who believe in security by obscurity (which itself is a fool’s precaution), dogmatists who misinterpret the principle of URI opacity and preach that web publishers should publish completely opaque URLs,  opaque even to the web publishers themselves, and a tiny but vocal contingent that for reasons I cannot fathom argue against URI design even within totally unrelated conversations.  It is for this reason I think Mark’s statement is potentially very damaging.

And as for his comments about rest-discuss, it is quite possible he was referring to conversations in which I participated. If so, I believe he mistook the crux of the conversation on several levels. The first was that in many cases I was advocating URL design, not agonizing over it. Certainly, URI design is really not that hard, it just takes understanding core principles and best practices, and then applying them. And secondly, some people escalate conversations to raging debates when simple questions are asked about proper URL usage in context of REST and Web Architecture. Mark could easily yet wrongly have misinterpreted those as too much hand-ringing over URL design.

In summary I don’t really fault Mark’s comments as I appreciate them to be. And I think Mark does a great service for the web in his quest to educate people about the value of REST. But as Mark is justifiably well respected I’m worried Mark’s comments may be used to rationalize bad URL Design. As such, I definitely hope he updates his post to guard against his word’s misuse.

Footnotes

1. Mark prefers to use the term URI instead of URL whereas I obviously prefer the later when in the appropriate context. By definition, URLs are URIs with the difference being that a URL is dereferenceable. When discussing REST end-points, the term URL is applicable and the term URI which refers to non-derefenceable identifiers, is not. And don’t you forget it! ;-)

P.S. Oh, and I also couldn’t help but wonder if Mark was trying to get in a cheap dig when he wrote “…before blowing out my schedule on well-designed URIs“…?  But naaaaah, Mark’s a real professional and wouldn’t go for such an underhanded shot. ;-)

I got an email from the PayPal Developer Network today announcing PayPal’s new “NVP” (or “Name-Value Pair“) API. Clearly they’ve learned that the complexity of SOAP is counter productive to adoption. Here’s what the email had to say about their new API:

NVP Is Your Integration MVP
We’re proud to announce that PayPal’s Name-Value Pair API has launched. Complex SOAP structure is now gone. All API methods are supported, except for AddressVerify. Get exclusive sample code – download two SDKs (Java and .NET). Get Details

Taking a look at their examples (in .ASP, .PHP, or ColdFusion) and their SDKs (for Java and ASP.NET [v1.1]) it’s nice to see they are using POST instead of GET. The following is one of their functions from their PHP examples (CallerService.php) that illustrates how their code is calling their NVP API (I edited for line-length only):

function hash_call($methodName,$nvpStr)
{
   //declaring of global variables
   global $API_Endpoint,$version,$API_UserName,
          $API_Password,$API_Signature,$nvp_Header;

   //setting the curl parameters.
   $ch = curl_init();
   curl_setopt($ch, CURLOPT_URL,$API_Endpoint);
   curl_setopt($ch, CURLOPT_VERBOSE, 1);

   //turning off the server and peer verification
   //(TrustManager Concept).
   curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, FALSE);
   curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, FALSE);
   curl_setopt($ch, CURLOPT_RETURNTRANSFER,1);
   curl_setopt($ch, CURLOPT_POST, 1);

   //if USE_PROXY constant set to TRUE in Constants.php,
   //then only proxy will be enabled. if USE_PROXY constant
   //set to TRUE in Constants.php, then only proxy will be
   //enabled.
   //Set proxy name to PROXY_HOST and port number to
   // PROXY_PORT in constants.php
   if(USE_PROXY)
      curl_setopt ($ch,
                        CURLOPT_PROXY,
                        PROXY_HOST.”:”.PROXY_PORT);

   //NVPRequest for submitting to server
   $nvpreq= “METHOD=”.urlencode($methodName).
            “&VERSION=”.urlencode($version).
            “&PWD=”.urlencode($API_Password).
            “&USER=”.urlencode($API_UserName).
            “&SIGNATURE=”.urlencode($API_Signature).$nvpStr;

   //setting the nvpreq as POST FIELD to curl
   curl_setopt($ch,CURLOPT_POSTFIELDS,$nvpreq);

   //getting response from server
   $response = curl_exec($ch);

   //convrting NVPResponse to an Associative Array
   $nvpResArray=deformatNVP($response);
   $nvpReqArray=deformatNVP($nvpreq);
   $_SESSION[’nvpReqArray’]=$nvpReqArray;

   if (curl_errno($ch)) {
        // moving to display page to display curl errors
        $_SESSION[’curl_error_no’]=curl_errno($ch) ;
        $_SESSION[’curl_error_msg’]=curl_error($ch);
        $location = “APIError.php”;
        header(”Location: $location”);
    } else {
        //closing the curl
        curl_close($ch);
     }

return $nvpResArray;
}

Much nicer and simplier than having to go to all the effort to set up a SOAP call.

Unfortunately, PayPal missed a huge opportunity to make their new API fully RESTful. Instead of designing a URL-centric REST interface (with a hypermedia component to keep the purist or the pure RESTafarians happy) they instead tunneled method calls over HTTP!  They used methods like “DoDirectPayment” and “RefundTransaction” Sheesh! (Note: these links to their methods load slower than any website I can remember visiting in ages and while loading my browser does lots of clicking. What the heck is going on in there I have no idea! You can go to the main docs via a much faster downloading PDF here. Wow, if that isn’t usually an oxymoron!)

Though much easier than calling SOAP, clearly not very RESTful.  Three steps forward, and two steps back. Sigh…

Probably one of the most interesting projects related to URL Design on which anyone is currently working is the URI Templates project spearheaded by Joe Gregorio of IBM. While not sexy and not something most end users will ever see, infrastructure developers writing blogs, wikis, forums, content management systems, and ideally even web servers, proxies, and routers will be able to utilize URI Templates for configuration and more. Well, once URI Templates are finalized and made an IETF standard that is.

Examples uses could include URL Rewriting Rules, URL Virtualization Configuration, REST’s Hypermedia Requirement, within future HTTP Headers and HTML LINK Elements, and even for Sitemaps (in a hopeful revision to Sitemap.org’s Sitemap protocol.) URI Templates will provide Internet professionals far more flexibility in dealing with URLs such as making it possible to employ URL Construction in a controlled manner without violating that nastly little URI Opacity Axiom.

Unlike most URL Rewriting Tools, URI Templates are very easy to use and, with the possible exception of certain esoteric edge cases that are still being finalized, accessible to mere mortals. URI Templates are certainly nowhere near as difficult as the requirement to master regular expressions and their complex interations when using Apache’s mod_rewrite and IIS’ (3rd Party) ISAPI Rewrite. Almost certainly anyone who has managed to language their own website will have no trouble at all mastering the most common use-cases for URI Templates.

URI Templates use a simple syntax where braces denote variables to be replaced when the templates are converted to actual URLs. For example, if this blog were using URI Templates to configure this page, the URI Template would most certainly look like this:

http://blog.welldesignedurls.org/{year}/{month}/{day}/{post-slug}/

When the template variables have the following values (as they do for this post):

The resultant URL upon conversion will be:

http://blog.welldesignedurls.org/2007/01/03/introducing-uri-templates/

Pretty simple, huh?

However, be aware that before URI Templates can become an IETF standard there needs to be at least two interoperable implementations, if not a lot more. Mark Nottingham of Yahoo has implemented a URI Template parser in Javascript, but I don’t know for certain if that one will qualify for the purposes of standardization. Whatever the case, if you are working on any projects or products that could benefit from URI Templates I encourage you to participate in the URI working group via their mailing list and then implement the specification in your software projects or products.

Although it may appear to be a boring subject from the outside looking in, the standardization of URI Templates will be probably the most positive thing to happen for URL Design in over a decade.