Archive for the ‘Standards’ Category

Proposing HTTP Request Forwarding

Thursday, March 8th, 2007

I’ve been monitoring the ietf-http-wg mailing list and have noticed there is renewed actively around revising RFC 2616, the HTTP/1.1 specification. This renewed activity got me thinking it was time to discuss the need for HTTP Request Forwarding.

I’ll start by saying it is possible that Request Forwarding is already part of the HTTP spec and that I just overlooked it. Lord knows I’ve read enough W3C and IETF specs lately to raze a small forest if printed, so I could easily have missed that part as I read bleary-eyed through the specs. But I’ve asked the question privately of a few people that should know and they all said that HTTP does not support request forwarding; one of them pointed out that that is why VOIP needed SIP.

If the term “HTTP Request Forwarding” isn’t obvious from context, let me give a use-case to illustrate:

USE-CASE: Retaining control of URL Interfaces when outsourcing image hosting

  1. http://example.com
  2. http://image-servers-r-us.net
  3. http://example.com/index.html
  4. http://example.com/images/splash.png
  5. http://image-servers-r-us.net/example.com/splash.png

Assume client “A”, server “B” mapped to [1] and server “C” mapped to [2] (servers “B” and “C” are different computers with different IP addresses probably at different locations.) For this use case client “A” requests the HTML file at [3] which includes an <img> tag pointing to a graphic at [4].

The request from client “A” for the .html file at [3] goes to server “B” and the response is returne to client “A.” After parsing [3] client “A” realizes it also needs to download the image at [4] and requests the .png file from server “B” at [1].

HTTP Request Forwarding (Note use of HTTP 1.2 to make URL in example explicit)

However, server “B” knows that the .png file [3] is actually located on server “C” at [5] so it forwards the request to server “C” at [2]. Server “C” responds by returning the .png file directly back to client “A” and client “A” is none the wiser; i.e. client “A” still thinks that the image was returned by [4].

In addition, if client “A” is a browser and the user inspects the properties of the image, the user would find the URL to be [4], not [5]. No where in the response given to the client would there be information that the image came from [5] instead of [4] except that the IP address differs from the server that requested it. It is possible that a header could contain the forwarding information, but the client would not need to act on it except potentially for debugging.

There are several other use-cases as well, such as making it possible to fully virtualize a domain authority’s URL interface. However, the use-case of outsourcing images or any large static content illustrates a benefit that I believe should be obvious to most people. And for those that are interested, especially if you are involved in updating the spec, I could detail other use-cases if people are interested.

Now it is possible there are security implications with this that I have not considered. If so, I would hope that we could at least explore potential ways to mitigate those issues as opposed to immediately dismissing HTTP Request Forwarding out of hand.

In summary, adding a Request Forwarding functionality to HTTP would allow servers to maintain control of their URL interfaces while still being able to distribute loads and services to appropriate servers. Having such a feature could improve consistency in URL design and make it easier for website owners to restructure their websites without the level of broken links seen on the web today.

Interestingly, on Tuesday Scott Hanselman blogged about just this problem from a slightly different perspective; I present Scott’s post for your consideration.

Bitten by the URI Opacity Axiom

Thursday, January 25th, 2007

 

Jon Udel has a post today entitled Divergent citation-indexing paths. Funny that he wrote about this; it seems he and I are on such a parallel trajectory these days. For evidence, take a look at my post from last week I titled Lessons Learned from Delicious Praise.

In his post Jon states “Del.icio.us, unlike Bloglines, treats the URLs that you feed to its citation counter in a case-sensitive way.” I wonder if Jon is aware of the Tim Berners-Lee’s URI Opacity Axiom?  A correct reading of the URI Opacity Axiom would reveal that in that Bloglines is in the wrong and Del.icio.us is in the right in this case; i.e. URLs are case-sensitive and programs SHOULD NOT[1] infer that two URLs with different casing point to the same resource. (NOTE: case doesn’t matter with domain names but it DOES matter in URL paths.)

As a matter of fact the URI Opacity Axiom is such as a closely held belief among those I like to call “the Weborati” that if you even question it so as to understand it on certain W3C or related mailing lists you’ll be in for a firestorm as if you blasphemed the messiah! ;-0

All kidding aside, especially since some of those people who hold the URI Opacity Axiom dear read this blog (!), after spending the time to research it and really learn it I came to believe that it is a very good idea for people to follow the URI Opacity Axiom. And I’ll discuss why in the future when I have more time. Unfortunately, like many principled concepts some people have elevated the URI Opacity Axiom to the level of dogma, and many of those who preach it believe it means a distortion of what it really means.

So Jon identified a real-world problem that following the URI Opacity Axiom introduces yet it is somewhat of a “catch-22“; following the axiom creates real-world problems but not following it creates other real world problems. But longer term, I really don’t think it has to be this way, and I’m working on ideas to address this issue that may turn into draft proposals or recommendations or something else. Basically I think with some “layers” of technology added to the web we could have the best of both worlds.

As an aside, Del.icio.us could update links based upon 301 redirects and then website owners could 301 redirect to a(n ideally lowercased) canonical URL whenever their server receives a request for a URL that is not in the canonical URLs format. This assumes of course that the website owner/server operator has chosen for their URLs to effectively be case-insensitive[2].

  1. I use the “SHOULD NOT” in the same way RFC 2119 defines the use of the uppercased term.  
  2. In my opinion running a website with case-sensitive URLs either means the web developer just wasn’t thinking or that they don’t have a clue about the affect case-sensitive URLs have on website usability. Ah, but that’s another subject for another day. :)

Proposing URI Templates for WebForms 2.0

Thursday, January 11th, 2007

I recently had an off-list email conversation with Ian Hickson, the editor of the Web Application Hypertext Technology Working Group specifications (i.e. HTML5 and WebForms 2.0). I was proposing to him that the current WebForms 2.0 be draft specification be amended to include a URI Template in the “action” attribute of the FORM element. Because I believe so strongly in the benefit of this proposal and because such things are inline with the Well Designed URLs Initiatitve was envisioned to advocate for, I decided to publish it to our blog and reference it in the WHATWG blog. The following is what I sent to Ian in email:

I really want to see WHATWG incorporate URI Templates for Web Form actions[1]. i.e.:

<form
action="http://foo.com/{make}/{model}/”
method="get">
<input type="text" name="make" />
<input type="text" name="mode" />
<input type="submit" />
</form>

If I type “Honda” and “Civic”, it will do a get to:

http://foo.com/Honda/Civic/

Instead of the only current possibility being something like:

<form
method="get"
action="http://foo.com/cars.php">
<input type="text" name="make" />
<input type="text" name="mode" />
<input type="submit" />
</form>

Which would produce the following for “Honda” and “Civic”:

http://foo.com/cars.php?make=Honda&model=Civic

To which Ian replied in two parts. Here is the first part:

“Why not just write a server-side redirector? That’s a trivial one to write. Four lines of code, maybe 10 if you make the recommended security checks first. You could also do it with a little bit of JavaScript.”

Unfortunately, a server-side redirector is not an appropriate solution in one case for the use-cases this proposal would address and doesn’t work for two others:

  1. Server-side redirection requires two round trips instead of one. I don’t believe any reasonably competent web architect for a high traffic site would allow a redirect for a high-traffic site. Consider any search engine such as the flagship offering for Ian’s employer; is Google likely to implement a redirect on every search request? Not likely. Server-side redirection reduces response time (by half?) and increases (doubles?) the number of concurrent requests that servers must handle. Using server-side redirection would probably also increase bandwidth requirements a measurable amount.
  2. It is not possible via HTTP to redirect the body of a POST. Consequently, the following use-case cannot be duplicated with a server-side redirect:

    <form
    action="http://www.myblog.com/{topic}/”
    method=”post”>
    <select name=”topic”>
    <option value=”first”>My 1st Post</option>
    <option value=”second”>My 2nd Post</option>
    <option value=”third”>My 3d Post</option>
    </select>
    <input type=”text” name=”comment”>
    <input type=”submit”>
    </form>

  3. For those wanting to create a form to direct to a website they do not control, a server-side redirect is absolutely not an option. For example, assume that I wanted to run a page on my website that lets people navigate to the topics on the WHATWG blog using a FORM with a SELECT? It’s simply not possible as WebForms 2.0 is currently specified without Javascript (addressed in a moment), but would be easily possible using a template (note I left off the closing “</option>” tags for formatting reasons):

    <form
    action="http://blog.whatwg.org/{topic}"
    method="post">
    <select name="topic">
    <option value="feed-autodiscovery">
    Feed Autodiscovery
    </option>
    <option
    value="text-content-checking">
    textContent Checking
    </option>
    <option value="checker-bug-fixes">
    Bug Fixes
    </option>
    <option
    value="significant-inline-checking">
    Significant Inline Checking
    </option>
    <option value="charmod-norm-checking">
    Charmod Norm Checking
    </option>
    <option
    value="proposing-features">
    Proposing features
    </option>
    </select>
    <input type="submit">
    </form>

  4. My belief is having this capability would encourage a lot more linking of this type between pages on the web.

So yes, server-side redirection is possible in some cases but by no means all, and for those cases where it’s possible it is not optimal.

Moving on the Ian’s suggestion to use “a little bit of JavaScript” to meet this use-case, I will admit it is possible to use JavaScript but these are the drawbacks in viewing JavaScript as the solution for this use-case:

  1. JavaScript is simply not allowed in a wide variety of web applications such as wikis, forums, and other sites that solicit community content although many of these do allow HTML elements such as FORM.
  2. Some users disable scripting on certain sites, often by decree of their employers.
  3. Javascript’s cross-browser compatibility issues make it less reliable and people are far less likely to depend on it when money is at stake, and forms are frequently used in those contexts.
  4. “User agents with no scripting support” from Section 1.3 Conformance Requirements of the Web Applications 1.0 Working Draft (HTML5) that incorporates WebForms 2.0. Need I say more?
  5. Declarative code is far easier to debug than procedural code.
  6. Far more people know HTML than JavasScript given the much greater skill required to master the latter, and that is unlikely to change.

It’s interesting to note that in the preface to the introduction for Section 3 of the WebForms 2.0 Working Draft of 12 October 2006, the following note is made about how everything that repeating form controls offers can already be done in JavaScript and the DOM. The mere fact that they went to the trouble to include something as complex as repeating controls into HTML5 when it can be done with JavaScript and the DOM implies that well-known patterns in web architecture are better implemented declaratively instead of via JavaScript and the DOM:

Occasionally forms contain repeating sections. For example, an order form could have one row per item, with product, quantity, and subtotal controls. The repeating form controls model defines how such a form can be described without resorting to scripting.

Note: The entire model can be emulated purely using JavaScript and the DOM. With such a library, this model could be used and down-level clients could be supported before user agents implemented it ubiquitously. Creating such a library is left as an exercise for the reader.

So yes it is possible to use JavaScript in many cases, but it no where near optimal. Javascript should not be considered the solution for as well-defined and obvious patterns such as submitting to a clean URL.

To further drive home the value of this proposal, anyone monitoring the REST-discuss list for any length of time will see that most REST experts tend toward using (what I call :) well-designed URLs, i.e. URLs where the resource is identified by path instead of query string. With WebForm 2.0’s pending support of PUT and DELETE, it would be just short of a crime not to include support for posting to clean URLs in WebForms 2.0.

Since having this discussion with Ian via email it was since pointed out to me on rest-discuss by Mark Baker that my proposal as written would break the existing web so was a non-starter. For some reason I wasn’t thinking about that, probably because I was more concerned about getting Ian (who I like to call: Mr. “No :) to agree that URI Templates were needed. Still, the solution would be simple.

What follows are my examples from above recast using an optional template attribute that would override the action attribute for WebForms 2.0 compliant browsers. This would of course require the server to accept both query string parameters and clean URLs (and hopefully do a server redirect from the former to the latter), or the submit could be implemented using Javascript for older browsers when applicable. Note that I didn’t show an example using JavaScript but, as the WebForm 2.0 spec says(that) is left as an exercise for the reader”:

  1. <form
    action="http://foo.com/model"
    template=”http://foo.com/{make}/{model}/”
    method=”get”>
    <input type=”text” name=”make” />
    <input type=”text” name=”mode” />
    <input type=”submit” />
    </form>

  2. <form
    action="http://www.myblog.com/topic"
    template=”http://www.myblog.com/{topic}/”
    method=”post”>
    <select name=”topic”>
    <option value=”first”>My 1st Post</option>
    <option value=”second”>My 2nd Post</option>
    <option value=”third”>My 3d Post</option>
    </select>
    <input type=”text” name=”comment”>
    <input type=”submit”>
    </form>

  3. <form
    action="http://blog.whatwg.org/topic"
    template=”http://blog.whatwg.org/{topic}”
    method="post">
    <select name="topic">
    <option value="feed-autodiscovery">
    Feed Autodiscovery
    </option>
    <option
    value="text-content-checking">
    textContent Checking
    </option>
    <option value="checker-bug-fixes">
    Bug Fixes
    </option>
    <option
    value="significant-inline-checking">
    Significant Inline Checking
    </option>
    <option value="charmod-norm-checking">
    Charmod Norm Checking
    </option>
    <option
    value="proposing-features">
    Proposing features
    </option>
    </select>
    <input type="submit">
    </form>

So in summary I really hope that Ian, who definitely seems to be the gatekeeper for what goes into HTML5 and what doesn’t go into HTML5, can see his way clear to add this feature to WebForms 2.0. If his main issue with it is needing to have it written up for inclusion in the spec, I’m more than happy to help.

About URI Templates

Wednesday, January 3rd, 2007

Probably one of the most interesting projects related to URL Design on which anyone is currently working is the URI Templates project spearheaded by Joe Gregorio of IBM. While not sexy and not something most end users will ever see, infrastructure developers writing blogs, wikis, forums, content management systems, and ideally even web servers, proxies, and routers will be able to utilize URI Templates for configuration and more. Well, once URI Templates are finalized and made an IETF standard that is.

Examples uses could include URL Rewriting Rules, URL Virtualization Configuration, REST’s Hypermedia Requirement, within future HTTP Headers and HTML LINK Elements, and even for Sitemaps (in a hopeful revision to Sitemap.org’s Sitemap protocol.) URI Templates will provide Internet professionals far more flexibility in dealing with URLs such as making it possible to employ URL Construction in a controlled manner without violating that nastly little URI Opacity Axiom.

Unlike most URL Rewriting Tools, URI Templates are very easy to use and, with the possible exception of certain esoteric edge cases that are still being finalized, accessible to mere mortals. URI Templates are certainly nowhere near as difficult as the requirement to master regular expressions and their complex interations when using Apache’s mod_rewrite and IIS’ (3rd Party) ISAPI Rewrite. Almost certainly anyone who has managed to language their own website will have no trouble at all mastering the most common use-cases for URI Templates.

URI Templates use a simple syntax where braces denote variables to be replaced when the templates are converted to actual URLs. For example, if this blog were using URI Templates to configure this page, the URI Template would most certainly look like this:

http://blog.welldesignedurls.org/{year}/{month}/{day}/{post-slug}/

When the template variables have the following values (as they do for this post):

year: 2007
month: 01
day: 03
post-slug: introducing-uri-templates

The resultant URL upon conversion will be:

http://blog.welldesignedurls.org/2007/01/03/introducing-uri-templates/

Pretty simple, huh?

However, be aware that before URI Templates can become an IETF standard there needs to be at least two interoperable implementations, if not a lot more. Mark Nottingham of Yahoo has implemented a URI Template parser in Javascript, but I don’t know for certain if that one will qualify for the purposes of standardization. Whatever the case, if you are working on any projects or products that could benefit from URI Templates I encourage you to participate in the URI working group via their mailing list and then implement the specification in your software projects or products.

Although it may appear to be a boring subject from the outside looking in, the standardization of URI Templates will be probably the most positive thing to happen for URL Design in over a decade.