Blog for The Well Designed URLs Initiative

Throughout history there have been people who only saw things as they wanted them to be. People with strongly held beliefs whose values guided their actions be they counter-productive, detrimental, or worse just plain wrong; nothing mattered but to believe the world was as they wanted it to be. And it’s not just those from the past that are guilty; nay it seems that people the world over are now more ideological than any period in my own lifetime. I’ve give many examples, but whatever examples I gave I’d be sure to offend over half of my readers!

It’s probably pretty obvious that I’m talking mostly about war and religion in the above but it’s also sad to see the same from technologists. Case in point: the creators of DabbleDB and the Seaside Framework. There is an unfortunate school of thought among some web developers that Avi Bryant evidently shares[1] that clean URLs are simply not important, that they are just the obsession of overly pendantic developers pursing unimportant elegance. And those opinions are often rationalized by statements like these (on Mike Pence’s blog) that clearly exhibit confirmation bias:

“I have not had one single person ever mention in Dabble DB that the URL’s look funny. People are used to it. If you go to Amazon.com, the URL’s have all kinds of opaque identifiers in them. It is just not something that the average user cares about. I think it becomes an obsession for developers to have this sense of having a clean API exposed by their web application, but I think you can have a clean API that does not have to include every single page in your app, and I don’t think that every single page in your app has to be bookmarkable. I think that as long as a bookmark gets you back, roughly, to where you wanted to be, or for really crucial things to have permalinks, then you are fine.”

Well I guess Ari can’t say that anymore (that he hasn’t had one single person complain about DabbleDB’s URLs.) 

That said, why would Ari believe URLs to be unimportant anyway?  There is significant evidence all over the web that URLs are important, not the least of which has been document on this blog in the past. As best I can tell Ari’s regrettable belief occurs because of his desire to be unburdened from dealing with web architecture so that he can hoist highly stateful web apps onto an unknowning and unsuspecting public simply because that’s what Ari values. Basically Ari chooses to ignore the importance of URL design for both users and good web architecture and have his framework emit simply awful URLs simply because doing so makes coding and using his server-side framework so much easier. That’s similar to someone not addressing the unfortunate necessity of security simply because dealing with security is a PITA. (BTW, Amazon’s URLs are some of the worst and they only get away with it because of their early momentum. They are NOT a good example to emulate.)

So you see DabbleDB exhibits some very clear examples of really bad URLs. To see for myself I created a free trial account over at DabbleDB, which gave me my own well-designed URL (itself, not bad):

http://welldesignedurls.dabbledb.com/

Next I created an application called “Sites” and a first “category” that I named “Domains” (evidently in DabbleDB parlance a “category” is like a table to us relational database types.)  This gave me the following URL:

http://welldesignedurls.dabbledb.com/dabble/sites?view=2&_k=ZEiTkHyn

Not bad, but the “/dabble/” is unnecessary, the “view” could have been defaulted, and the “_k” is, well, is so gratuitous I doubt I need even further criticize it.  Clearly what I would have preferred to see is this:

http://welldesignedurls.dabbledb.com/sites/

Or at least:

http://welldesignedurls.dabbledb.com/apps/sites/

And I believe anyone would be hard pressed to explain why the actual URL DabbleDB uses is better or why the URL I proposed would not be workable. Still, all is not so bad to this point because it appears DabbleDB will respond appropriately to:

http://welldesignedurls.dabbledb.com/dabble/sites

Of course anyone bookmarking the URL vs. composing the URL for a blog will be linking to two different URLs as per web architecture, which has its own perils for the owner of the website. (I’m of course assuming public URLs for this use-case which is possible via DabbleDB Commons, itself having a great URL of http://dabbledb.com/commons, but many usability problem still exist in closed environments how most DabbleDB databases will be used.)

But matters get much worse when we drill down into the “Domains” category I created. Compare the following two URLs and guess which one I envisioned vs. the one Dabble generated:

http://welldesignedurls.dabbledb.com/dabble/sites?view=2&_k=qPDotnwm

http://welldesignedurls.dabbledb.com/sites/domains/

And if we click on the name of the domain “welldesignedurls.org“, if gets even worse:

http://welldesignedurls.dabbledb.com/dabble/sites?entry=7&view=2&_k=jGMmkZyZ#objectEditor

Again, I would have liked to have seen:

http://welldesignedurls.dabbledb.com/sites/domains/welldesignedurls.org/

Why is this important?  Because cognition of the meaning of the URL is used in a significant number of contexts by humans, often in the context of where only recognition (vs. URL construction) is important. In email, in the browser history list, in older bookmarks, on printed communication, and more.  By analogy imagine how much harder computers would be to use if users had no choice but to always navigate the tree structure of a deeply nested directory instead of simply copying and pasting the path from, for example, Windows’ Explorer or the Mac’s Finder to an Open File dialog[2]. Just imagine what it would be like if a path to the user’s directory was named “C:\%GSkstyrWshs\@9KBHasklp\” Ye-Gads!)

There are still further cases where clean URL design is important. For bloggers composing their links having the ability to learn a link structure rather than having to navigate to each page they want to link (such as on Wikipedia) is invaluable. For marketers wanted to convey a location in advertising for their customers and prospects to visit. And especially for users of web that are heavily data-oriented where users are involved in editing, navigating to, and communicating various application states (a.k.a. web pages) to their colleagues, such as an app like DabbleDB. If ever there was a category of web apps where good clean URL design is critical, it would be online databases!

So NO Ari, URL Design IS important. I hope you can learn this and make changes to DabbleDB and Seaside before it’s too late for you, and worse, for your users.

Footnotes

• 1.) How ironic Avi would name his blog “HREF Considered Harmful” as HREFs are truly one of the core foundations of web architecture.
• 2.) Yes I know that some people don’t ever ccpy and paste paths but many of the more intelligent and/or aware users do.

“…one should take an hour or so and really think about their website’s ‘public face.’”

-Scott Hanselman on “A Website’s Public Face“

“A URL is like a big “YOU ARE HERE” sign for each page of your site. It should allow people to get a sense of where they are in your site, even if they decide not to use that information for navigation.”

-Keith Devins on URL Design

This is quiz #2 of our ongoing URLQuiz series.

In this quiz, there are 26 pairs of URLs (A..Z) and for each pair the questions is: “Which of these two URLs are equivlent?” i.e. which return the same resource when dereferenced, and “Which can be cached as the same URL?”

You should answer ‘Yes‘, ‘No‘, or ‘Maybe‘ where ‘Maybe‘ means ‘It might return the same resource but should not be cached according to the specs.’

To answer, leave a comment and ideally explain your reasoning for each. Feel free to group answers based on your reasoning and/or the answer given (Yes, No, and Maybe.) Print it out and take the quiz with pencil and paper if you serious about getting it right, and feel free to use a computer or browser or whatever to test your results before answering. Good luck!

About the TLD .foo[1]

Clarification (2007-March-02): Some people have stated that the server could possibly return the same resource for any two given URLs so they felt the answer could never be ‘No.’ I definitely see their point, for example http://mysite.foo/bar and http://mysite.foo/bazz could possibly return the same thing but nobody would ever reasonably expect them to do so on their on. So let me clarify to say that I meant a quiz taker to select ‘No‘ in the case where where the resource returned would definitely not be the same thing unless the developer or server admin explicity programmed or configured them to do so. On the other hand ‘Maybe‘ would be used in the case where someone might reasonably expect the two URLs to return the same resource even though the RFC 3986 would define the two URLs as being different such as in [footnote 2], or when it depends on the O/S of the server as in [footnote 3]. Regarding fragments, the question is “In a transaction between a client and a server, is the cache allowed to view them as the same?” Regarding which can be cached I was looking for what are appropriate per the spec, not necessarily whether any particular software in the cloud (i.e. routers, proxies, browsers, etc.) actually does cache but instead “Would it be allowed to cache?”

Questions

1. The ‘www’ domain
   1. http://mysite.foo/
   2. http://www.mysite.foo/
2. Letter casing in path
   1. http://mysite.foo/Index.htm
   2. http://mysite.foo/index.htm
3. Letter casing in domain
   1. http://MySite.foo/index.htm
   2. http://mysite.foo/index.htm
4. Index.htm vs. Default.aspx
   1. http://mysite.foo/Index.htm
   2. http://mysite.foo/Default.aspx
5. Trailing slash on domain
   1. http://mysite.foo
   2. http://mysite.foo/
6. Trailing slash on path
   1. http://mysite.foo/path
   2. http://mysite.foo/path/
7. Empty question mark
   1. http://mysite.foo/
   2. http://mysite.foo/?
8. Empty parameter
   1. http://mysite.foo/?
   2. http://mysite.foo/?param=
9. Port 80
   1. http://mysite.foo/
   2. http://mysite.foo:80/
10. Port 443
    1. http://mysite.foo/
    2. http://mysite.foo:443/
11. Https vs. Port 443
    1. https://mysite.foo/
    2. http://mysite.foo:443/
12. Ftp vs. Http
    1. ftp://mysite.foo/
    2. http://mysite.foo/
13. Letter casing in parameter name
    1. http://mysite.foo/?param=bar
    2. http://mysite.foo/?Param=bar
14. Letter casing in parameter value
    1. http://mysite.foo/?param=bar
    2. http://mysite.foo/?param=Bar
15. Hash vs. no hash
    1. http://mysite.foo
    2. http://mysite.foo#
16. Hash vs. Fragment
    1. http://mysite.foo#frag
    2. http://mysite.foo#
17. Fragment vs. no Fragment
    1. http://mysite.foo#frag
    2. http://mysite.foo
18. Plus vs. Space in path
    1. http://mysite.foo/url+design
    2. http://mysite.foo/url design
19. Space vs. Encoded Space in path
    1. http://mysite.foo/url design
    2. http://mysite.foo/url%20design
20. Plus vs. Encoded Plus in path
    1. http://mysite.foo/url+design
    2. http://mysite.foo/url%2Bdesign
21. Slash vs. Encoded Slash in path
    1. http://mysite.foo/top/second
    2. http://mysite.foo/top%2Fsecond
22. Ampersand vs. Encoded Ampersand in path
    1. http://mysite.foo/abc&xyz
    2. http://mysite.foo/abc%26xyz
23. Ampersand vs. Encoded Ampersand in parameter value
    1. http://mysite.foo/?q=abc&xyz
    2. http://mysite.foo/?q=abc%26xyz
24. Equals vs. Encoded Equals in path
    1. http://mysite.foo/abc=xyz/
    2. http://mysite.foo/abc%3Dyxz/
25. Equals vs. Encoded Equals in parameter value
    1. http://mysite.foo/?q=abc=xyz
    2. http://mysite.foo/?q=abc%3Dyxz
26. Parameter order
    1. http://mysite.foo/?abc=123&xyz=987
    2. http://mysite.foo/?xyz=987&abc=123

P.S. Don’t stress if you can’t answer them all. It took me months to uncover all these nuances, and if I were taking this quiz I doubt I could get them right all in one sitting.

FootNotes

1. I’m using the non-existent top-level domain “.foo” to avoid giving any link-love to arbitrary example sites that don’t deserve it! For the purpose of the quiz, just assume that “.foo” is a functioning top level domain.
2. Question A.
3. Question B.

Wow. Over the past several days there have been numerous articles about URL design in one format or another, so much so that I’ve not been able to comment on them all in a reasonable amount of time.

Rather than wait, I decided to go ahead and give them all some link love and plan to discuss the issues they raise in due time:

• Tim Bromhead discussed Which is better for your site: www or no www?,
• Matthew Inman of SEOmoz talks about how attention to URL design in Dear Digg.com: I’m going to save you a million dollars with three lines of code,
• Scott Hanselman blogged about getting bitten by multiple URLs pointing to his site in Google PageRank considered subtle,
• Jessica Bowman of Business.com covered related terrority in her post Should URLs in links use index.html? on Search Engine Journal,
• Jeff Atwood explains how to do URL Rewriting to Prevent Duplicate URLs, and
• Microsoft’s own Scott Guthrie blogged about ASP.NET and URL Rewriting in Tip/Trick: Url Rewriting with ASP.NET.

So glad to see so many new URLians appearing on the horizon!

As I was going through my Akismet spam filter today reviewing the 87 comment spam I got during the prior ~24 hours to ensure I didn’t delete any legitimate comments, it occurred to me that maybe there is a simple solution to comment spam.

What if blog apps could simply mark a hyperlink with?:

rel=”spam”

The simple idea is that rather than delete spams, blogs could start maintaining a special page of links to comment spammer’s websites using rel=”spam” on the “A” element. Basically this would be PageRank in reverse. The search engines would then apply negative weighting to anything marked spam and give the spammers the exact opposite of what they were pursuing when they unethically tried to game the system!\.

 For example, Google could give negative PageRank for a spam link compared to positive PageRank for a non-spam link. Google could also weight the relevency of the link text negatively vs. the positive value it would give a non-spam link. This would have the affect of distributing the watch-dogging of spammers out onto the web without requiring any new infrastructure, and it would create a clear disincentive for comment spammers instead of the lack of disincentive from “nofollow.”

Are there problems with this I’m not foreseeing?  Probably.  I already know that people would try to game the system for negative purposes, and that’s to be expected. Still, I think that for the most part anyone simply using it to field a grudge or in as attempt to harm a competitor would be doing it by definition on such a small scale that it would have no effective. Given that the many comment spammers automate, they can end up with huge numbers of comment spam links. If the search engines merely weighted a spam link as 1/10th the negative value of a positive link, it would certainly still be effective.

Of course the hard-core Linux faithful would immediately spam-link to Microsoft.com just to spite them! But I really don’t (currently) see how that couldn’t be detected and managed via policies and algorithms. For example, if a company has a large number positive links it could be exempt from the effects of spam links. And I’m sure automated methods or methods using collective intelligence could emerge to resolve these problems the vast majority of time. The rest could be handled via policy; get caught spam-linking someone inappropriately and get your domain pulled from the index!

What’s more, it would give bloggers a sense of purpose when they review their spam filters instead of them feeling like the time spent was just a waste. I know that if my efforts to detect comment spammers could get them lower PageRank, I’d feel good about monitoring my comments for spam as I would be doing a service for the public good. And I’m sure most other bloggers would feel the same.

Now I know that Microformats.org has the similiar proposal VoteLinks, but that is about registering opinion as opposed to calling out gamers of the system. VoteLinks is also much broader than what I’m suggesting.  If we keep the focus really narrow — shine a spolight on spam so that the search engines can erradicate it — then I’m pretty sure it would be a success.

What do you think?  Good idea?  Filled with holes I’ve not considered?  I look forward to your feedback.

Something that would be fun would be to see a list of the top 10 worst designed URLs on the web, a.k.a. URLs that Suck! You know the type, you cringe when you see ‘em, especially if you frequently use their site.

So I’m calling on you to help in this quest. If you see a really bad URL, either place a comment on this page, or better yet, tag it with the following on del.icio.us:

urls-that-suck

If you want to tag a specific URL, please do that. But if you want to indicate the entire site’s URLs suck then tag their home page as well! Tag as many of a site’s different types of URLs that suck , but be fair to the site and only tag one of each type as we don’t want to overrepresent the suckiness of a site’s URLs.

And what’s more, tell your friends about it. It’d be great to get an army of URLians to keep an eye out for those really horrific URLs. Together we can shine a light on these abominations and give their owners a reason to bow their head in shame! ;-)

As soon as I get a statistically significant sampling[1] of sucky URLs, I’ll announce my pick for the Top 10 “URLs that Suck” Awards; probably at the end of 2007, but maybe sooner. I’ll also honor the top ten contributors and as well showcase anything or anyone that’s a standout for any reason. And if anyone is so inclined, talk to me about sponsoring these awards; you know you wanna!

If you are onboard for this, please leave a comment letting us know you plan to be on the lookout. And to kickstart this, I will nominate the very first site that has some truly awful URLs that Suck!:

Amazon.com

P.S. And if they are any excellent graphic designers out there who would like to help out with the cause, how about designing some badges that bloggers could post on their website linking to this post with a catchy slogan, something like “Help Stamp Out URLs that Suck!” or “Expose Sucky URLs”, or better; you get the idea!

1. What is a “statistically significant sampling?” I don’t know yet, but I’ll know it when I see it. ;-)

For our web applications — our website, wiki, blog, and future forums — we chose to segment each by sub-domain; “wiki.”, “blog.”, and “forums.”, respectively. Rather than having subdirectories underneath the root of our www domain such as “/wiki/”, “/blog/”, and “/forums/”, we chose to use sub-domains to allow flexibility and maybe even scalability in the future.

Having the each application on a domain that is different from the blog and forums mean we can host on different machines and at different locations, as appropriate. However as I write this I realize that this rationale violates probably one of the most important best practices for URL Design; don’t allow your backend implementation dictate your URL structure!

It also means our URLs will be by 4 characters shorter[1] and since we chose a long domain name, that’s important. Still, there are pros and cons to both approaches and my gut tells me we’ll uncover further considerations as we continue researching this topic.

Related to this topic, we’ll discuss the URL Structure patterns for websites, wikis, blogs, forums, and other web apps in future posts.

1. Assuming we were to continue using “www.” as a sub-domain for our main site, which itself is another topic for us to address in the future.

Here’s a run down of the web application software we are using or planning to use for the Well Designed URLs Initiative:

Blog

At “blog.” is running WordPress using Linux, Apache, MySQL, and PHP. We have direct experience with WordPress, Typepad, dasBlog, and Community Server, and among these options WordPress is definitely the best blog software for the URL Aficionado.

Wiki

At “wiki.” runs Mediawiki, also on using Linux, Apache, MySQL, and PHP, and is the same software used by as Wikipedia. Although setting up clean URLs can be a real pain on Mediawiki if you are not a regular expressions wizard, we think Mediawiki is the most mature wiki software, and nobody can debate that it has the best known user-interface.

Forums

To be at “forums.” will almost certainly run on vBulletin which also runs on Linux, Apache, MySQL, and PHP. Even though vBulletin isn’t open-source and costs US$85/year to lease[1], as users we have used practically every forum in existence and in our opinion nothing comes close to the usability and performance of vBulletin. We only wish that the vendor of vBulletin could envision a business model leveraging open-source so those who for whatever reason won’t use commercial software would be able to consider vBulletin; we think that would make the forum world a nicer place. Well that, and changing their default template; the visual layout as shipped is horrible!

CMS

To be at “www.“; we’ll probably use Drupal, Django, or similar when we get to installing and configuring it. In other words, when we really need the features of a CMS, we’ll add one. ;-)

The all the web applications we mentioned are great, and we can highly recommend them to almost anyone. The downside of using them though is being separate apps they are not well integrated. But, we’ll see what we can do about that in the future. Hopefully someone will see what a huge need and certain demand for integrating open-source web apps based on MySQL and they will solve this problem just as the WSGI team seems to be solving a similar problem in the Python web app world.

1. Or you can purchase vBulletin for US$160 with one year of updates.

One of our plans is look at major topics from many different angles. Though this will allow us to revisit an issue numerous times, our main reason for covering topics in this manner is that making sure our coverage is complete and includes all references is really difficult. And we (well at least, I) have a tendency to postpone writing on a topic until I’m sure my research is exhaustive. Unfortunately, that often leads to me writing about something long after it’s relevant!

So rather then delay an important or timely topic, the plan is to both write about them numerous times with each time taking a slightly perspective and references different resources. And we plan to publish those multiple angles in the chronological of our normal blog publishing cycle.

But we also plan to maintain a section of Major URL-related Topics by consolidating the content from our numerous chronological posts on a topic and often augmenting with additional content not published during our normal blog cycle. You’ll find those pages at the following URL:

http://blog.welldesignedurls.org/major-topics/

Unlike our category pages that are generated by a SQL query from our prior posts by WordPress, we’ll actually be hand writing the content for these major topic pages. And we’ll do our best to keep those pages updated to cover our most recent research and views on the subject. In addition, we’ll reference any of our related posts and also provide quotes and links from related conversations on the Internet including those from other bloggers, mailing list archives, and online magazines.

Our goal is to make these major URL-related topic pages your most thorough, up-to-date, and hopefully concise resource on the subject. And we hope you’ll link to them if you need to reference one of our major URL-related topics when writing an article or posts for your own blog.

NOTE: To see why this is labeled “Part 1a” see [1].

Although many people view the URL to be a rather obscure little creature, the URL is the arguably the most important of the web’s three essential legs; the “First Consul” of the triumvirate of web technologies; the truly unsung hero of the web.

When Tim Berners-Lee originally designed the Web, he did so by creating three core independently-specified yet inter-related technologies on top of the existing TCP/IP-based Internet. Used together, those three technologies leveraged each other to create spectacular value for the world.

The first of those three technologies, HTML, has been broadly appreciated by web developers and even lately a tremendous number of end-users for allowing them to layout text and graphics how they prefer them to be display[2]. The second, HTTP, is intimately familiar to most web users, but only as the prefix they see in countless television ads, on business cards and billboards[3], and in their web browser’s URL field as they use hypertext to navigate the web.

But whereas both HTML and HTTP are independent of the other, both rely heavily on the URL. Without the URL, HTML would have no Hypertext and HTTP could neither retrieve documents[4] from websites nor issue redirects when documents move. Berners-Lee even “canonized” the URL[5] in his 1999 Wired Interview:

“Well, the most important thing that was new was the idea of URI — or URL [it was UDI back then, universal document identifier]. The idea that any piece of information anywhere should have an identifier, which will not only identify it, but allow you to get hold of it. That idea was the basic clue to the universality of the Web. That was the only thing I insisted upon.”

But like the man behind the curtain to whom we are told to pay no attention in the Wizard of Oz, many web technologists have mistakenly promoted the notion that the URL should only be used for behind-the-scenes plumbing and thus hidden from view. Sadly, these technologists rationalize their stand by stating that mere-mortals couldn’t possibly understand URLs anyway, but nothing could be further from the truth. For this reason and more, URLs true potential has gone unfulfilled as they have been woefully under-appreciated in comparison to the power and benefits they can provide both the average web user and the web at large.

The URL, a.k.a. the URI, is more than just some behind-the-scenes, don’t-speak-until-spoken to technology. URLs provide a means to identify both concepts and tangible things and then allow related documents to be retrieved that can provide more information and/or further reference URLs. While most people think in terms of how they personally use URLs, they rarely envision the orders of magnitude of serendipitous uses by others of the URLs they assign and then publish.

It’s often said that what you don’t know can’t hurt you, but unfortunately this lack of understanding the power of the URL creates huge missed opportunity for most people. Just one simple example should illustrate this and that is the value of Google’s search engine indexer; without URLs there is no chance Google will send anyone to learn more about a concept or tangible thing that could have been made accessible by URL but wasn’t. Seeing how much money Google makes from advertisers seeking traffic to their website, it’s amazing that more people don’t realize the value of the indexable URL. But Google’s use of URLs is the mere tip of the iceberg when compared to all the serendipitous use one can receive by assigning and publishing Well Designed URLs for everything they view should be notable to others.

Tim Berners-Lee has been quoted as saying “Everything of importance deserves a URL“[6] and it is our opinion that is probably his most profound statement ever. So one of the key goals of the Well Designed URLs Initiative is to right the wrong of past technologists who have been hiding URLs from the average user’s view. By elevating URL’s the significance in the minds of both Internet professionals and the masses we hope to be the catalyst for the positive change that the recognition of URL’s value will bring.

1. This part of our introduction was published out of order hence it’s designation as “1a” instead of as “2.” During the process of writing it became clear that this needed to be said but unfortunately we had already published numerous posts that would have obviously come after this one. Ah well, that’s what happens when you start publishing before you finalize the outline. :-)
2. Graphic designers would argue this point, but that would be nit picking for the purposes of this blog post.
3. Albeit the trend is toward omitting and hence assuming the “http” written and spoken communication.
4. The Weborati would correct me and say that HTTP retrieves “resources“, but why quibble over common use when this post is non-normative?
5. My use of the term “canonized” is obviously just a bit of “dramatic license.”
6. Actually Tim uses URI instead of URL which is his preference vs. mine. Go here to read about URL vs. URI here.

I’m taking the day off but wanted to wish everyone a Happy New Year hoping 2007 will be everyone own best year yet!

In the hustle and bustle of these holidays, I didn’t manage to churn out all of my planned posts for our introduction series. So with nothing else to publish today it seems only fitting I introduce you to the works of someone I met recently. His name is Ben Coffey and he’s an absolutely brillant kid[1] from the U.K. who shares my passion for User-Centric User Design.

His work that I spoke of is his ebook entitled “Useful URLs” It is available for download at his website inelegant.org in both HTML and PDF formats. Further kudos to Ben for he licensed Useful URLs using a Creative Commons “by-nc-sa“ license meaning you’re free to copy and distribute it as long as you attribute him as author, you don’t use for commercial purposes, and you share any of your own modifications or additions with the same license.  Ben says that Useful URLs is a work-in-progress so he plans on updating it continuously. I’ll be sure to make mention here when he updates it, especially if he adds new chapters.

That said, Useful URLs is both a quick read and also:

Highly Recommended!

• Sorry for calling Ben a kid, but at 23 he’s barely more than half my age, and I still feel relatively young! But the more I talk to him, the more I think I barely know half what he does, at least with respect to programming and the Internet! I’m actually afraid to talk to him about anything else for fear I’ll learn that I barely know half as much on those topics as well!

Here at the Well Designed URLs Initiative we plan to address a wide audience and cover a plethora of URL-related topics. If it wasn’t obvious from yesterday’s post we plan to publish content for a variety of roles so we will categorize all our posts by the audience we are targeting.

Using our audience categories you can subscribe to our RSS feed then configure your feed reader to filter out all but those topics which are likely to appeal to you so as not to be overwhelmed by the rest. Some of our audience categories encompass other categories such as Everyone and Internet Professionals so we’ll plan to tag only the highest level category that applies to avoid duplication. For example, if you are a web developer you might want to filter out all but the Everyone and Internet Professionals, and Web Developers categories. Of course, if that’s too much trouble just subscribe to our entire feed and just ignore those posts that don’t interest you.

The following is the list of categories we’ve set up by audience role:

• All Internet Professionals
• Conference Promoters
• Teachers
• Everyone
• Framework Developers
• Hosting Providers
• Infrastructure Providers
• Leading Companies
• Open-Source Participants
• Content Publishers
• SEO Consultants
• Software Vendors
• Standards Participants
• Web App Providers
• Web Designers
• Web Developers
• Website Owners

NOTE: If you read this post shortly after it is published, most of those links above will just redisplay this post. Let me explain. Because of the way our WordPress blog software works, those links would have displayed a 404 Not Found error if no posts existed for the given category. To avoid that I’ve tagged this post with all audience categories contradicting what I said above; that we would only put a post in its highest level category. Moving forward we shouldn’t need to do this again.

We’ve already said that we plan to question the status quo, but we want to make that point even stronger; we plan to advocate for positive change.

At least one person has already suggested that our activities would be best accepted if we, like the microformat initiative aimed to merely “uncover what already works” on the web and then dispassionately documented it for discussion. Although we do plan to research and then explain current patterns and best practices we don’t plan to make those our only activities; we didn’t pursue this merely to be the scribes of current behavior.

Instead, we are promoting this initiative because we envision specific solutions to current URL usage problems, and we believe those solutions will significantly improve both the web’s accessibility and user’s experience with the web. It is this vision that motivates us to pursue this path even if it means that at times it might be an uphill battle.

But what specifically do we plan to advocate? To give you an idea I prepared the following chart, organized by who will be the target of our advocacy:

Although this list of objectives is significant, it doesn’t represent the entirety of what we hope to accomplish via this initiative. Stay tuned.

I’m taking the day off to observe the holiday that millions of people observe worldwide. Whether you celebrate Christmas or observe some other holiday, it’s my hope that you and your loved ones are happy and healthy this holiday season. See you tomorrow.

Yesterday we said we plan to make recommendations. That may have concerned some, especially those who are involved in the standards process. But don’t worry, we won’t make them in a vacuum; we intend to engage everyone interested to participate and influence the creation of the recommendations, especially the leading experts in the field of web architecture. We pledge to evaluate everyone’s input objectively, and incorporate input without prejudice.

What’s more, we aim to respect all the standards created by recognized standard’s bodies such as the Internet Engineering Task Force and the World Wide Web Consortium. Further, we plan to respect the spirit of all standards and acknowledged axioms.

Though we did expose a long list of agenda items, some of those may fall by the wayside once we engage the community. Those were our ideas but as we are not perfect some of them may not be either. Our goal is to make things better, not worse, and intellectual honesty is our only ally in addressing reality.

But that doesn’t mean we intend to always be non-controversial. No, we intend to push the envelope and lead some people out of their comfort zone. Like a child constantly asking “Why?” we plan to challenge conventional wisdom. We want to ensure that the axioms and standards defining web architecture are still based on actual requirements. We don’t want to see progress impeded simply because the framers of the web shared the universal human trait of being usable to foresee all future innovations and their requirements.

Though potentially disruptive, we think our willingness to question prior assumptions will ultimately lead to a better web for all.

One of our main goals is to publish an evolving list of URL Design and related recommendations that the world of web architects, website administrators and web developers will respect and embrace. Our recommendations will include advocacy for, but not limited to the following:

• Establishing Sitemaps as fundamental in URL Design,
• Recognizing URL Patterns as first class concepts,
• Conventions for file-extension to media-type mapping,
• Deprecating Regular Expressions for URL Rewriting,
• Use of URI Templates where applicable,
• Hardening URI Templates for real world usages,
• URL-related Metadata in HTML and HTTP,
• Formalizing the use of Canonical URLs as identifiers,
• Exposing Canonical URLs to users,
• Leveraging the Permalink to identify Canonical URLs and more,
• Handling URLs with Embedded Session Identifiers,
• Handling URLs with Embedded User Identifiers,
• Handling URLs with Embedded Print Modes,
• URL Virtualization to overcome platform limitations,
• Well-known Media-Types for REST’s Hypermedia Constraint,
• Making URIs Dereferencable whenever possible,
• Strategies for dereferencing every URI to a resource,
• Creating Unique URLs for all Dereferenceable Resources,
• Rationalizing Content Negotiation vs. Unique URLs,
• A Web Request Media-type for POST, PUT, and DELETE,
• Strategies for serving Alternate Language Representations,
• Making AJAX, Flash, and PDF resources dereferenceable,
• Publishing Multiple URL Paths to the same resource,
• Metadata to allow greater URL equivalence evaluation,
• Recommending software based on URL configurability,
• Recommending  to vendors how to improve URL configurability,
• Exposing Segmentable Content via URL Fragments,
• Exposing URLs for all deferencable content,
• Strategies for handling Broken Links,
• Strategies for URL-related Education for all users,
• And even more.

So you see, we’re planning to bite off an awful lot here but we think by doing so we’ll benefit all web users. Didn’t realize that URLs had so much to room for improvement? Stay tuned…

There is a belief among the people who guide the web’s technical architecture, people I like to call the “Weborati“, that as few constraints as possible should be placed on web publishers and web users. This is referred to as the Principle of Minimal Constraint. An example of such a constraint is the Robots Exclusion Standard that uses the “robots.txt” filename to inform search engines of pages to avoid. Because of this robots.txt “standard”, a web publisher can’t use a file named robots.txt in the root of their website for any other purpose besides the Robots Exclusion Standard without disabling their ability to inform web robots which files to ignore.

Although using the Well-known Name “robots.txt” was justified by Roy T. Fielding, use of well-known names such as those required by P3P and Favicon continue to be a sore point among the Weborati. However, contrary to some initial concerns, we here at the Well Designed URLs Initiative agree that using well-known names is A Bad Thing(tm) in Web Architecture. So let me state for the record: It is not our intention to encourage any new well-known names.

On the other hand, we do intend to recommend convention. Good conventions can increase productivity immensely by eliminating the need to evaluate alternate yet arbitrary choices. One such convention is the multi-column web page layout complete with a menu along either the top or one of the side columns. Prior to that established layout, right for many websites but not all, web designers spent considerable time evaluating different layouts. I, for one, am glad to have gotten past that stage.

Good conventions can aid common understanding if those conventions are widely used. For example the Internet sub-domain naming convention of “www” has helped many non-technical people recognize a web address on a business card, a TV advertisement, and on the side of a bus.

And good conventions can spawn an entire industry segment, unleashing a completely new layer of value. When Jesse James Garrett published his now well-known essay entitled “Ajax: A New Approach to Web Applications,” he recognized a design pattern that many people had been using in obscurity, and that recognition catalyzed a mega-million dollar segment of the web development industry.

Though we could only hope our efforts have the same effect, we do hope to influence how people perceive and use the lowly URL. For example via research and a consensus of opinion, it would be greeat to establish a widely used convention for how multi-lingual websites design their URLs.

Although conventions can be constraining, conventions are not in and of themselves requirements. Just as it is well known that “www” probably means “website”, it is not a requirements and the fact many websites use alternate sub-domain names prove it. For example, “newyork,” “london,” and “paris” could all easily be used as sub-domain names for relevant websites. Or a website can easily use no sub-domain at all!

So it is our hope we can get people from the Weborati down to the non-technical website owners to appreciate and embrace numerous new URL conventions in the future.

So what makes us “the” expert on URL Design as we are not (yet?) associated with any standards body like the W3C or the IETF? Nothing really, other than we have a keen interest in the URL Design and its ramifications, and are proactively devoting time to this initiative.

For me personally, I’ve spent lots of time spent studying any and every bit of writing I could find regarding URL Design, and even more time spent pondering the subject and writing about it. Lastly, it was my writing about “Well Designed URLs” and other’s subsequent interest that lead me to start this initiative.

Plainly put, I and the others involved are the first to take significant initiative to study and then advocate for User-Centered URL Design. We feel so strongly about this that we believe it should be taught in universities as part of the new field of “Web Science.”

As with any body of wisdom it has to start somewhere, and because no one else was focusing on URL Design, we decided we would start I here; it’s that simple.

That said, if you feel that you have something significant to contribute to URL Design then by all means join us and become one of the experts yourself. And if we do a good job, we may become recognized by the W3C and/or the IETF, and that will make us the experts. :)