DOIs, URIs and Cool Resolution

This post originally appeared on the Wordpress.com version of this blog.

The art of happiness is to serve all -- Yogi Bhajan

Once we get beyond the question of the basic HTTP URI-ness of the digital object identifier (DOI) --- since for each DOI there exists DOI-based URIs due to the dx.doi.org and hdl.handle.net proxies, this issue is moot --- and old-skool questions of "coolness" based on the relative brittleness over time of creative URI encoding [1], we are then left with the more substantial question of whether DOI-based HTTP URIs really "behave" themselves within the "Web-of-Objects" universe. The purpose of this post is to identify the problem and propose a potential solution, implementation of which will require certain changes to the current Handle System platform. I believe that if the proposed changes are made, lingering questions concerning the "URI-ness" of DOIs (and Handles) will disappear, once and for all.

Note: It is beyond the scope of this post to present all of the gory background details regarding the Handle System, the DOI, and the 1998 and 2008 versions of "Cool URIs." If there is enough interest in a stand-alone article, I will happily consider writing a longer version in the future, perhaps as piece for D-Lib Magazine.

With the increasing influence of semantic web technologies there has been strong interest in assigning actionable HTTP URIs to non-document things, ranging from abstract ideas to real world objects. In the case of URI-named, Web-accessible physical items --- sensors, routers and toasters --- this is sometimes referred to as The Web of Things. Until 2005 the community disagreed as to what an HTTP URI could be assumed to represent, but a June 2005 decision by the W3C TAG settled the issue: If a server responds with an HTTP response code of 200 (aka a successful retrieval), the URI indeed is for an information resource; with no such response, or with a different code, no such assumption can be made. This "compromise" was said to have resolved the issue, leaving a "consistent architecture." [3]

The result of this decision was to force consensus on how to apply the long-established principles of HTTP content negotiation in more consistent ways. In particular, "human" and "machine" requests to a given entity URI --- a top-level URI representing a "thing" --- should be treated differently; for example, there should be different responses to requests with HTTP headers specifying Accept: text/html (for an HTML-encoded page) versus Accept: application/rdf+xml (for RDF-modeled, XML-encoded data). This is most often seen in the semantic web and linked data worlds, where it is now common to have both textual and machine readable manifestations of the same URI-identified thing.

Modern web servers including Apache have been engineered to handle these requests through content negotiation [4]. Through standard configuration procedures, site administrators specify how their servers should respond to text/html and application/rdf+xml requests in the same way they specify what should be returned for alternate language- and encoding- requests; "en," "fr," etc. Typically, when media-specific requests are made against entity URIs representing concepts, the accepted practice is to return a 302 Found response code with the URI to a resource containing a representation of the expected type, such as an html-encoded page or an XML document with RDF-encoded data.

Many readers of this post will be familiar with the basic idea of HTTP proxy-based Handle System name resolution: A HTTP resolution request for a DOI-based URI is made to a proxy --- a registration-agency run proxy such as dx.doi.org or the "native" Handle System proxy hdl.handle.net --- the appropriate local handle server is located, the handle record for the DOI is resolved, and the default record (e.g. a document information page) is returned to the client as the payload in a 302 Found response. In a Web of Documents this might make sense, but in a universe of URI-named real-world objects and ideas, not so much.

The 2008 document provides two requirements for dealing with URIs that identify real world objects:

1. Be on the Web: Given only a URI, machines and people should be able to retrieve a description about the resource identified by the URI from the Web. Such a look-up mechanism is important to establish shared understanding of what a URI identifies. Machines should get RDF data and humans should get a readable representation, such as HTML. The standard Web transfer protocol, HTTP, should be used.
2. Be unambiguous: There should be no confusion between identifiers for Web documents and identifiers for other resources. URIs are meant to identify only one of them, so one URI can't stand for both a Web document and a real-world object.

In the post-2005 universe of URI usage as summarised above and detailed in [2], if DOI-based URIs are used to represent conceptual objects these rules will be broken! For example, Handle System proxies today cannot distinguish between Accept: codes in the request headers; the only possible resolution is the default (first) element of the Handle record. (For hackers or merely the curious out there, I encourage you to experiment with curl at your command line or Python's urllib2 library, hitting the DOI proxy with a DOI-based URL like http://dx.doi.org/10.1109/MIC.2009.93.) This problem with how proxies resolve DOIs and Handles is a lingering manifestation of the native Handle System protocol not being HTTP-based and the system of HTTP-based proxies being something of a work-around, but the vast majority of DOI and Handle System resolutions occur through and rely on these proxies.

One possible solution would be to enable authorities --- Registration Agencies --- who operate within the Handle System to configure how content negotiation within their Handle prefix space is handled at the proxy. For document-based use of the DOI an example of this would be to return the URI in the first element of the Handle record whenever a text/html request is made and (for example) the second element whenever an application/rdf+xml is made. When a request is made to the proxy, request-appropriate representation URIs would be returned to the client along with the 302 Found code. This approach treats the DOI-based URI as a conceptual or entity URI and gives the expected responses as per [2]. pax vobiscum...

Readers familiar with the Handle System will appreciate that there are many potential schemes for relating HTTP content type requests to elements of the Handle record; in the example above I use position (index value), but it is also possible to use special TYPEs.

Handle servers are powerful repositories and can implement potentially many different models other than redirection as described above. Sometimes, for example, the desire is to use a Handle record as the primary metadata store. In that case, the preferred application/rdf+xml might very well be to return an RDF-encoded serialisation of the Handle record. How this is handled should be a feature of the Handle server platform and a decision by registration agencies based on their individual value propositions, and not locked in by the code.

I eagerly look forward to your comments and reactions on these ideas!

Update 1: In a comment to this post, Herbert Van de Sompel argues that the real question is, what should DOIs represent? Herbert asserts that DOI-based URIs should model OAI-ORE resource aggregations and that Handle System HTTP proxies should behave according to OAI-ORE's HTTP implementation guidelines. Herbert's suggestion doesn't conflict with what I've written above; this is a more subtle and (arguably) more robust view of how compound objects should be modeled, which I generally agree with.

Here's how OAI-ORE resolution would work following the Handle proxy solution I've described above: Assume some DOI-based HTTP URI doi.A-1 identifies an abstract resource aggregation "A-1" (In OAI-ORE nomenclature doi.A-1 is the Aggregation URI). Following the given HTTP implementation example, let there be two Resource Maps that "describe" this Aggregation, an Atom serialization and an RDF/XML serialization. Each of these Resource Maps is (indeed MUST be) available from different HTTP URI's, ReM-1 and ReM-2, but the desired behaviour is for either to be accessible through the DOI-based Aggregation URI, doi.A-1. Let these two URIs be persisted in the Handle record, preferably using TYPEs which distinguish how they should be returned to clients based on the naming authority's configuration of the HTTP proxy. By the approach I describe above, the Handle System proxy would then respond to resolution requests for doi.A-1 with 303 See Other redirects to either ReM-1 or ReM-2 depending upon MIME-type preferences expressed in the Accept: headers of the requests.

Update 2: Complete listing of MIME types for OAI-ORE Resource Map serializations. Follow-up conversations with Herbert Van de Sompel, Carl Lagoze and others have reminded me I neglected to mention how the OAI-ORE model recommends handling "HTML" (application/xhtml+xml and text/html) requests! This is not a minor issue, since the purpose of ORE is to model aggregations of resources and not resources themselves, and so it is not immediately clear what such a page request should return. My solution (for the purposes of this blog post) is for Handle System HTTP proxies to respond to these requests also with 303 See Other redirects, supplying redirect URIs that map to appropriately-coded "splash screens."

For completeness, the table below (repeated from [5]) lists the standard MIME types for Resource Map serializations. Continuing with the major theme of this post, Handle System HTTP proxies resolving requests for DOI-named ORE Resource Maps should follow these standards so the clients may request appropriate formats using HTTP Accept: headers.

Resource Map Type	MIME type
Atom	application/atom+xml
RDF/XML	application/rdf+xml
RDFa in XHTML	application/xhtml+xml

If a client prefers RDF/XML but can also parse Atom then it might use the following HTTP header in requests:

Accept: application/rdf+xml, application/atom+xml;q=0.5

The table below list the two common MIME types for HTML/XHTML Splash Pages following the W3C XHTML Media Types recommendations.

Resource Map Type	MIME type
XHTML	application/xhtml+xml
HTML (legacy)	text/html

Thus, if a client wishes to receive a Splash Page from the Aggregation URI and prefers XHTML to HTML then it might use the following HTTP header in requests:

Accept: application/xhtml+xml, text/html;q=0.5

As noted in [5] there is no way to distinguish a plain XHTML document from an XHTML+RDFa document based on MIME type. It is thus not possible for a client to request an XHTML+RDFa Resource Map in preference to an RDF/XML or Atom Resource Map without running the risk of a server correctly returning a plain XHTML Splash Page (without included RDFa) in response.

The Handle record for a given DOI or Handle identifying an ORE aggregation would therefore contain a set of URIs reflecting the mappings in the tables above. A content-negotiation-savvy Handle System HTTP proxy would then return the appropriate URI in the 303 Found response, based on its configuration and policies.

References:

• [1] Tim Berners-Lee, Cool URIs don't change (1998)
• [2] Leo Sauermann, et.al., Cool URIs for the Semantic Web (2008)
• [3] Tim Berners-Lee, What HTTP URIs Identify (2005)
• [4] Apache Foundation, Content Negotiation (Apache 2.2)
• [5] Carl Lagoze and Herbert Van de Sompel, et.al., Common MIME Types for Resource Maps and Splash Pages, ORE User Guide: HTTP Implementation (17 Oct 2008).

See the ensuing comments at my Wordpress.com version of this blog...

DRM & Me Part III: DOIs, Metadata and Long Tails

In Part II of this retrospective I discussed the NetRights years and our novel approach to binding static and dynamic metadata to objects in the early days of the Web. In this installment I'll cover my years at Yankee Rights Management (YRM) (a division of YBP, Inc., especially the development of Copyright Direct(tm) and my personal realization of the potential of content identifiers and their associated metadata. Note: It was actually during my YRM years that I coined my now-infamous expression (referenced in Part II of this series), Metadata is the lifeblood of e-commerce!

YBP, originally known as Yankee Book Peddler and now a division of Baker & Taylor, have been a leader in using information technology to provide books and other materials, including bibliographic data --- metadata! --- to university and research libraries for more than 35 years. YBP executive Glen M. Secor also happened to be a professor of law at the Franklin Pierce Law Center specializing in copyright law, with a particular interest in the unique challenges of copyright in the emerging digital, networked environment. Glen and I first met when I presented my early Ph.D. work at DAGS'95 in Boston (prior to the founding of NetRights) and from that point on took an interest in this metadata-oriented, iconoclastic approach to copyright. Glen spearheaded YBP's investment in NetRights in 1996, and with the sale of NetRights in 1997 I joined with Glen to launch Yankee Rights Management (YRM) in mid-1997.

One of YRM's goals was to build a business solving rights management problems for stakeholders in YBP's ecosystem, especially scientific/ technical/ medical (STM) publishers and their university and research customers. With the help of Kelly Frey, then VP of Business Development for the Copyright Clearance Center (CCC), we conceived of Copyright Direct(tm), which soon became the first web-based, real-time, pay-as-you go copyright permissions service for a wide variety of multimedia types. As with LicensIt(tm), the usage model for Copyright Direct(tm) would be simple:

1. From a web page or PDF document, the user would click on a distinctive green "Copyright Direct" icon
2. A mini-window would pop up clearly identifying the work and presenting available options for that item
3. The user would step through a short series of menus to specify their use and, if available, transact their request (via credit card!) and receive their permissions
4. If the usage they needed was not available, the system collected the user's plain-text request and began a managed workflow between the user and the rightsholder
5. When all parties agreed, the agreement became a "template" and was added as an available option --- the system learned and adapted
6. At the end of each month, rightsholders would receive royalty payments.

Glen Secor, Jennifer Goodrich and I demonstrated my Copyright Direct prototype to a variety of stakeholders and thought leaders at the Frankfurt Book Fair in October, 1997 and collected critical feedback. We returned "triumphantly" in October 1998 with a booth in the main hall, a live Copyright Direct demo (now powered by the fledgling DOI standard and a major "beta" rightsholder: the IEEE!

But throughout 1998-1999 we also came to realize a fundamental problem with the Copyright Direct model: it depended not only on a ready supply of clean descriptive metadata from rightholders, but also upon a rich set of rightsholder-generated rights metadata, including pricing and other licensing templates, none of which existed! Our goal was to use lightweight, easily accessible permissions transactions to provide "found money" to rightholders, but it cost too much to generate the metadata required to fuel the system! In the September 2006 issue of D-Lib magazine I extrapolate this problem in my article, Handle Records, Rights and Long Tail Economies.

Chris Anderson's "long tail" argument (see also his Long Tail blog) asserts that modern systems based entirely on metadata make "unlimited selection" economically viable. I argue that yes, metadata really is the lifeblood of e-commerce and is the enabler of phenomena like the seemingly-unlimited selection of products through Amazon.com ("make everything available, help anyone find it!"), but all metadata must somehow still be generated, verified and published, and the cost of creating and supporting the neccessary metadata supply chains must not exceed the anticipated value that can be redeemed. Since the demand of a given "unit" may be exceptionally low, the "per unit" cost of creating or aggregating each unit's metadata halo must be near-zero!

These principles can be extrapolated to the "Web of Data"; indeed, by coupling Linked Data principles with a low-overhead infrastructure for authenticating metadata assertions, the cost of metadata may indeed approach zero. I'll talk about that in a future blog entry...

DRM & Me Part II: "Copyright for the rest of us!"

In Part I of this retrospective I covered the raw beginnings of my interest and research in enabling copyright in the digital, networked environment. In this second part I'll discuss work my colleagues and I did to take these ideas commercial, and I'll continue to focus on core principles of my work in content identification and metadata architecture, summed up by this quote (attributed to me!): Metadata is the lifeblood of e-commerce!

As the spring of 1995 approached it became clear that there was an opportunity to make a unique contribution to improving the world of copyright in the digital, networked environment. As I prepared to present a paper at ED-MEDIA 95 in Graz, Austria, I was approached by local businessman who had been principals in a successful software company, Corporate Microsystems, Inc., that had just been acquired by a global enterprise software company. As the story goes, they were looking for an original idea upon to base their next start-up, and I was looking for a strategy for implementing my research ideas that would scale well beyond what I was capable of doing part-time as a researcher at IML. Over the summer of 1995 my future partners Gerry Hunt, Theo Pozzy, Henry Adams, Hal Franklin and I held numerous planning meetings, and on 1 November 1995 NetRights, LLC was born!

We started NetRights at a time when other players, in particular InterTrust (then still called EPR) and IBM InfoMarket were starting to draw attention to their robust, encryption-based "envelope" strategies for "protecting copyright" --- quotes intentional! --- and the term digital rights management wasn't yet in standard use. Taking a clue from my prototype work at Dartmouth, the core idea behind LicensIt(tm) (later @attribute) was to "objectify" flat multimedia objects using secure wrappers whose primary objective was to provide structured metadata about the object in hand. Our goal was to provide rich static and Internet-served dynamic metadata to facilitate "conversations" between creators and users of content. Our motif for "experiencing" copyright was a simple and elegant: A user sees a photo, audio clip, video, even an embedded text snippet; they "right-click" on it and a tabbed set of property pages is displayed; they use those various pages to view descriptions of the content, to start emails with the creator or other contributors, to view default terms of use, even to initiate live rights transactions, all while staying within the context of use.

From a technical standpoint we were using OLE structured storage in very much the same way as XML (and especially RDF) is used today. Our development team, including Mark Schlageter, Norm Tiedemann, Mark Markus and Dan O'Connor (our sole Mac-head!), created amazing tools that let us design not only these metadata structures, but to actually create "soft" property-page layout templates (think CSS!) that were packaged with the metadata, enabling customized content-specific views. Considerable infrastructure was required to make all of this work, starting with OLE services installed on the user machine, to the tools for design and packaging, to back-end services for object registration. Also, major, bet-the-company decisions about PC vs Mac, "networked COM" (which became ActiveX), Spyglass/IE vs Mosaic/Netscape support, etc. To a startup company, Bill Gates' commitment of Microsoft to "embracing and extending" the Internet in late 1995 was helpful!

Trade journals like Seybold took notice and wondered whether our "kinder, gentler" approach to copyright, which by that time (June 1996) we were calling "enhanced attribution," might actually be a better option than so-called "opaque packages." Publishers were torn; they liked the obvious value our approach was bringing to the user and the fact that we were actually facilitating the copyright process, but they also couldn't get over their perceived need for "strong protection."

Today we see echoes all over the Internet of infrastructure and technology that make "copyright for the rest of us" radically easier than it was at NetRights birth in 1995. First and foremost are systems of globally unique, persistent object identifiers, in particular the Digital Object Identifier (DOI), implemented on CNRI's Handle System. (As it happens, that same 1996 issue of Seybold also carried an article about the birth of the DOI!) RDF provides a universal information model for conveying metadata assertions (local and remote) about objects; RDFa provides a way to do this within (esp.) web documents. The recent massive and growing interest in publishing Linked data by organizations, including governments, has fortified distributed metadata as a means of conveying object information from a variety of sources. And special mention must be made of The Creative Commons, which has applied most of these techniques to not only make the process of copyright readily accessible to creators and users all over the world, but also to make content use safe through the explicit and unambiguous communication of terms of its use.

Providing immediate, unambiguous expression of copyright information and connections to processes for any piece of content was my mantra starting in the lab at Dartmouth, then at NetRights, and following our acquisition in 1997 by Digimarc, with the creation of Copyright Direct(tm) at Yankee Rights Management (YRM) and my subsequent involvement with the content identification and metadata communities. More on that in our next installment...

DRM & Me: A 15-year retrospective (Part 1)

Fifteen years ago, in November 1994, I was two years into a Ph.D. program at the Thayer School of Engineering at Dartmouth College. I had entered Dartmouth with a background in computer engineering and an interest in "special-purpose systems," a narrow field that focuses on creating computing systems that are exceptionally good at a very narrow range of operations, such as particle-in-cell simulation or gene sequence processing. This interest led me across campus to become a research assistant in Dr. Joseph V. Henderson's pioneering Interactive Media Lab at Dartmouth Medical School --- at first to consider the infrastructural problems of delivering IML's high-value multimedia training programs across the Internet, and by mid-1994 over a novel set of technologies known as the "World Wide Web."

As the story goes, the IML team was preparing a major set of demos for a visit by Dr. C. Everett Koop, a Dartmouth alumnus, area resident and recently retired as one of the more influential Surgeons General the United States has ever had. My particular focus was creating an interactive web site for IML, focusing in particular on the delivery of several key video sequences via the web. Several of us worked long into the night to migrate a few select videos into tolerable Quicktime format and suitable "thumbnails," then onto the lab's server, then linked (for downloading) from web pages, and finally viewable on the demo Mac.

When Joe arrived on the morning of our demo, I greeted him with (something like), "Joe, I got the 'Binding Sequence up on the Web!'" His incredibly insightful response was:

John, that's great!...John, that's terrible!

Joe preceded to express his concerns about two fundamental implications of my "success":

• The copyright implications, especially as many IML programs were funded by private entities that retained certain rights to the works;
• The implications of dis-aggregating medical and other training programs and delivering their content out-of-context, possibly doing harm to their message due to loss of design integrity.

Joe framed the challenge for me: to study the question of rights management from the perspective of multimedia production. In 24 hours, I learned that this was an important and rising issue that was not going away; that very little research had been done on the question from a practical standpoint; that the few proposed solutions at the time were overly simplistic, equating "copyright management" with "security" and in fact did neither; and no one appeared to be considering the issues from the perspective of the creator. In 24 hours, my Ph.D. topic was born!

This leads us to 1 November 1994 when I presented my dissertation proposal, which included as an example research artifact my Mr. Copyright(tm) prototype --- quickly re-named at the urging of my committee and others to LicensIt(tm). LicensIt demonstrated in the form of a easy-to-use, desktop "appliance" the key ideas of (a) binding actionable copyright metadata to multimedia objects, and (b) user-friendly, real-time, networked copyright registration. The LicensIt desktop icon said it all: modeled after the famous Stuffit(tm) coffee grinder, users dragged and dropped their content (initially GIF files) onto LicensIt; a dialog popped up to collect (and display) their descriptive and other metadata and to enable them to select their "registration server" from a menu of choices; their work was registered. By way of both the static metadata and the registry, users would be able to contact the principals involved in the creation of the item. I envisioned several other options, including registering digital signatures to allow users to authenticate a work in hand, as well as enveloping the work in an encrypted envelope.

It is important to note that the focus of my work at that time was on enabling copyright by binding static and dynamic metadata to content and especially to make it as accessible as possible within the context of use; content security was only a secondary concern. "Enablement" means that although a desktop client is interesting, plugins for creation tools like Photoshop, Acrobat and Macromedia Director, and enjoyment tools like Mosiac --- this was 1994!! --- would be infinitely more interesting and useful! I assumed that one day, creators would be mixing and matching content found around the web, and at least commercial and other highly visible producers would want/need to "do the right thing" w.r.t. copyright and thus would benefit from instantly accessible attribution, bound to the item. Note that I was heavily influenced at that time by the writings of Prof. Henry H. Perritt, Jr. whose concept of permissions headers was not only an inspiration for me, but I believe anticipated Creative Commons licensing templates.

Fifteen years later, we can at least say the world is different! The world we imagined 15 years ago of rampant "re-mixing" of content has arrived; licensing models such as Creative Commons have improved awareness; but still the infrastructure does not accommodate the discovery and transmission of rights information as readily as it should. With the rise of new data-centric models such as Linked Data (a practical outcome of Semantic Web research) and the acceptable of persistent identifier systems including the Handle System and the Digital Object Identifier, we're getting there...

Next installment: The NetRights and YRM years...