Yesterday on the Creative Commons weblog:

Today we announce a search engine prototype exclusively for finding Creative Commons licensed and public domain works on the web.

Indexing only pages with valid Creative Commons metadata allows the search engine to display a visual indicator of the conditions under which works may be used as well as offer the option to limit results to works available licenses allowing for derivatives or commercial use.

This prototype partially addresses one of our tech challenges. It still needs lots of work. If you’re an interested developer you can obtain the code and submit bugs via the cctools project at SourceForge. The code is GNU GPL licensed and builds in part upon Nathan Yergler’s ccRdf library.

We also have an outstanding challenge to commercial search engines to build support for Creative Commons-enhanced searches.

And it hasn’t melted down yet.

Ben Adida wrote most of the code that needed to be written in Python (not much — PostgreSQL with tsearch2 full-text indexing does all of the heavy lifting). Former government employee Justin Palmer wrote an earlier prototype in AOLserver/Tcl, also using PG/tsearch2. (Turns out we needed the flexibility of running under Apache. I’ll miss AOLserver/Tcl when I last touch it, but I’ll also be glad to be rid of it.) I did a PHP hack job on the front end, and Matt Haughey made it look good (for end users, not code readers) in a matter of minutes.

Although everything possible sucks about this implementation, it is already a valuable tool for finding CC-licensed and public domain content — stuff you can reuse with permission already granted. Neeru Paharia was the visionary here, seeing that it would be valuable even if it sucked in every way technically.

Stephen Downes is exactly right about the long term goal:

Of course, this is only a step – such a search engine would not be useful for many purposed; copyright information needs, in the long run, to define a search field or a type of search, not a whole search engine.

With great justification major search enginges have ignored pure metadata for a long time, at least five years. Pure metadata, with no visibility, is nearly universally ill-maintained or fraudulent. I hope that this Creative Commons prototype inspires some people at major search engines to think again about metadata, but I think semantic HTML is what will finally prove useful to such folks, in no small part because it isn’t pure metadata. I’ll post on incremental semantic search engine features in the near future.

Tantek �elik comments on Creative Commons use of rel="license", citing my small-s semantic web and CC post to cc-metadata (reproduced below).

I agree with Tantek’s comments, though I wouldn’t advise removing admittedly ugly and potentially redundant RDF-in-HTML-comments, at least not until mozCC and ccRdf and consequently some dependent code go case insensitive.

There are currently at least two Creative Commons metadata cases where a simple rel="license" attribute won’t do, requiring RDF:

• The current resource needs to make statements about other resources, e.g., when those resources aren’t adept at carrying their own metadata or staying in one place.
• One wants to include rich metadata, for example concerning a creator or rights holder, though that particular example I suppose cries out for rel="tipjar" where no further requirements are present.

In my view, metadata-enabled web tools will do well to include a RDF model layer, whether the statements be gleaned from semantic [X]HTML, parsed from human language, or mainlined from some RDF encoding, and whatever the tool’s internal knowledge representation. Content creators will do well to produce the simplest, most utilitarian metadata possible.

I’m turning a bit sour on the phrase “lowercase semantic web”. I like semantic [X]HTML. I like RDF. All in the service of our near-term goals. All in the service of the Semantic Web, which will surely be a superset of the RDF web. I dig real world semantics in any case.

As I mentioned
<http://lists.ibiblio.org/pipermail/cc-metadata/2003-December/000243.html>
I find "semantic HTML" very interesting — it keeps the metadata close
to the presentation, militating against "metacrap" and can be used to
populate the big-S Semantic Web through RDF generation.

Since then the RDF-in-XHTML proposal that builds on semantic HTML has
moved ahead and generalized, see <http://www.w3.org/2004/01/rdxh/spec>
"Gleaning Resource Descriptions from Dialects of Languages" is a pretty
good description.

Also, Kevin Marks and Tantek Celik headed up a very nice BoF at Etech
<http://wiki.oreillynet.com/etech/hosted.conf?RealWorldSemantics> in
which they discussed current small-semantic web implementations. See
that URL for some good links.

Largely, people are using the "rel" attribute of "a" elements
<http://www.w3.org/TR/xhtml1/dtds.html#dtdentry_xhtml1-strict.dtd_a>
to describe "the relationship from the current document to the URI
referred to by the element. The value of this attribute is a
space-separated list of link types."

<http://www.w3.org/TR/xhtml2/mod-attribute-collections.html#col_Hypertext>
(I can’t find the equivalent documentation for XHTML1, but rel is
supported, per the DTD above).

A neat thing on the presentation side is that CSS selectors can actually
change the document rendering based on rel attributes — making the
metadata not just close to the presentation, but part of it.

Anyway, a rel attribute on anchors removes the big problem with assuming
that a link to a license indicates that a page is available under that
license — the page could be linking to the license for any reason. <a
rel="license" href="http://creativecommons.org/licenses/by/1.0/"/> on
the other hand, is no more ambiguous than the following RDF snippet

<Work about="">
<license rdf:resource="http://creativecommons.org/licenses/by/1.0/"/>
</Work>

and can be used to generate the same.

The upshot is that I’m planning to recommend adding a rel="license"
attribute to links to CC licenses where the license applies to the
current page, have <http://creativecommons.org/license/> spit that out,
and encourage other apps to support the same.

Note that this is all entirely complementary with RDF. All apps should
continue to use/generate/support RDF, and RDF is required for making
license (or any metadata) statements about resources other than the
enclosing one.