Mike Linksvayer

The $300 “ultra deluxe edition” of Nine Inch Nails‘ Ghosts I-IV, limited to 2500 copies, sold out in a couple days (I believe released Sunday, no longer available this morning). There are some manufacturing costs, but they don’t appear to be using any precious materials. So if an artist typically makes $1.60 on a $15.99 CD sale, profit from sales of the limited edition already matches profit from a CD selling hundreds of thousands of copies.

Then there are non-limited sales of a $75 merely “deluxe edition”, $10 CD, and $5 download, and whatever other products NIN comes up with around Ghosts.

The ultra deluxe success seems to me to validate the encouragement by some to pursue large revenue from rabid fans and collectors willing and able to pay for personalization, authenticity, embodiment, etc., rather than attempting to suppress zero cost distribution to the masses.

Speaking of distribution, click on the magnet to search for a fully legal P2P download of Ghosts, assuming you have the right filesharing software installed.

nin_ghosts_I-IV_mp3.zip (283.7 MB)

Two 1,200 foot towers are planned for the northwest corner of 1st and Mission Streets in San Francisco, site of a few run down buildings, one of which Bitzi had offices in for most of 2001 (spruced up some since then). Will San Francisco planners allow rapacious developers to destroy history? I hope so. Onward to Sanhattan!

Via SF Cityscape.

Just noticed that the current LimeWire beta (4.11.0) includes optional copyright filtering. See the features history and brief descriptions for users and copyright owners:

In the Filtering System, copyright owners identify files that they don’t want shared and submit them for inclusion in a public list. LimeWire then consults this list and stops users from downloading the identified files “filtering” them from the sharing process.

If you sign up for an account as a copyright owner you can submit files (with file name, file size, SHA1 hash, creator, collection, description) for filtering. Users can turn the filter on and off via a preference.

LimeWire.org now features a blog with pretty random content. I notice that another PHP Base32 function (which makes a whole lot more sense than the one included in Bitcollider-PHP — I swear PHP’s bitwise operators weren’t giving correct results and worked around that, but was probably insane) is available with a hint that someone is building an “open source Gnutella Server in PHP5.”

Remember that LimeWire is Open Source P2P and thus pretty trustworthy — and you can always fork.

Congratulations to MusicBrainz for taking care of a longstanding substandard feature — a proprietary and not very scalable acoustic fingerprinting technology (Relatable TRM). Today MusicBrainz announced integration with MusicIP’s MusicDNS fingerprinting service, full details in the announcement.

Funny thing, I just cleared all the (old, mostly gathered in 2001) TRM tags from Bitzi a couple weeks ago.

Creative Commons license tracking is also now enabled at both MusicBrainz and MusicIP, no doubt more on that at the CC weblog in the near future.

Belated congratulations to MusicBrainz for signing their first commercial deal in January.

I wrote some about MusicBrainz about 15 months ago. I predict the next 15 months will be very good for what I’ll call “open music infrastructure.”

On the SXSW06 Tagging 2.0 panel Thomas Vander Wal just cited Bitzi as (more or less) a non-successful predecessor to Tagging 2.0 applications, saying something like “things like Bitzi (mumble) Cory Doctorow called metacrap.”

Vander Wal recently explained in a comment at Joho the Blog:

The big thing that was different, from say Bitzi, was people tagging information in their own vocabulary for their own reuse. Tagging information for others as a priority seems to make it far less accurate as a person may not understand the terms they are using (well understand them as other may).

He’s right. There’s too little private benefit to “tagging” at Bitzi, largely because what interfaces to what you have individually contributed are lame to the extent they exist. The Bitzi use case is rather different from del.icio.us and Flickr but it can learn a lot from them.

This year Gordon Mohr had the devious idea to do preemtive reviews of CodeCon presentations. I’ll probably link to his entries and have less to say here than last year.

Daylight Fraud Prevention. I missed most of this presentation but it seems they have a set of non-open source Apache modules each of which could make phishers and malware creators work slightly harder.

SiteAdvisor. Tests a website’s evilness by downloading and running software offered by the site and filling out forms requesting an email address on the site. If virtual Windows machine running downloaded software becomes infected or email address set up for test is inundated with spam the site is considered evil. This testing is mostly automated and expensive (many Windows licenses). Great idea, surprising it is new (to me). I wonder how accurate evil readings one could obtain at much lower cost by calculating a “SpamRank” for sites based on links found in email classified as spam and links found on pages linked to in spams? (A paper has already taken the name SpamRank, though at a five second glance it looks to propose tweaks to make PageRank more spam-resistant rather than trying to measure evil.) Fortunately SiteAdvisor says that both bitzi.com and creativecommons.org are safe to use. SiteAdvisor’s data is available for use under the most restrictive Creative Commons license — Attribution-NonCommercial-NoDerivs 2.5.

VidTorrent/Peers. Streaming joke. Peers, described as a “toolkit for P2P programming with continuation passing style” I gather works syntactically as a Python code preprocessor, could be interesting. I wish they had compared Peers to other P2P toolkits, e.g., JXTA.

Localhost. A global directory shared with a modified version of the Azureus BitTorrent client. I tried about a month ago. Performance was somewhere between abysmal and nonexistent. BitTorrent is fantastic for large popular files. I’ll be surprised if localhost’s performance, which depends on transferring small XML files, ever reaches mediocrity. They’re definitely going away from BitTorrent’s strengths by uploading websites into the global directory as lots of small files (I gather). The idea of a global directory is interesting, though tags seem a more fruitful navigation method than localhost’s hierarchy.

Truman. A “sandnet” for investigating suspected malware in. Faux services (e.g., DNS, websites) can be scripted to elicit the suspected malware’s behavior, and more.

The wily Lucas Gonze is at it again, defining ‘lightnet’ and ‘darknet’ by example, without explanation. The explanation is so simple that it probably only subtracts from Gonze’s [re]definition, but I’ll play the fool anyhow.

Usually darknet refers to (largely unstoppable) friend-to-friend information sharing. As the name implies, a darknet is underground, or at least under the radar of those who want to prohibit certain kinds of information sharing. (A BlackNet doesn’t require friends and the radar doesn’t work, to horribly abuse that analogy.)

Lightnet, as far as I know, is undefined in this context.^*

Anyway, Lucas’ definition-by-example lumps prohibited sharing (friend to friend as well as over filesharing networks) and DRM together as Darknet. Such content is dark to the web. It can’t be linked to, or if it can be, the link will be to a name,^** not a location, thus you may not be able to obtain the content (filesharing), or you won’t be able to view the content (DRM).

Lightnet contnet is light to the web. It can be linked to, retrieved, and viewed in the ways you expect (and by extension, searched for in the way you expect), no law breaking or bad law making required.

^* Ross Mayfield called iTunes a lightnet back in 2003. Lucas includes iTunes on the dark side. I agree with Lucas’ categorization, though Ross had a good point, and in a slightly different way was contrasting iTunes with both darknets and hidebound content owners.

^** Among other things, I like to think of magnet links and Bitzi as attempting to bridge the gap between the web and otherwise shared content. Obviously that work is unfinished. As is making multimedia work on the web. I think that’s the last time I linked to Lucas Gonze, but he’s had plently of crafty posts between then and now that I highly recommend following.

At SXSW today Eric Meyer gave a talk on Emergent Semantics. He humorously described emergent as a fancy way of saying grassroots, groundup (from the bottom or like ground beef), or evolutionary. The talk was about adding rel attributes to XHTML <a> elements, or the lowercase semantic web, or Semantic XHTML, of which I am a fan.

Unfortunately Eric made some incorrect statements about the uppercase Semantic Web, or RDF/RDFS/OWL, of which I am also a fan. First, he implied that the lowercase semantic web is to the Semantic Web as evolution is to intelligent design, the current last redoubt of apolgists for theism.

Very much related to this analogy, Eric stressed that use of Semantic XHTML is ad hoc and easy to experiment with, while the Semantic Web requires getting a committee to agree on an ontology.

Not true! Just using rel="foo" is equivalent to using a http://example.com/foo RDF property (though the meaning of the RDF property is better defined — it applies to a URI, while the application of the implicit rel property is loose).

In the case of more complex formats, an individual can define something like hCard (lowercase) or vCard-RDF (uppercase).

No committee approval is required in any of the above examples. vCard-RDF happens to have been submitted to the W3C, but doing so is absolutely not required, as I know from personal experience at Bitzi and Creative Commons, both of which use RDF never approved by committee.

At best there may be a tendency for people using RDF to try to get consensus on vocabulary before deployment while there may be a tendency for people using Semantic XHTML to throw keywords at the wall and see if they stick (however, Eric mentioned that the XFN (lowercase) core group debated whether to include me in the first release of their spec). Neither technology mandates either approach. If either of these tendencies to exist, they must be cultural.

I think there is value in the ad hoc culture and more importantly closeness of Semantic XHTML assertions to human readable markup of the lowercase semantic web and the rigor of the uppercase Semantic Web.

It may be useful to transform a rel="" assertions to RDF assertions via GRDDL or a GRDDL-inspired XMDP transformation.

I will find it useful to bring RDF into XHTML, probably via RDF/A, which I like to call Hard Core Semantic XHTML.

Marc Canter as usual expressed himself from the audience (and on his blog). Among other things Marc asked why Eric didn’t use the word metadata. I don’t recall Eric’s answer, but I commend him for not using the term. I’d be even happier if we could avoid the word semantic as well. Those are rants for another time.

Addendum: I didn’t make it to the session this afternoon, but Tantek Çelik’s slides for The Elements of Meaningful XHTML are an excellent introduction to Semantic XHTML for anyone familiar with [X]HTML.

Addendum 20050314: Eric Meyer has posted his slides.

Here you’ll find a little PHP API that wraps the single file metadata extraction feature of Bitzi’s bitcollider tool. Bitcollider also can submit file metadata to Bitzi. This PHP API doesn’t provide access to the submission feature.

Other possibly useful code provided with Bitcollider-PHP:

• function hex_to_base_32($hex) converts hexidecimal input to Base32.
• function magnetlink($sha1, $filename, $fileurl, $treetiger, $kzhash) returns a MAGNET link for the provided input.
• magnetlink.php [file ...] is a command line utility that outputs MAGNET links for the files specified, using the bitcollider if available (if not kzhash and tree:tiger are not included in MAGNET links).

Versions of this code are deployed on a few sites in service of producing MAGNET links or urn:sha1: identifiers for RDF along these lines, both in the case of CC Mixter.

Criticism welcome.

Two bloggers on using Bitzi.

Neil Turner recommends “Wonderful Life” by Ordinary People feat. Tina Cousins. Apparently the track is hard to find. Having lost a copy once due to disk troubles, Neil submitted the file to Bitzi. Now others can use the Bitzi ticket to find the file he recommends (and Neil will have an easier time in the event of another storage failure). I was pleasantly surprised to find that after a few tries I could download the exact file from two Gnutella hosts. Unfortunately dance music and electronica aren’t my thing.

Fareed of Cairo and “survivor of a car crash” writes about checking file integrity:

Have you ever downloaded files and you were not sure of it’s integrity. Now their is a way to be sure, a site called Bitzi allows you to check file integrity. Users can submit using a program called Bit Collider file hashes to the site or verify that the integrity is ok.

So Bitzi can help identify good and bad files. Good luck avoiding future hard disk and car crashes!

Continuation of MusicBrainz Discovery (I).

One notable thing about MusicBrainz is that Rob Kaye and a small number of core developers and supporters have pursued a consistent vision for roughly six years with very little funding or even understanding outside this small group. It isn’t easy to really “get” MusicBrainz (I think it took me two years), though I think that at some point in the next few years everyone will “get” MusicBrainz more or less all at once.

If you’re a geek it’s hard not to get hung up on MusicBrainz use of acoustic fingerprint-based technology. Acoustic fingerprinting is fragile in three ways — it is subject to false positives and false negatives, there is no open source implementation of the concept, and the technology MusicBrainz uses, Relatable TRM, is proprietary and requires a centralized server. Indeed, many of the technology questions at Tuesday’s music metadata panel concerned acoustic fingerprints.

It is important to understand that while MusicBrainz uses acoustic fingerprints, it does not rely on them. TRM matching is just one mechanism for track identification. File metadata included in (e.g., ID3 tags) or with (filename) the file can and I believe are used to match existing records, as could track duration and file hashes (see if Bitzi or a P2P network has any metadata for the file in question). Additionally, file identification is only one component of MusicBrainz.

If you’re not a geek, you won’t notice acoustic fingerprints, because you wouldn’t, and because you’re not likely to get that far. So what the heck does MusicBrainz do? Here’s an attempt:

• MusicBrainz can organize your music collection. Download the tagger.
• MusicBrainz uniquely identifies artists, albums, and songs, facilitating rich and precise music applications, all on a level playing field.
  • Not at all speculative potential: include a MusicBrainz song identifier in a blog post, cover art (with your Amazon referrer of course) automagically appears in blog post, blog aggregator publishes top n lists and personalized recommendations.
  • Another: publish a playlist of MusicBrainz identifiers and others can recreate the experience so defined with no file transfer involved.
  • There are several others, some that could be offered by MusicBrainz itself, outlined in MusicBrainz tomorrow. I have to quote one because it’s fun:
    

    Music Genealogy: MusicBrainz may keep track of which artists/performers/engineers contributed to a piece of music, and when these contributions took place. Combining this contribution data with data on how artists influenced each other will create a genealogy of modern music. Imagine being able to track Britney Spears back to Beethoven!

• The MusicBrainz database, created by the community, will remain free, unlike others.

Having been around for awhile, MusicBrainz has run into many of the technical and social problems inherent in music metadata and an evolving community website, and produced much good documentation on solutions, realized and potential. Here’s a sampling:

• Data License Issues and the solution.
• Style Guidelines — you wouldn’t imagine how hard good capitalization rules are, for starters. Ugh. Don’t even attempt to think about Classical metadata!
• Cross-platform desktop app, using wxPython, same as Creative Commons.
• How to identify a compact disc.
• Tracking relationships among tracks, albums, artists, and much more.
• Survival of the fittest and steps towards survival of the fittest — how do you take some wiki lessons and apply to a large and highly structured database and featureset?
• MusicBrainz Metadata Initiative — web services and RDF, hand rolled.
• MusicBrainz Non-Profit White Paper, already linked above this list is very interesting on several subjects.
• How the server works.
• Everyone loves stats.

By the way, as of Wednesday MusicBrainz has a blog.

Earlier this evening I gave a brief introduction (slides PDF) to MusicBrainz at SDForum’s Emerging Technology SIG meeting on music metadata in the stead of MusicBrainz founder and leader Rob Kaye, who couldn’t make it up to Palo Alto. (I’m fairly familiar with MusicBrainz, having worked with Rob at Bitzi and getting updates when we cross paths in this small world.)

If I could pick a theme for the meeting (which included two other very interesting speakers — Stephen Bronstein of the Independent Online Distribution Alliance and David Marks of Loomia), and for recent months in general, it would be that in case you haven’t noticed, it’s clearly now a discovery problem, not a delivery problem.

SIG leader William Grosso led off with some quotes from the much-discussed Wired magazine article The Long Tail, which seems to have captured this zeitgeist. (Grosso also had a novel to me presentation technique — a slideshow of potentially relevant slides plays while he speaks, and if a slide happens to be relevant to the current sentence, he uses the slide to augment the point. Is there a name for this?)

Obviously there was tremendous interest in Creative Commons in this context, and several people seemed to be happy to learn of CC’s search engine and the great services and products offered by the Internet Archive (free hosting for CC-licensed audio and video, built in format conversion), Magnatune (all CC-licensed music label) and more.

Unfortunately in the eleven years I’ve been in the SF bay area I only definitively recall attending two previous SDForum events — a 1994 talk by Atari Jaguar developers in San Jose and in 2001 an evening with Phil Zimmermann in San Francisco (I suspect others who were there would deem the “an evening with” cliche appropriate in this case). This evening’s meeting was a total geekfest. I hung around for well over an hour commiserating on all manner of software development topics (I think that’s what “SD” stands for) with a number of hardcore geeks (no whatever-Dilbert’s-boss’s-name-is there) while two guys were lauging their asses off whiteboarding issues with Unicode encoding (as far as I could tell). I’ll have to go back.

More about what I’ve learned about MusicBrainz over the years and in preparing for the evening in a future post.

Update: part 2

Morpheus, a popular filesharing client at version 4.5:

Includes free Bitzi anti-spoofing look-ups to download only the files you want.

That’s an accurate description (and has been true since at least Morpheus 4.1.1). If you see something you’re interested in downloading in search results, you’re a right-click away from a Bitzi lookup. If Bitzi users have judged the file to be spam, virus-laden, or corrupted, you’ll get a response similar to this:

Thirty Bitzi users have judged this file as dangerous or misleading (one bozo recommended the file). A few of the judgement notes tell the story:

Every time I write something,it will come up in my searching files. The size is 105,2 kb every time.

and

stupid isaveclub.com spam, coming from 69.44.152.159:255….discuises itself under many wma files

If Bitzi users have judged the file you’re interested in to be worthy, you’d see something similar to this:

That file happens to be the current release for Windows of Bitzi’s bitcollider file metadata collection tool.

Derek Slater notes that Bitzi metadata is itself subject to spoofing, and

even if Bitzi helps people sort out spoofs, the technological arms race will continue.

Very true. Bitzi is dependent upon community policing, and a concerted effort to create dangerous files and submit fraudulent judgements to Bitzi would work, at least for awhile. There are steps Bitzi can take to militate against such attacks should they become a problem. Unfortunately, as I noted recently, development proceeds at a bear in winter’s pace.

It should also be noted that Morpheus is just one of several applications that enable Bitzi lookups or submissions, though most of these send users to a Bitzi web page rather than integrating raw data from Bitzi lookups into their user interface as Morpheus has (see screenshots above).

After over three and a half years I am finally the best bitizen, as defined by a formula that takes into account the amount of file metadata contributed to Bitzi and the quality of that metadata, as rated by other “bitizens” (Bitzi users):

How did I obtain this dubious (as a Bitzi cofounder) honor?

I’ve more or less consistently reported some of the random junk I may have encountered (though around 30 Bitzi users have reported more, all over a shorter active period) and more importantly, have occasionally taken care to add accurate metadata to reports.

On the negative side, I’ve more or less consistently failed to put much effort into adding new features since Bitzi went into hibernation. If I had, doubtless many more prolific than I (nearly everything I download is from the web — file sharing networks, especially post-Napster (really post-AudioGalaxy), are still practically useless in my estimation, unless you have lots of time to kill, i.e., you’re a bored teenager) would’ve stuck around and I’d be nowhere near numero uno.

Download the bitcollider (file metadata reporting tool) and knock me off my throne!

Ed Felter spreads a SHA-1 hash collision rumor.

Eric Rescorla does a good job of explaining that even if true, a collision is not of great practical import.

Why? Most uses of SHA-1, including Bitzi, rely on the practical impossibility of finding content that will generate a specific hash.

A collision merely means that two pieces of content (”messages” in crypto-speak) have been found that generate the same arbitrary hash.

For reasons that aren’t all that intuitive, it is much harder to find a specific match than an arbitrary collision.

I think in day-to-day experience a good analogue would be this: it’s pretty easy to find odd coincidences if you look. If you know how to conjure up specific odd coincidences on demand, tell me.

All that said, Bitzi also uses the Tiger hash, which is not from the same family as SHA-1, as an insurance policy among other things.

Disclaimer: I am not a crypto expert. If true this rumor may be huge news for crypto theorists.

Limacat discovers and understands Bitzi, calls it “the most important stuff” (um, found in one browsing session) and “it is possible that partial support for Bitzi will land in PMX.” Limacat didn’t provide a link for PMX, so I had to dig: Ladies and Gentlemen: Personal Metadata Exchanger!. Looks like vapor so far.

Slyck reports on a major DirectConnect upgrade. DirectConnect hasn’t seen much interest from the press or technologists, but it does have a significant userbase, with 215,880 users currently online according to the Slyck home page, slightly smaller than Gnutella’s 234,618. I have no idea how Slyck obtains those numbers.

Anyhow, it is good to see that DirectConnect has adopted file hashing, specifically THEX (Tree Hash EXchange) using the Tiger hash. This allows DirectConnect clients to find exact alternate download sources and to verify downloads as they progress and opens to door to future MAGNET and Bitzi lookup support.

Here’s a MAGNET link that includes the tiger tree root (second component of the bitprint) and a corresponding Bitzi info lookup by urn:tree:tiger.

Jay-Z_Construction_Set.zip (631.9MB)

Announced today. Copied from the Creative Commons home page:

We’re very happy to announce the winners of the GET CREATIVE! Moving Images Contest: First Place goes to Justin Cone, for the inspired and powerful short film “Building on the Past,” which uses all sorts of Prelinger Archives footage to great effect. Second Place: Sheryl Seibert, for “Mix Tape,” which perfectly captures the found-art ethos of Creative Commons and uses the Creative Commons-licensed song “Mix Tape” by Jim’s Big Ego. Third Place: Kuba and Alek Tarkowski, for “CCC,” a historical look at free culture. Check them out, download them, mirror them, share them with friends. Thanks to all of you who made submissions!

The first place entry is really good, though my favorite scene is midway through the third placer — “a mutation of the system, if you will.”

MAGNET/Bitzi links for easy sharing and info:

Justin_Cone_-_Building_On_The_Past.mov (7.0MB)

Sheryl_Seibert_-_Mix_Tape.mov (31.8MB)

Kuba_and_Alek_Tarkowski_-_CCC.mpg (14.9MB)

Pleasant Blogger says “Bookcrossing + Orkut + Bitzi + HotOrNot = Mediachest“.

I don’t think addition is the correct operation. Perhaps we could say all of

• Bookcrossing ∩ Mediachest ≠ ∅
• Orkut ∩ Mediachest ≠ ∅
• Bitzi ∩ Mediachest ≠ ∅
• HotOrNot ∩ Mediachest ≠ ∅

or something more interesting if I actually knew set theory and notation.

Seriously though, lots of people realize that social networks can facilitate navigation, discovery, trust, filtering, communication and the like in many domains. Will people move on from sites that encourage building lists of “friends” for the sake of building such lists (and dating, I hear) to sites that use social networks to enhance other functions a la Mediachest or will the likes of Friendster and Orkut add more utility? Probably something else. Consider that

• Orkut has only scratched the surface of what a pure social networking service could offer. There are no collaborative filtering or recommendation features for starters. I don’t think Orkut is near an 80/20 sweet spot, or wherever diminishing returns set in for a pure social networking site.
• Sites with huge existing memberships haven’t added social networking to their offerings. Friends.yahoo.com does not exist.
• I’m forgetting stuff, but not the decentralized path. See FOAF and XFN. Atop which every value-add you can imagine (a miniscule subset of the total) will be built in the semi-near future, like by 2009.

Bitzi has had a very simple social network feature since May, 2001, “interesting bitizens”. Mine (and those interested in me) are currently listed on the right side of my bitizen page. We still haven’t built any features using these relationships, apart from an ignored popularity contest. Eventually. Before 2009.

As posted on the Creative Commons weblog:

The source materials for both “Get Creative” and “Remix Culture” are now available. Download the .fla file for either and you can get creative and remix “Get Creative” or “Remix Culture” with ease.

Quicktime versions of both movies are also now available. Now it’s easier than ever to download, display and share “Get Creative” and “Remix Culture” (right-click on links to download and save).

Thanks to Ibiblio for hosting all of these files. The Quicktime movies are also available at the Internet Archive here and here. The Internet Archive will also host your Creative Commons-licensed movies and music free of charge. Get started.

MAGNET and Bitzi links:

Creative_Commons_-_Get_Creative.fla (46.1MB)

Creative_Commons_-_Get_Creative.mov (6.9MB)

Creative_Commons_-_Get_Creative.swf (5.5MB)

Creative_Commons_-_Remix_Culture.fla (94.9MB)

Creative_Commons_-_Remix_Culture.mov (6.8MB)

Creative_Commons_-_Remix_Culture.swf (6.8MB)