Archive for November, 2005

Redefining light and dark

Monday, November 28th, 2005

The wily Lucas Gonze is at it again, defining ‘lightnet’ and ‘darknet’ by example, without explanation. The explanation is so simple that it probably only subtracts from Gonze’s [re]definition, but I’ll play the fool anyhow.

Usually darknet refers to (largely unstoppable) friend-to-friend information sharing. As the name implies, a darknet is underground, or at least under the radar of those who want to prohibit certain kinds of information sharing. (A BlackNet doesn’t require friends and the radar doesn’t work, to horribly abuse that analogy.)

Lightnet, as far as I know, is undefined in this context.*

Anyway, Lucas’ definition-by-example lumps prohibited sharing (friend to friend as well as over filesharing networks) and together as Darknet. Such content is dark to the web. It can’t be linked to, or if it can be, the link will be to a name,** not a location, thus you may not be able to obtain the content (filesharing), or you won’t be able to view the content (DRM).

Lightnet contnet is light to the web. It can be linked to, retrieved, and viewed in the ways you expect (and by extension, searched for in the way you expect), no law breaking or bad law making required.

* Ross Mayfield called iTunes a lightnet back in 2003. Lucas includes iTunes on the dark side. I agree with Lucas’ categorization, though Ross had a good point, and in a slightly different way was contrasting iTunes with both darknets and hidebound content owners.

** Among other things, I like to think of magnet links and as attempting to bridge the gap between the web and otherwise shared content. Obviously that work is unfinished. As is making multimedia work on the web. I think that’s the last time I linked to Lucas Gonze, but he’s had plently of crafty posts between then and now that I highly recommend following.

Machine learning patterns

Sunday, November 27th, 2005

I first heard of the Silicon Valley Patterns meetings from Alex Chafee a few years ago while participating in his “bootstrap” practice group. SVP sounded like fun, but I only got around to attending a meeting this spring, a one-off on led by Johannes Ernst (notes). I was going to write something about that meeting, but just can’t get worked up about digital identity.

SVP’s next extended track was on , a topic I have some interest in and very cursory knowledge of from reading popular books on AI. The track lasted from May through October. Mostly our study was guided by Andrew Moore’s statistical data mining tutorials, with occasional reference to Russell & Norvig.

I don’t think any of the regular attendees were machine learning experts, but with occasional contributions from everyone, I think everyone was able to increase their knowledge of the material. Overall a gratifying method of learning, though not a perfect substitute for lecture.

My secondary take way from the track was that I need a serious brush up on calculus and statistics, neither of which I’ve studied, and barely used, in fifteen years. I’m working on that.

The current SVP track should be very different–hands on Ruby on Rails practice. I’m attempting to justify putting in the time…

Online IQ test scam

Saturday, November 26th, 2005

Jesus H. Christ, also known as James Christian, has a 144 score, a certificate of intellectual achievement, and is sending away for information from a diploma mill. Let me explain…

Someone posted to a mailing list I’m on the retarded suggestion that everyone on the list take the “Classic IQ test (NB rel=”nofollow”, and that is my suggestion to humans as well).

I recalled reading recently that the maximum IQ according to this “most thorough and scientifically accurate IQ Test on the Web” is 144. Like an idiot, I (or rather “Jesus H. Christ”) took the test, and got 144. No accomplishment–there are no hard questions on the test, unless you happen to be bad at remembering cliches.

I of course declined their offer to pay for an “in-depth analysis” of my IQ score, but today I received an email from Tickle tests:

Jesus, As a top-scorer on Tickle’s IQ Test, the in-depth analysis of your IQ score is FREE.

FREE! How could I pass that up? Well, I had to click past around two dozen “free offers” to get to my “free analysis.” Many of the offers were from diploma mills. I filled one form out in a completely misplaced hope to short circuit the process (I had to click past several more before getting to my analysis). They do make an effort to validate leads. I had to provide a real address, and they would not take “Jesus Christ” as a name.

James Christian of Hayes Valley, San Francisco, your PhD information packet is on its way.

The “in-depth analysis” when finally obtained, proved to be ever so slightly interesting, and not because it told me anything about my, ahem, intellect. The analysis says that a scoring 144 on the Tickle test corresponds to an IQ of between 134 and 144. So, even if that range is accurate (and there’s no reason to think it is), Tickle gives people a feel-good top of the range number. The analysis also lists purported average scores by U.S. state, ranging from 110 for Mississippi and West Virginia to 115 for the District of Columbia. Supposedly this test has been taken over 30 million times, so I’d expect averages to be much closer to 100, if the test were at all meaningful.

I found an abstract for one small study of online IQ tests:

This double-blind study utilized 60 participants placed into one of three equally distributed categories according to their composite ACT/SAT scores. Comparisons of the Reynolds Intellectual Assessment Scales (RIAS) with internet IQ tests from,, and showed low to modest correlations, questioning the overall validity of web IQ tests.

One of the investigators has another abstract in these proceedings (p. 23) titled Web-Based IQ Tests: A Concept Whose Time Has Not Yet Come.

Here’s Jesus H. Christ’s certificate of intellectual achievement:

Maybe after James obtains a degree he can set up his own PhD certified testing service.

Update 20051128: A post from the person who sent the hey-let’s-everyone-take-the-tickle-iq-test suggestion to a list I’m on. (By the way, calling the suggestion “retarded” was a joke, mostly.)

Democracy and Decision: The Pure Theory of Electoral Preference

Friday, November 25th, 2005

Democracy and Decision, a 1993 book by economist/philosopher and political philosopher , undermines a relatively little known (to me) side of –the assumption that voters vote in accordance with their () interests.

The authors make a convincing case that because an individual voter is essentially never decisive, the rational voter will vote expressively, even if the vote that gains the voter the highest expressive value would be against the voter’s instrumental interests, if the voter were decisive. The authors summarize their proposition as “Rational action ⇏ psuedorational voting.”

The following rendition of Table 2.2. Electoral choice as a quasi-prisoners’ dilemma (p. 28) illustrates a simple case where voters will vote according to their expressive values and against their instrumental values, as their probability of casting a decisive vote approaches nil.

  All others
Each Majority for a Majority for b Tie (probability → 0)
Vote for a 5 105 5
Vote for b 0 100 100

The authors make a reasonable case that voters’ instrumental and expressive values often are divergent. War seems to be a particularly strong case (p. 50):

How is it, then, that such mammoth exercises in irrationality seem to have been pursued so vigorously and with such popular enthusiasm in this most democratic of ages? The voters’ dilemma provides a possible explanation. Consider the individual voter contemplating a vote between competing political candidates in a setting where international relations are tense. One candidate offers a policy of appeasement, recognizing the enormous cost in lives and resources that any antagonistic stance might involve. the other candidate stands for national integrity — “By God, we are not going to be pushed around by these bastards.” We might well presume that few voters, making a careful calculation of the costs and benefits to themselves and those they care about, would actually opt for war. Just as individuals, in situations of interpersonal strain, will often swallow their pride, shrug their shoulders, and stroll off rather than commit themselves to an all-out fight (particularly one that might imply someone’s death), so the interest of most voters would be better served by drawing back from the belligerent course. Yet a careful reflective computation of the costs and benefits of the alternative outcomes to herself (and those others relevant to her concerns) is precisely what the voter does not entertain: Any such computation is essentially irrelevant. What is relevant, we might suppose, is the opportunity to show one’s patriotism, one’s antipathy to servility, one’s strength of national purpose.

Of course expressive preferences may be for peace instead. In either case, and for any issue, the main point is that “it will be the symbolic power of the policy rather than the costs and benefits the policy scatters on particular voters that will be most relevant.” (p. 51, emphasis in original)

A chapter is devoted to the probability that a vote is decisive–roughly speaking, the probability an election is decided by one vote, given an odd number of votes. It turns out the calculation of this probability is not straightforward, but any reasonable attempt seems to result in an infinitesimal value.

and widespread belief in the argument against voting for minor party candidates would seem to indicate that voters do not vote expressively (surely the proportion of voters who could increase their expressive returns by voting for a “third party” candidate is higher than the roughly one percent who actually do so in U.S. presidential elections). However, at least four non-instrumental factors explain strategic voting: established parties have economies of scale in advertising, rationally habitual voting, voting for a candidate’s top competitor may give the highest expressive returns if a voter’s primary expresive desire is to “boo” the candidate, and being seen as voting “responsibly” is itself an expressive return.

One possibility I believe the authors do not address is that voters may irrationally believe there is a significant probability that their votes may be decisive. After all, the probability calculation is not obvious, and people presumably have terrible intuitions about very large (or small) numbers. The only two small hints of voter irrationality I noticed were on page 121–some voters may be irrationally instrumental–and the following odd quote from page 171:

One who intends through his vote to bring about the election of candidate X is on all fours with someone who steps on a crack with the intention of thereby breaking his grandmother’s back. Irrespective of what they may believe they are doing, they are in fact not acting intentionally to secure favored outcomes.

The fundamental lesson of the domination of voters’ instrumental preferences by expressive preferences is that homo economicus is a poor model for voter behavior.

Another way to put this is to distinguish “p-preferences” (those expressed when voting) from “m-preferences” (market preferences, or those expressed when the actor is decisive). The authors then discuss “r-preferences” (outcomes an actor may prefer upon reflection, but finds himself unwilling to act upon, e.g., a glutton may reflectively prefer to refuse a third serving of cake, but not actually do so) and the related concept of , items underconsumed even in ideal markets.

Voting dominated by expressive preferences could lead to the political provision of merit goods. However, demerit goods could also be provided.

The authors close with an analysis of the constitutional implications of expressive voting, e.g., what does it mean for federalism, the secret ballot, or representative democracy? Nothing is said in this chapter that hasn’t been said countless times without the benefit of a theory of expressive voting.

At the top of this post I said that the assumption the assumption of instrumental voting by public choice theory is relatively unknown to me. My very uneducated summary of the insight of public choice can be summed up as “concentrated interests trump diffuse interests.” The reason I considered theories of voting unimportant in this context is that voters are obviously diffuse. In my mind, the concentrated interests are not voter blocs, but organizations that manage to overcome the obstacles to collective (political) action (e.g., individual corporations, trade groups, and unions) and politicians themselves. I’m not sure what, if any, impact expressive voting has on this side of public choice theory. One impact may be that expressive voting within organizations lowers the bar for collective action.

There’s more to be said about the book, particularly on merit goods and related subjects (but it’s been a few months since I read Democracy and Decision, and my grasp on the subtleties is fading fast) and much more on the implications of expressive preferences outside the context of electoral contests, a subject the authors explicitly do not cover.

Learning by selling

Wednesday, November 23rd, 2005

Arnold Kling:

I’ve always felt that going to business school was a substitute for being an entrepreneur, not a complement. Those who can, sell. Those who can’t, sit in class.

Kling goes on to say that activities like product design and pitching to investors are sitting in class equivalents.

I couldn’t agree more. I’m guilty of metaphorically sitting in class and of hypocrisy–I’ve seen the urgent need for plain old selling in just about everything I’ve ever been involved with, and encouraged those with officious responsibility for marketing and sales to get out and sell, sell, sell, all the time avoiding doing any sales myself.

Technical people aren’t responsible for selling, right? The wise ones make themselves responsible. I’m a slow learner.

WUXGA LCD stretch

Monday, November 21st, 2005

I’ve been needing a notebook refresh for awhile and was planning to get a HP dv1000 (1280×768 display, ~5.2 pounds, under $1000, good Linux compatibility, and Nathan seemed to like his similar model).

Then I realized that I could get a laptop with a 1920×1200 () display. I had to have one. I missed the 1600×1200 21″ CRT I used for years and there’s reasonable sounding research that more screen is an easy productivity boost.

I bought a Dell Inspiron 6000 (my first choice was a Dell Latitude D810, for its , but I couldn’t justify a several hundred dollar premimum for an otherwise similarly equipped machine).

A number of people told me that 1920×1200 on a 15 inch widescreen would be impossible to read. Not true at all. Some people also told me that a nearly 7 pound laptop would be a major drag. So far it hasn’t been. Apart from a tiny Inspiron 2100 I used temporarily for several months this one is about the weight I’m accustomed to (and I walk or bicycle 5 to 15 miles on days I don’t telecommute–I vastly prefer this to “working out”).

I think the large monitor productivity study is right. I feel more productive than I have since giving up my desktop and 21″ CRT. If you spend most of the day doing “knowledge work” in front of a computer, especially programming, get yourself a super high resolution display pronto.

I encountered a couple of oddities regarding the WUXGA display after installing Ubuntu Linux on the new machine.

First, Ubuntu’s installer correctly detected the 1920×1200 display and Intel 915 (GMA900) graphics. The generated /etc/X11/xorg.conf only had modelines for 1920×1200. However, the driver was unaware of the 915′s support for 1920×1200, so ran at 1600×1200. I’m surprised it ran at all, given that xorg.conf contained no configuration for that resolution.

The other odd thing is that the entire screen was used to display 1600×1200 pixels–everything was stretched horizontally by 20 percent. I would’ve strongly expected 1600×1200 running on a 1920×1200 LCD screen to not use the screen’s full width–320 horizontal pixels should’ve been unused. Every description of screens that I’ve (very casually) read says something about each (discrete) pixel being controlled by an individual transistor. There’s no tweaking display size or orienting the display with an LCD like there is with a CRT. My uneducated guess is that X was using or some similar method to stretch 1600 virtual pixels onto 1920 real pixels. [Update 20051122: As Brian suggests in a comment below the stretching is done by hardware and controlled by BIOS settings--"LCD Panel Expansion" on the Inspiron 6000, enabled by default.]

The problem was fixed by running 915resolution, following this example:

  • Download 915resolution
  • make install (or just copy the binary provided)
  • Create /etc/init.d/rc.local with a single line:

    /usr/sbin/915resolution 49 1920 1200

  • sudo chmod +x /etc/init.d/rc.local
  • sudo update-rc.d rc.local start 80 S .

After rebooting X ran beautifully at 1920×1200.

Stagnant hosting prices

Wednesday, November 9th, 2005

I’ve noticed for the past year or so web hosting prices seem to have stagnated. This after prices plummeted from around 2001 through 2003. That steep drop presumably was largely an effect of the bust, but I’ve come to expect continually dropping prices for computer-related products over any period longer than say a quarter.

Some evidence from a couple low end hosting companies I’ve used:

ServerMatrix “SuperResellerz” dual Xeon, 1G RAM, 2x200G disk, 1.5T/month bandwidth: $299/month in October, 2004 (, $299/month currently.The only differences appear to be they used to offer steeper monthly discounts for larger setup fees and the Xeon CPUs have gone from 2.4GHz to 2.8GHz. (They also now offer Debian, which I believe is a very recent addition, but irrelevant for pricing.)

ServerBeach “Dual 2600 PRO” dual Athlon, 2G RAM, 2x80G disk, 2T/month bandwidth: $299/month in July, 2004 (, $269/month currently. No apparent changes.

So why haven’t prices eroded in the past year or so? Accoring to Network World hosting prices aren’t just stagnant, rather they’re soaring, with waiting lists for data center space in some cities. The only reason given:

This call for floor space and services at carrier data centers and the accompanying price increases are being driven by corporate efforts to improve disaster recovery and regulatory compliance.

That sounds more realistic than the only demand-side pressure I’d thought of, half-seriously: “” application deployment and an apparently coincident belief that pure hardware scaling is the way to go.

Most Rights Denied

Saturday, November 5th, 2005

Ryan King has created a funny spoof of Creative Commons licenses–the Uncreative Uncommons
Humor Link Back Don’t Repeat 0.1beta3 license–compare to the Creative Commons Attribution-NonCommercial-ShareAlike 2.5 license. Can you use hu-lb-dr? Nope:

The UU license is itself availble under the UU license, which means, no. See stipulation #3: “You may not paraphrase, repurpose or in any way retell the content. It is like “telling someone else’s joke” and that’s not cool.”

Ha ha.

Someone ought to create a CC license deed spoof for EULAs and :

See the EFF’s A User’s Guide to EULAs for more ideas.