Over the weekend, I converted my laptop to use the ext4 filesystem.  So far so good!  So far I’ve found one bug as a result of my using ext4 in production (if delayed allocation is enabled, i_blocks doesn’t get updated until the block allocation takes place, so files can appear to have 0k blocksize right after they are created, which is confusing/unfortunate), but nothing super serious yet.  I will be doing backups a bit more frequently until I’m absolutely sure things are rock solid, though!

I am using the latest ext4 patches and the tip of the e2fsprogs git repository.  Hopefully when we get the bulk of the patches merged into the mainline kernel after the 2.6.26 ships and the 2.6.27 merge window opens, and after I ship out e2fsprogs 1.41 (I have one work-in-progress pre-release, with another coming soon), it’ll be ready for much more wide-spread testing.

In addition to the excellent crew of ext4 developers, I’d like to call out for special thanks Gary Howco and Holger Kiehl, two early users/benchmarkers of ext4 who tried our latest code, and reported bugs that had previously escaped attention by developers (who had been mostly testing the code via the same old test suites); their additional workloads and benchmarks flushed out a few additional bugs.   Thanks, guys!!

Hopefully after a few weeks of my using ext4 for real-live work, I’ll find a few last few bugs to be fixed, and/or feel much more confident it’s ready for me to recommend to others for their production data.

Originally published at Thoughts by Ted. Please leave any comments there.

Pity poor Dr. Ari Jaaksi from Nokia. He gave a talk at the Handsets World conference in Berlin on Tuesday, where according to ZDnet, he lectured Open Source Developers that they needed to learn why DRM and other closed technologies were necessary, because of business issues such as subsidized (device) business models. I suspect he wasn’t prepared for the reaction, which took the form of a major fuss on Slashdot, as well as some declarations from a few people on the maemo-users mailing lists that they would never buy another Nokia device. Dr Jaaski then posted today on his blog an entry entitled, “Some learning to do?”, where he stated that while Nokia needs to learn how the open source world works (not just licenses and legal issues, but also the spirit), that the open source world also needed to learn as well — about WHY things are the way they are. He ended the post with “I’m not a teacher, I’m a learner” — which is fair enough.

I actually think that Dr. Jaaksi is right; we all should be interested in learning, and having talked with a number of Nokia employees, I can testify that their hearts are very much in the right place, and there are some strong reasons regarding the demands of carriers, the subsidized business model of handsets in the US, the attitudes of suppliers for components in mobile devices, that very much tie their hands — as well the hands of every other handset company out there. Progress in this space will come slowly, and but I believe there is hope that in the long run things will get better.

However, at the same time, I would suggest to Dr. Jaaksi that it’s also important how to communicate with the open source community — or with any community that holds views sometimes with great passion. Many people do not react well to being told that they need to learn something — even if it is true that they do. Admitting that they need to learn is for some people tantamount to admitting ignorance, or even worse admitting stupidity, and so they are loath to do this. Telling them this just makes them defensive and angry.

My suggestion for how get people to learn without making them defensive is to use a modified Socratic method. It works especially well with the Open Source crowd, because many of them are engineers, and engineers are by nature problem solvers. So if you give them a problem to solve, they will go to work trying to find potential solutions, and thus start thinking and learning about the problem from a different point of view. Hence, a better approach is to give them a series of business constraints (time to market, the demands of carriers, the fact that most end users prefer to buy subsidized phones with 1-2 year lock-in contracts, the intransigence of suppliers, why a single company can’t afford to create their own chipsets from scratch and have to rely on companies like Broadcom, etc.) and then tell them — we’d love to delight our customers in any way we can, and clearly you are very passionate about DRM, open device drivers, and other OSS issues. We however have to create successful products and sell them at a price such that we don’t go out of business — can you help us work through all of these constraints? Let’s work together so we can make an open source mobile platform that meets the spirit as well as the legal requirements of the open source world!

Dr. Jaaksi and his team are struggling with a hard problem, and I am very sympathetic to their issues. And, they have to be commended for investing as much time and effort on the Maemo platform, as well as other Linux and Open Source platforms as they have to date. The strength of the Open Source community is that we work together to solve our mutual problems, while celebrating and giving permission to people to “scratch their own itches”. In order to do that, we need to learn how to better communicate with each other, and that is something that goes in both directions.

When I was at MIT as an undergraduate, far too many years ago than I care to admit, I took economics classes and Sloan School of Management classes (even though they were not necessary for a Computer Science degree) for that very reason. I encourage open source advocates to learn how to walk a mile or two in business people’s shoes, since if you want to influence them them to change in the ways that you desire, you need to know where they are coming from — just shouting and throwing rotten tomatoes generally won’t get you very far. On the flip side, people who are on the corporate side of the divide also need to understand that certain ways of engaging the community can be more productive than others. At the end of the day, hopefully we can all agree with Dr. Jaaksi that everyone will benefit if we all commit to learning from each other.

Update: I’ve fixed Dr. Jaaksi’s name so it is spelled correctly; my apologies for the error.

Originally published at Thoughts by Ted. Please leave any comments there.

Andrew Binstock interviewed Donald Knuth recently, and one of the more amusing tidbits was this:

I currently use Ubuntu Linux, on a standalone laptop—it has no Internet connection. I occasionally carry flash memory drives between this machine and the Macs that I use for network surfing and graphics; but I trust my family jewels only to Linux.

More seriously, I found his comments about about multi-core computers to be very interesting:

I might as well flame a bit about my personal unhappiness with the current trend toward multicore architecture. To me, it looks more or less like the hardware designers have run out of ideas, and that they’re trying to pass the blame for the future demise of Moore’s Law to the software writers by giving us machines that work faster only on a few key benchmarks! I won’t be surprised at all if the whole multithreading idea turns out to be a flop, worse than the “Itanium” approach that was supposed to be so terrific—until it turned out that the wished-for compilers were basically impossible to write.

Let me put it this way: During the past 50 years, I’ve written well over a thousand programs, many of which have substantial size. I can’t think of even five of those programs that would have been enhanced noticeably by parallelism or multithreading. Surely, for example, multiple processors are no help to TeX….

I know that important applications for parallelism exist—rendering graphics, breaking codes, scanning images, simulating physical and biological processes, etc. But all these applications require dedicated code and special-purpose techniques, which will need to be changed substantially every few years.

This is a very interesting issue, because it raises the question of what next-generation CPU’s need to do in order to be successful. Given that it is no longer possible to just double the clock frequency every 18 months, should CPU architects just start doubling the number of cores every 18 months instead? Or should they try to concentrate a lot more computing power into an individual core, and optimize for a fast and dense interconnect between the CPU’s? The latter is much more difficult, and the advantage of doing the first is that it’s really easy for marketing types to use some cheesy benchmark such as SPECint to help sell the chip, but then people find out that it’s not very useful in real life.

Why? Because programmers have proven that they have a huge amount of trouble writing programs that take advantage of these very large multicore computers. Ultimately, I suspect that we will need a radically different way of programming in order to take advantage of these systems, and perhaps a totally new programming language before we will be able to use them.

Professor Knuth is highly dubious that the later approach will work, and while I hope he’s wrong (since I suspect the hardware designers are starting to run out of ideas, so it’s time software engineers started doing some innovating), he’s a pretty smart guy, and he may well be right. Of course, another question is whether what would we do with all of that computing power? Whatever happened to the predictions that computers would be able to support voice or visual recognition? And of course, what about the power and cooling issues for these super-high-powered chips? All I can say is, the next couple of years is going to be interesting, as we try to sort out all of these issues.

Originally published at Thoughts by Ted. Please leave any comments there.

There’s been some controversy generated over my use of the terminology of “Organic” and “Non-Organic” Open Source. Asa Dotzler noted that it wasn’t Mozilla’s original intent to “make a distinction between how Mozilla does open source and how others do open source”. Nessance complained that he didn’t like the term “Non-Organic”, because it was “raw and vague - is it alien, poison, silicon-based?” and suggested instead the term “Synthetic Open Source”, referencing a paper by Siobhán O’Mahony, ” What makes a project open source? Migrating from organic to synthetic communities”. Nessance referenced a series of questions and answers by Stephen O’ Grady from Red Monk, where he claimed the distinction between the two doesn’t matter. (Although given that Sun is a paying customer of Red Monk, Stephen admits that this might have influenced his thinking and so he might be “brainwashed” :-).

So let’s take some of these issues in reverse order. Does the distinction matter? After all, if the distinction doesn’t matter, then there’s no reason to create or define specialized terminology to describe the difference. Certainly, Brian Aker, a senior technologist from MySQL, thinks it does, as do folks like me and Amanda McPherson and Mike Dolan; but does it really? Are we just saying that because we want to take a cheap shot at Sun?

Well, to answer that, let’s go back and ask the question, “Why is Open Source a good thing in the first place?” It’s gotten to the point where people just assume that it’s a good thing, because everybody says it is. But if we go back to first principals maybe it will become much clearer why this dinction is so important.

Consider the Apache web server; it was able to completely dominate the web server market, easily besting all of its proprietary competitors, including the super-deep-pocketed Microsoft. Why? It won because a large number of volunteers were able to collaborate together to create a very fully featured product, using a “stone soup” model where each developer “scratched their own itch”. Many, if not most, of these volunteers were compensated by their employers for their work. Since their employers were not in the web server business, but instead needed a web server as means (a critical means, to be sure) to pursue their business, there was no economic reason not to let their engineers contribute their improvements back to the Apache project. Indeed, it was cheaper to let their engineers work on Apache collaboratively than it was to purchase a product that would be less suited for their needs. In other words, it was a collective “build vs. buy” decision, with the twist that because a large number of companies were involved in the collaboration, it was far, far cheaper than the traditional “build” option. This is a powerful model, and the fact that Sun originally asked Roy Felding from the Apache Foundation to assist in forming the Solaris community indicates that at least some people in Sun appreciated why this was so important.

There are other benefits of having code released under the Open Source license, such as the ability for others to see the implementation details of your operating system — but in truth, Sun had already made the Source Code for Solaris available for a nominal fee years before. And, of course, there are plenty of arguments over the exact licensing terms that should be used, such as GPLv2, GPLv3, CDDL, the CPL, MPL, etc., but sometimes those arguments can be a distraction from the central issue. While the legal issues that arise from the choice of license are important, at the end of the day, the most crucial issue is the development community. It is the strength and the diversity of the development community which is the best indicator for the health and the well-being of an Open Source project.

But what about end-users, I hear people cry? End users are important, to the extent that they provide ego-strokes to the developers, and to the extent that they provide testing and bug reports to the developers, and to the extent that they provide an economic justification to companies who employ open source developers to continue to do so. But ultimately, the effects of end-users on an open source project is only in a very indirect way.

Moreover, if you ask commercial end users what they value about Open Source, a survey by Computer Economics indicated that the number one reason why customers valued open source was “reduced dependence on software vendors”, which end users valued 2 to 1 over “lower total cost of ownership”. (Which is why Sun Salescritters who were sending around TCO analysis comparing 24×7 phone support form Red Hat with Support-by-email from Sun totally missed the point.) What’s important to commercial end users is that they be able to avoid the effects of vendor lock-in, which implies that if all of the developers are employed by one vendor, it doesn’t provide the value the end users were looking for.

This is why whether a project’s developers are dominated by employees from a single company is so important. The license under which the code is released is merely just the outward trappings of an open source project. What’s really critical is the extent to which the development costs are shared across a vast global community of developers who have many different means of support. This saves costs to the companies who are using a product being developed in such a fashion; it gives choice to customers about whether they can get their support from company A or company B; programmers who don’t like the way things are going at one company have an easier time changing jobs while still working on the same project; it’s a win-win-win scenario.

In contrast, if a project decides to release its code under an open source license, but nearly all the developers remain employed by a single company, it doesn’t really change the dynamic compared to when the project was previously under a closed-source license. It is a necessary but not sufficient step towards attracting outside contributors, and eventually migrating towards having a true open source development community. But if those further steps are not taken, the hopes that users will think that some project is “cool” because it is under an open-source license will ultimately be in vain. The “Generation Y”/Millennial Generation in particular are very sensitive indeed to Astroturfing-style marketing tactics.

Ok, so this is why the distinction matters. Given that it does, what terms shall we use? I still like “Organic” vs “Non-organic”. While it may not have been intended by the Mozilla Foundation, the description in their web page, “only a small percentage of whom are actual employees [of the Mozilla Foundation]”, is very much what I and others have been trying to describe. And while I originally used the description “Projects which have an Open Source Development Community” vs “Projects with an Open Source License but which are dominated by employees from a single company”, I think we can all agree these are very awkward. We need a better shorthand.

When Brian Aker from MySQL suggested “Organic” vs “Non-Organic” Open Source, and I think those terms work well. If some folks think that “Non-Organic” is somehow pejorative (hey, at least we didn’t say “genetically modified Open Source” :-), I suppose we could use Synthetic Open Source. I’m not really convinced that is any much more appetizing, myself, however.

So what would be better terms to use? Please give me some suggestions, and maybe we can come up with a better set of words that everyone is happy with.

Originally published at Thoughts by Ted. Please leave any comments there.

The Open Source Commands

Really good ideas that companies should take to heart.

Open Source Commandments II: Passover Penguins

More really good ideas, especially for companies like Sun…

Did Canonical Just Get Punked by Red Hat and Novell?

Interesting thoughts about Linux desktop strategies

rPath to OEM SUSE Linux Enterprise Server from Novell for Appliances

I know a bunch of the folks at rPath, and I very much respect their technology; I think this is a very good thing for them.

Does Microsoft CEO Steve Ballmer need an intervention?

Does anyone think a Microsoft/Yahoo merger makes sense besides Mr. Ballmer?

Originally published at Thoughts by Ted. Please leave any comments there.

Brian Aker dropped by and replied to my previous essay by making the following comment:

I believe you are hitting the nail on the “organic” vs “nonorganic” open source. I do not believe we have a model for going from one to the other. Linux and Apache both have very different models for contribution… but I don’t believe either are really optimized at this point.

Optimization to me would lead to a system of “less priests” and more inclusion.

I made an initial reply as comment, and then decided it was so long that I should promote it to a top-level post.

I assume that when Brian talks about “organic open source” what he means is what I was calling an “open source development community”. Some googling turned up the following definition from Mozilla Firefox’s organic software page: “Our most well-known product, Firefox, is created by an international movement of thousands, only a small percentage of whom are actual employees.”

This puts it in contrast with “non-organic” software, where all or nearly all of the developers are employed by one company. (And anyone who proves talented at adding features to that source base soon gets a job offer by that one company. By that definition we can certainly see projects like Wine, Mysql, Ghostscript (at one time), and others as fitting into that model, and being quite successful. There’s nothing really wrong with the non-organic software model, although many of them have struggled to make enough money when competing with pure proprietary softare competitors, with MySQL perhaps being the exception which proves the rule.

In most of these cases, though, the project started more as an organic open source, and then transitioned into the non-organic model when there was a desire to monetize the project — and/or when the open source programmers decided that it would be nice if they could turn their avocation into a vocation, and let their hobby put food on the family table.

Solaris, of course, is doing something else quite different, though. They are trying to make the transition from a proprietary customer/supplier relationship to trying to develop an Open Source community — and what Jon’s candidate statement pointed out is that they weren’t really interested in creating an organic open source developer community at all, but they wanted the fruits of an open source community — with plenty of application developers, end-users, etc., all participating in that community.

We don’t have a lot of precedent for projects who try to go in this direction, but I suspect they are skipping a step when they try to go to the end step without bothering to try to make themselves open to outside developers. And by continuing to act like a corporation, they end up shooting themselves in the foot. For example, the OpenSolaris license still prohibits people from publishing benchmarks or comparisons with other operating systems. Very common in closed-source operating systems and databases, but it discourages people from even trying to make things better, both within and outside of the Open Solaris core team. Instead, they respond to posts like David Miller’s with “Have you ever kissed a girl?”. (Thanks, Simon, for that quote; I had seen it before, but not for a while, and it pretty well sums up the sheer arrogance of the Open Solaris development team.)

So while Linux may not be completely optimized in terms of “less priests” and more inclusion, at least over 1200 developers contributed to 2.6.25 during its development cycle. Compared to that, Open Solaris is positively dominated by “high priests” and with a “you may not touch the holy-of-holies” attitude; heck, they won’t even allow you to compare them to other religions without branding you a heretic and suing you for licensing violations!

Originally published at Thoughts by Ted. Please leave any comments there.

I was recently checking to see what, if any follow-up there had been from Sun’s ham-handed handling of the Open Solaris Trademark, and I ran across this very interesting comment from John Plocher’s Candidate Statement for the Open Solaris Governing Board:

“I also think there was a misunderstanding about what Sun desired when it launched the community (in part) to encourage developers to adopt and use Solaris. My take is that, while there *is* value in getting more kernel, driver and utility developers contributing to and porting the (open) Solaris operating system, there is significantly *more* value in having a whole undivided ecosystem based on a compatible set of distributions, where application developers, university students, custom distro builders and users are all able to take advantage of each other’s work.

Put these two things together, and you can see Sun’s predicament. Sun *wanted* a community that empowered application developers, but *got* a community aimed squarely at kernel hackers. Whether you see this as the “kernel.org -vs- Ubuntu” fight, or the “fully open -vs- MySQL model” argument, in my opinion, it all is simply a reflection of the above mismatched expectations.”

So that explains why it’s take three long years to try to get basic open source development tools (such as putting Open Solaris source code in a distributed SCM located outside of the Sun firewall) for Open Solaris. It was never was Sun’s intention to try to promote a kernel engineering community, or at least, it was certainly not a high priority for them to do so. This can be shown by the fact that as of this writing they still are using the incredibly clunky requester/sponsor system for getting patches into Solaris; setting up a git or mercurial server is not rocket science. This lack explains why Linus gets more contributions while brushing his teeth than Open Solaris gets in a week.

So if you run into a Sun salescritter or a Sun CEO claiming that OpenSolaris is just like Linux, it’s not. Fundamentally, Open Solaris has been released under a Open Source license, but it is not an Open Source development community. Maybe it will be someday, as some Sun executives have claimed, but it’s definitely not a priority by Sun; if it was, it would have been done before now. And why not? After all, they are getting all of the marketing benefit of claiming that Solaris is “just like Linux”, without having to deal with any of the messy costs of working with an outside community. As a tactical measure, astroturfing is certainly a valid marketing trick. But after three years, the excuse of “just you wait a little longer, we’re just trying to figure this open source community stuff out”, is starting to wear a little thin.

Furthermore, if (as John Ploncher claims) this was about “empowering application programmers”, why was it that Sun’s first act was to trumpet how wonderful it was to release the Solaris source code under a Open Source license? This only seems to make sense if the Open Solaris initiative was really a cynical marketing tactic to try to save Solaris from being viewed as irrelevant. If that was Sun’s intention, I think it is fair to say that from a marketing point of view, the tactic has been at least partially successful — although as John has admitted, the goal of creating a full community with application developers, university students, and so on, hasn’t materialized for Open Solaris. Sun has the dream; the Linux community is living it.

However, from business standpoint, I wonder if Sun will really be able to sustain their Solaris engineering team if they will really be doing all of the work themselves, and outside contributions continue at the rate of 0.6 patches per day. After all, the margins when you are selling low-cost AMD servers are much lower than when you are selling über-expensive SPARC servers. With Linux, we have a major advantage in that kernel improvements are coming from multiple companies in the ecosystem, instead of being paid for by a single company. And given that 70-80% of Sun’s AMD servers are running Linux, not Solaris, it’s not clear how Sun justifies their Solaris engineering costs to their shareholders. Furthermore, if Solaris on x86_64 were to actually take off, there’s nothing to stop competitors from selling Solaris support — except the competitors won’t have to pay the engineering costs to maintain and improve Solaris, so they would be able to provide the support much more cheaply than Sun could. So while Sun’s marketing tactics have kept Solaris alive in some verticals, I have to question how successful Sun will be in the long-term.

Update: I’ve posted a reply to Brian Aker’s comments as a follow up that would probably be interesting to those folks who are interested in the ideas found in this essay.

Update^2: John Plocher’s name is spelled with an ‘h’, which I had omitted. My deep apologies to him for getting his name wrong. I’ve fixed it in the post.

Originally published at Thoughts by Ted. Please leave any comments there.

This morning, I just wasted two hours of my life trying to deal with a bill with AT&T. I am a work-at-home employee, and my company has a contract with AT&T so that when I deal 1-700-xxx-xxxx, I can reach the internal corporate phone network. In addition, long distance calls on my home office line are billed to the company at the pre-negotiated corporate rates. I also had a (long-dormant) AT&T long distance account, dating from before I started working at this company. Starting at the beginning of the year, that account grew a $18 monthly fee. When I tried to make it go away, the AT&T consumer side of the house said that according to Verizon, that was because I had my long distance service through AT&T. Which was true, in a sense — AT&T was providing my service, but through a corporate account. But the consumer side of AT&T didn’t understand that, and were in my opinion, willfully ignorant.

Several phone calls later, I got the 1-800 number for the AT&T corporate side of the house, and those folks said they couldn’t look at consumer/personal AT&T accounts, and implied it was my fault that I had both accounts on the line. (This took over an hour while the support person tried multiple things, all of which was not helpful at all.)

I finally googled for AT&T CEO’s office, and found a number on the consumerist.com web site that claimed to be the AT&T CEO’s office. It had long since been directed to a help center, but after I told my tale, I was quickly transferred to an executive customer support person, who was able to fix the problem in ten minutes.

The only questions remaining are:

1) Why can’t the different parts of AT&T talk to one another?
2) Will this problem really be solved, or will I see another bill in a month or two and have to spend more time dealing with this mess all over again?
3) When will VOIP services put AT&T long distance out of its misery? (Given the absolute frustration of this morning, this can’t happen soon enough.)
4) Will AT&T compensate me for two hours of frustration and of my life that I will never get back. (Not bloody likely.)

One thing is for certain, it will be a long, long, LONG time before I will ever voluntarily choose AT&T to provide service for just about anything. Even an iPhone wouldn’t be enough inducement….

Originally published at Thoughts by Ted. Please leave any comments there.

Although I lot of people have been lauding the Kindle, I recently decided to go with the Sony PRS-505 instead. Yes, the Kindle has built-in EVDO access, and the ability to buy books without a computer, or even browse the web; and yes, the Sony has once again demonstrated it can’t create a compelling 21st century computer application to save its life. However, it had a few things that at least for me, made a better choice for me than the Kindle:

1. The Sony is thinner — I want to be able to slip it into my laptop case and have it take the absolute minimum amount of space.
2. The Sony simply looks much more elegant than the Kindle; steel with a leather cover simply looks a lot better than white, cheasy plastic.
3. I’m not interested in buying a lot of DRM’ed ebooks; ergo, I won’t be buying may books from either Sony or Amazon’s web sites. It is highly likely that within 2 years I will be buying a more advanced eBook reader, possibly one with color, and I don’t want to be locked into a single format where I have to go and repurchase all of my books just because some the latest and greatest eBook reader uses an incompatible DRM technology from whatever Sony or Amazon has used.
4. The Sony is $100 cheaper. Given that something better will be available within 2-3 years at the very most, and possibly sooner, I’m just not interested in spending $400 on a first generation prototype.
5. Perhaps the most important, the Sony has the really, really good open source support. Kovid Goyal’s libprs500 project supports the Sony PRS-500 and PRS-505, and has very good version tools, allowing people to convert eBooks previously stored in HTML, PDF, TXT, Microsoft Reader (.lit), IDPF/Open eBook (.epub) into Sony’s format. And with a little bit of work, it does a very, very good job with the conversion. Better yet, its ability to convert multiple HTML pages into a single eBook, with credible table of contents, means that libprs500 can pull down the New York Times, the Economist, etc., automatically format it into a single eBook which you can save onto your Sony Reader, and then read it while you are on the airplane. No muss, no fuss. I can also take various books that are available on the web as HTML and also convert them into an eBook which can be used by the Sony Reader very easily.

This last point is I believe one of the best reasons why the Sony Reader will be able to compete very successfully with the Kindle. The libprs500 software is written as a python application, and it will work on Windows, Linux, and MacOS — and its GUI user interface is far better than the truly pathetic Sony Connect software. Score one for Open Source! In my opinion, Sony should send a very nice gift certificate to Kovid as a thank you; his open source project has added an immeasurable amount of value to their product.

The only thing that you can’t do using the libprs500 software is buy DRM’ed books which are locked to the Sony Reader — but that isn’t something that many people will be particularly interested in, I suspect. OK, I did buy Pillars of the Earth, which was available on the Sony site for $6 dollars — hmm, cheaper than Amazon’s $9.99 — but that was an investment I was willing to flush down the toilet when the PRS-505 becomes obsolete, mainly so I could test what buying a DRM’ed book would be like from the Sony web site. But I probably won’t be buying many books with DRM that way. On the other hand, I am quite willing to spend quite a bit more money on non-DRM’ed books from publishes such as Baen Books.

Here’s to the hope to the publishing industry figures things out faster than RIAA’s member companies. In the meantime, I will be mostly pretending that the both the Sony and Amazon eBook stores with their proprietary DRM’ed books don’t exist…

Originally published at Thoughts by Ted. Please leave any comments there.

Bryan Cantrill recently wrote a blog entry, where among other things, he philosophized on the concept of “perfect code”. He compares software to math, arguing that Euclid’s greatest common denominator algorithm shows no sign of wearing out, and that when code achieves perfection (or gets close to perfection), “it sediments into the information infrastructure” and the abstractions defined by that code becomes “the bedrock that future generations may build upon”. Later, in the comments of his blogs, when pressed to give some examples of such perfection, he cites a clever algorithm coded by his mentor to divide a high resolution timestamp by a billion extremely efficiently, and Solaris’s “cyclic subsystem”, a timer dispatch function.

Watching his talk at Google where his introduction and sidebar book review on Scott Rosenberg’s “On Dreaming in Code”, it’s clear that he very passionately believes that it is possible to write perfect code, and that one should strive for that at all times. Perhaps that’s because he mostly writes code for operating systems, where the requirements change slowly, and for one OS in particular, Solaris, which tries far harder than most software projects to keep published interfaces stable for as long as possible. In contrast, the OS I’ve spent a lot of time hacking around, Linux, quite proudly states that at least inside the kernel, interfaces can not and should not be stable. Greg Kroah-Hart’s “Stable API Nonsense” is perhaps one of the strongly and most passionate expositions of that philosophy.

I can see both sides of the argument, and in their place, both have something to offer. To Bryan’s first point, it is absolutely true that interfaces can become “bedrock” upon which entire ecosystems are built. Perhaps one of the most enduring and impactful example would be the Unix programming interface, which has since become enshrined by POSIX.2 and successor standards. I would argue, though, that it is the interface that is important, and not the code which initially implemented it. If the interface is powerful enough, and if it appears at the right time, and the initial implementation is good enough (not perfect!), then it can establish itself by virtue of the software which uses it becoming large enough that it assumes an important all out of scale with its original intention.

Of course, sometimes such an interface is not perfect. There is an apocryphal story that when S. Feldman at AT&T labs first wrote the ‘make’ utility, that he did so rather quickly, and then put it available for his fellow lab members to use, and then went home to sleep. In some versions of the story he had stayed up late and/or pulled an all-nighter to write it, so he slept a long time. When he came back to work, he had come up with a number of ways to improve the syntax of the Makefile. Unfortunately, (so goes the story) that too many teams were already using the ‘make’ utility, so he didn’t feel he could change the Makefile syntax. I have no evidence that this ever took place, and I suspect it is an urban myth that was invented to explain why Makefiles have a rather ugly and unfortunate syntax that many would call defects, including the use of syntactically significant tab characters which are indistinguishable from other forms of leading whitespace.

Another example which is the bane of filesystem designers everywhere are the Unix readdir(2), telldir(2), and seekdir(2) interfaces. These interfaces fundamentally assume that directories are stored in linear linked lists, and filesystems that wish to use more sophisticated data structures, such as b-trees, have to go to extraordinary lengths in order support these interfaces. Very few programs use telldir(2) and seekdir(2), but some filesystems such as JFS maintain two b-trees instead of one just to cater to telldir/seekdir.

And yet, it is absolutely true that interfaces can be the bedrock for an entire industry. Certainly no matter what its warts, the Unix/Posix interface has proven the test of time, and it has been responsible for the success of many a company and many billions of dollars of market capitalization. But is this the same as perfect code? No, but if billions of dollars of user applications are going to be depending on that code, it’s best if code which implement such an interface be high quality, and should attempt to achieve perfection.

But what does it mean for code to be perfect? For a particular environment, if the requirements can be articulated clearly, I can accept that code can reach perfection, in that it becomes as fast as possible (for the given computer platform), and it handles all exception cases, etc., etc. Unfortunately, in the real world, the environment and the requirements inevitably change over time. For example, Bryan’s cyclic subsystem, which he proudly touts as being if not perfect, almost so, and which executes at least 100 times a second on every Solaris system in the world. I haven’t looked at the cyclic system in any detail, since I don’t want to get myself contaminated (the CDDL and GPLv2 licenses are intentionally incompatible, and given that companies — including Sun — have sued over IPR issues, well, one can’t be too careful), but waking up the CPU from its low-power state 100 times a second isn’t a good thing at all if you are worried about energy conservation in data centers — or in laptops.

For example, on my laptop, it is possible to keep wakeups down to no more than 30-35 times a second and it would be possible to do more but for an abstraction limitation. Suppose for example an process wants to be sleep and then receive a wakeup 1 milliseconds later, and so requests this via usleep(). At the same time, another application wants to sleep until some file descriptor activity takes place, or after 1.2 milliseconds takes place. 0.3 milliseconds later, a third process requests a sleep, this time for 0.8 milliseconds. Now, it could be that in all of the above cases, the applications don’t actually need exact timing; if they all get their wakeups plus or minus some fraction of a millisecond, they would be quite cool with that. Unfortunately, the standard timer interfaces have no way of expressing this, and so the OS can’t combine the three wakeups at T+1.0, T+1.1, and T+1.2 milliseconds into one wakeup at T+1.1ms.

So this is where Greg K-H’s “Stable API Nonsense” comes into play. We may not be able to solve this problem at the userspace level, but we darn well can solve this problem inside the kernel. Inside the kernel, we can change the timer abstraction to allow device drivers and kernel routines to provide a timer delta plus a notion of how much accuracy is required for a particular timer request. Doing so might change a timer structure that previously external device drivers had depended upon — but too bad, that’s why stable ABI/API’s are not supported for internal kernel interfaces. Is this that the interface could have been extended? Well, perhaps, and perhaps not; if an interface is well designed, it is possible it can be extended in an API and/or ABI compatible way. There usually is a performance cost to doing so, and sometimes it may make sense to pay that that cost, and sometimes it may not. I’ll talk more about that in a future essay.

Yet note what happened to the timer implementation. We have a pretty sophisticated timer implementation inside Linux, that uses heap data structures and buckets of timers for efficiency. So while I might not call it perfect, it is pretty good. But, oops! Thanks to this new requirement of energy efficiency, it will likely need to get changed to support variable levels of accuracy and the ability to fire multiple timers that are (more or less) coming due in a single CPU wakeup cycle. Does that make it no longer perfect, or no longer pretty good? Well, it just had a new requirement impact the code, and if criteria for perfection is for the abstraction defined by the code to be “bedrock” and never-changing, and no need to make any changes in said code over multiple years, I would argue that little to no code, even OS code, can ever achieve perfection by that definition — unless that code is no longer being used.

(This is the first of a multi-part series of essays on abstractions that I have planned. The next essay which I plan to write will be entitled “Layer surfing, or jumping between layers”, and will be coming soon to a blog near you….)

Originally published at Thoughts by Ted. Please leave any comments there.

I am currently in the Hilton Portland & Executive Tower hotel, and since I fly entirely too much, I got upgraded into a room which contains a printer. Thinking that I would try using it, I hooked it up to my laptop (running Ubuntu Gutsy), selected System->Administration->Printing on the desktop, and then clicked on New Printer. To my astonishment, when the dialog box came up, the system had already autodetected the fact that I had an HP OfficeJet KX60xi printer connected to the parallel port, had recommended which driver I should use, and a few “next” and “continue” clicks later, the printer was installed, and 15 seconds later I was able to print to it.

Users of MacOS systems are probably used to such things, but this was faster and easier than what Windows asks of users who want to install a new printer. Coming from a Unix background, I would have been quite pleased if I was able to set up the printer after manually select the printer type and driver from a dialog box. Simply not having to su to root, edit some config files, and then restart some daemons, would be a major advance. But this completely exceeded my expectations. Well done to everyone in the CUPS and GNOME community who worked to make this possible!

Originally published at Thoughts by Ted. Please leave any comments there.

It’s interesting to see how far, and yet how much more work we need to do on power management for Linux. I recently got a new laptop — a Lenovo Thinkpad X61s — and using the powertop tool, I was able to configure my system to the point where in what I can “airplane mail reading mode” (mailbox preloaded into memory, USB disabled, wireless and ethernet disabled, backlight down to 30% brightness, sloppily written power hogs like Firefox and Notes — every single application writer should be forced to run powertap and explain why their program feels it necessary to constantly wake up the CPU), I can get my usage down to about 9.8 watts. Using the 8 hours extended battery, that’s 8 hours of battery life, although granted doing very little. On the flip side, if I’m doing a major kernel compile, I can drive up utilization up to almost 30 watts, which means less than 3 hours of battery life. So that’s definitely the good news; Linux can sharply reduce its power consumption to the point where it is highly competitive with Windows. (And probably better than Vista, just because that OS is so heavy and bloated.) So thanks and and a tip of the hat to Intel and to Arjan van de Ven for making such a useful tool like Powertop available.

So now for the bad news. Getting down to this level of power saving thriftness, where the laptop is carefully sipping only the minimal amounts of power from the battery, is definitely a work in progress. First of all, you can only get this level of power savings by unloading a specific USB driver, uhci_hcd. This will disable low speed devices (including unfortunately the fingerprint reader and the EVDO WWAN device if you were silly enough to buy one that was built into the laptop as opposed to a stand-alone card that you can swap between laptops and lend out to friends as necessary). But how many users are going to open up a terminal window, su to root and type the command “rmmod uhci_hcd”? And know how to reload the driver using “modprobe uhci_hcd” when they need to use the USB devices again?

A similar problem exists for Network Manager; when the user disables the network by right-clicking on the applet, why doesn’t it automatically bring down the interface, instead of forcing the user to manually su to root and then type the command “ifconfig eth0 down; ifconfig wlan0 down”?

A more serious problem is the Intel Wireless driver for the 4965. Even with the wlan0 interface configured down, and with the RF kill switch enabled, keeping the iwl4965 driver loaded will still cost you an extra full watt of power. When you’re down to 9.6 watts, that means that keeping the iwl4965 driver loaded when you don’t need it will cost you a 10% reduction in your battery life! That’s just sloppy, and hopefully it will be fixed in a future update to the iwl4965 driver, but as long as you don’t mind manually removing and reloading it, you can work around this power-saving oversight.

A bigger issue, though, one for which no workaround exists, is that unlike the ipw3945 drivers, which at least had private, non-standard iwpriv commands to engage the 802.11’s power-saving features, the iwl4965 driver has neither the non-standard Intel iwpriv interfaces, nor the standard iwconfig interfaces for enabling any kind of powersaving features, including changing the transmit power of the card. So while powertop deserves plenty of   kudos, iwl4965 deserves a wag of the finger from a power saving viewpoint. No doubt Intel just needs to allocate more money to its Open Source Technology Center so it get more of its crack developers to work improving Linux support for their processors and chipsets.

Speaking of which, I’m still waiting for an Intel x.org 965GM driver that can support compiz/beryl and simultaneously show video clips at the same time… And being able turn off the 50 interrupts/second generated by the video card when they aren’t needed because 3-d graphics aren’t currently in use, without requiring a restart of the X server, would also be a nice touch. The bottom line is that Linux power savings and Linux support for laptops in general is much better than it was a year ago, and a lot of credit has to go to the efforts of Intel’s teams producing such good work as powertop, their wireless drivers, and their open X server drivers. We still have a lot of work left to be done, though!

Originally published at Thoughts by Ted. Please leave any comments there.

In a previous post, I had recommended the 4130 NIST-Traceable Temperature Controller to control the temperature in a slow cooker. Unfortunately, that particular controller has a range that tops out at 60 degrees C / 140 degrees F, which is enough for cooking beef for long periods of time, but not enough for say, cooking duck confit, which for which a sous vide temperature of 80 degrees C is recommended. In addition, the 4130 is pretty expensive; almost $150. It’s possible to add a resistor to change the range of the 4130, but the temperature displayed by the controller is no longer correct, and you have manually create a conversion table between true temperature and the temperature as seen by the controller.

I’ve recently come across a cheaper and better possibility, the Ranco ETC-111000-000 Temperature Controller which is only half the price and comes with a much larger working range (-30 to 220 degrees F). The price with the AC cord already wired in is $75; and the version which just has a 120VAC SPDT relay is only $60.

A bit more about food safety. There has always been a lot of concern about bacteria growth and botulism, for good reason — and so therefore the recommendations for cooking temperature have a lot of safety margin in them — to the point now that the USDA recommends that steaks be cooked to at least 145 degrees F, which is well within what had traditionally been called “medium”, and chicken to at least 165 degrees F, which is enough to really destroy taste and texture. Sous vide cooking, especially some of the more low temperature variants, have raised a lot of concerns, to the point where a few years ago New York City (temporarily) banned it, causing a great outcry in the foodie community, since many top restaurants use sous vide techniques.

First of all, any recommendation about internal temperatures and food safety that doesn’t also factor in time is massively oversimplifying the problem. Here is a table taken from “Food Safety Hazards and Controls for the Home Food Preparer”, published by the Hospitality Institute of Technology in 1994:

Temperature, F	Time, 5D kill	Time, 6.5D kill
130	86.42 minutes	112.34 minutes
135	27.33 minutes	35.53 minutes
140	8.64 minutes	11.23 minutes
145	2.73 minutes	3.55 minutes
150	51.85 seconds	1.12 minutes
155	16.40 seconds	21.32 seconds
160	5.19 seconds	6.74 seconds
165	1.64 seconds	2.13 seconds

This table lists the time to reduce bacteria concentrations of Salmonella and E. coli from 100,000 to 1 (5D) or 3,162,277 to 1 (6.5D). The FDA and USDA recommend cooking hamburger to 5D destruction. Since it is extremely unlikely for there to be more than 100 Salmonella organisms per gram of meat, a 5D kill will reduce Salmonella concentrations to no more than 1 organism per kilogram. So whether you cook a piece of meat to an internal temperature of 165 degrees for 1.64 seconds, or hold it at 130 degrees for 90 minutes, the effect on Salmonella and E. coli bacteria will be the same.Of course, one concern is that in many forms of cooking, particularly oven roasting and grilling, the temperature of the food as it heats up may not be even; so how do you guarantee that all parts of the food product has been brought up to the requisite 165 degrees? One way that recipe authors, fearful of liability concerns, have done so is to tell people to cook meat to much higher internal temperatures, to provide that extra safety margin at the expense of dessicated, horribly tasting turkey or chicken. But the advantage of sous vide cooking is by immersing the food in a water bath, the excellent heat conductivity of water helps guarantee that the entire body of meat will get raised to desired temperature relatively quickly. (How quickly depends on the thickness of the meat, obviously). So if you hold a roast beef that has been vacuum packed in a Foodsaver bag for five or six hours, there should be no question that all of the common bacteria has been inactivated, and that amount of time at 130 degrees F should be sufficient to inactivate 99.9 percent of all botulism toxin molecules (not there should be any in a fresh piece of meat, of course!)

However, sous vide temperatures are not enough to kill bacterial spores, in particular C. botulinum, which is responsible for botulism. This requires temperatures far in excess of boiling water at sea level. For example, home canning protocols recommend holding the food product at 250 degrees F for at least 15 minutes. This is an issue in the restaurant business because very often food would be cooked sous vide, and then stored in the vacuum sealed bags for potentially weeks (yes, you could be eating an extremely expensive meal at a top-end French restaurant that had been cooked several weeks ago, and reheated just before serving; yummy, no?). If the food packages aren’t cooled quickly enough, and then allowed to warm to the danger zone, it’s possible that in the anerobic environment the C. botulinum spores could germinate and then start producing toxin. But, if you are cooking home sous vide where you are serving the food right after it has been cooked, this shouldn’t be a concern.

Originally published at Thoughts by Ted. Please leave any comments there.

I got a call from one of the researchers from ABC news this evening. Apparently they are planning on doing a segment on Hans Reiser on their 20/20 show, and the researcher knew enough that there were some serious technical inaccuracies with the script (at the level of “so when Hans was creating reiser4, was that software; was he writing a program?” and “does the program run on the hard drive?”), and so they were looking for help on some technical issues on a background basis. She didn’t take any quotes from me; what she needed was help understanding the technical issues around filesystems and Linux. I tried to help explain what a filesystem was that would make sense to a lay person in 15 seconds; I have no idea whether the researcher got it, and whether she’ll be able to make changes in the script that will be vaguely coherent. We’ll see. I spent a lot of time working with Joshua Davis, providing background material for his Wired article, and he still got a bunch of the technical details wrong. The 20/20 researcher did ask me some silly questions such as “whether we were surprised when murder charges were filed against Hans?” Well, yes. What did she expect me to say? “Oh, yes, were always worried someone would get hurt…” Not! Sigh…. I might not agree with Hans’s filesystem design principles or his tactics in trying to get reiser4 accepted, but I always respected him as a fellow open source programmer.

The researcher mentioned to me that Hans had done a talk at Google that was available at the Google Video site. Out of curiosity, I took a look at it. It was interesting; the last time I had seen Hans was in 1999, at the Linux Storage Management Workshop in Darmstadt, Germany that was organized by Matthew O’Keefe at Sistina Software. Compared to how he looked back then, and the picture of him in the Wired article, taken a 10-12 months after his talk at Google in February, 2006, I was struck by how much heavier (and older at least compared to my memory of him, which was much closer to this picture from his resume) in the Google Video. I also remember him as being a much more dynamic and energetic speaker, and I was struck by how slowly he spoke, with lots of long pauses and “umms” and “ahhs”. He seemed to be a much better public speaker in Darmstadt, but in the Google talk he seemed very tired as he gave the talk.

One of the questions that the researcher asked me was whether I thought he was a genius or not. I told her that I thought he was quite bright in terms of raw intelligence, but that his Social IQ wasn’t as high as you might want for someone to be successful in gathering volunteers to work on an Open Source project, and in working with others in an Open Source development community. Looking at the video, I think that is very much true. He was a terrible public speaker, but some of the points he made about optimizing B-tree algorithms made sense. I might disagree with his philosophy over filesystem design and benchmarking, and I might not be terribly impressed by his social skills, but in terms of being a talented computer scientist, he was and is that.

Is he guilty of the crime that he has been accused of? I have no idea. But from looking at his talk and knowing what I know of him, I have the sense of a greek tragedy. He’s been working on some of his ideas since his undergraduate days in UC Berkely in the early eighties — which he entered after finishing his 8th grade. And as research ideas, I think he might have gotten some very interesting results out of trying some of the things that he wants to do around operating system namespaces. (I think they are doomed in a production system descended from Unix, since application programmers are unlikely to rewrite their programs to take advantage of reiserfs’s performance characteristics — which would mean instead of using a Unix-like configuration file, treating a directory hierarchy containing the configuration information like a Windows registry. Still, as an academic, Plan 9-like system, I could have seen it has being a potentially very interesting Systems Research effort.) Unfortunately, his skills at public speaking and his ability to work with other people have handicapped him, and I know that has frustrated him deeply. So I have a lot of sympathy for him, and I hope that he is innocent, and will be found innocent. But only time will tell….

Anyway, according to the researcher, they are currently scheduling their segment about Hans on 20/20 on October 19th. I’m sure that schedule is subject to change, but it’ll be interesting to see how they treat Hans in their coverage. Hopefully it will be fair, and not overly sensationalistic, but unfortunately my faith in today’s TV edutainment-focused news program isn’t terribly high. My impression was that the research wanted to do a good job, but she was burdened by a very tight deadline, and in the end, the decision of what goes on the air and what doesn’t won’t be up to her. So we’ll see.

Originally published at Thoughts by Ted. Please leave any comments there.

I recently came across three very well written and very thoughtful blog postings by Rick Cook (author of the Wiz Zumwalt Wizardry series):

1. Copyrights, Whack-the-Gopher, and SFWA — Why I Quit
2. The Economics of Theft: Son of Whack the Gopher
3. WHACK THE GOPHER III: The Return of the Mutant Grandson

The incident which kicked off these postings was an informal DMCA takedown notice posted by the Vice President of the Science Fiction Writers of America, Dr. Andrew Burt which was created by a process only slightly more sophisticated than using word search for “Asimov” and “Silverberg”. Unfortunately, this takedown notice erroneously included a junior high school teacher’s list of 300+ recommended books (which naturally contained the strings “Issac Asimov” and “Robert Silverberg”, an on-line published science-fiction magazine that referenced the science fiction author Isaac Asimov in a review, and a creative-commons licensed science fiction novel which had been deliberately published on the web for free distribution, and for which the author had explicitly forbidden the SFWA from taking any action on behalf of his books.

The entire incident was wonderfully filled with irony — from the fact that an organization of writers who purport to write about what the future might bring given scientific and technological advances could so totally fail to get the Internet or understand that such a campaign might cause them to alienate their readers and fan base, to the the fact that Doctor Andrew Burt, Ph.D. is a professor in computer science at the University of Denver with a research interest in copyright and electronic piracy, could so incompetently foul up a DMCA takedown request and not understand that “grep” might result in false positives that would require human checking (”Andrew? Your alma mater is calling; they would like their degree back…”)

One good thing that has come out of this whole mess is that it has been, as junior high school teacher Nick Singer put it, “a teachable moment”, and an opportunity for people to reflect about issues of copyright, the rights of authors, ebooks, and the Internet. This is a hard problem; I very strongly believe that (at the same time) “Art wants to be free; Artists want to be paid” (to use a phrased coined by a friend of mine, Jesse Vincent (blog here), to the point that I’ve been willing to put my money where my mouth is. So far the solutions for achieving this are nowhere near perfect, but they are certainly better than sending out shotgun DMCA takedown requests. In any case, Rick Cook’s thoughts on the subject are a worthy contribution to the subject. He says he’s going to do one more article on his blog proposing an economic solution to this problem; I can’t wait to see what he has to say on the subject.

Originally published at Thoughts by Ted. Please leave any comments there.

Because I’ve been getting a little frustrated with the lack of trackback support, prohibition against Javascript, etc. I’ve decided to set up a Wordpress installation on my private machine. For now, I have things set up so that posts made on my Wordpress blog will get mirrored onto my LiveJournal account. I may do some friends-locked posts still on LJ, but my general philosophy is to not post anything I might consider private on a blog, friends-locked or no. So we’ll see how often I make use of that option.

Originally published at Thoughts by Ted. Please leave any comments there.

Recently, Josh Berkus blogged about The Myth of the Benevolent Dictator. In it he complained about people who try to posit that a benevolent dictator was "an unalloyed asset" to their project, and that there were many successful and valid forms project governance other than the singleton project leader. Furthermore, he argues that the best way is to form a community, and then ends by suggesting that people who say "dictatorship good" is really saying "democracy bad", and perhaps the motives of the person using that argument should be questioned.

Given that I was one of the first people (and probably the first person) to use that phrase to describe Linus's governance model, obviously I feet the need to respond to his thoughts on the matter.

First of all, the term "benevolent dictator" comes from an old saying, "the best form of government is the benevolent dictator --- there's only one problem, finding the benevolent dictator"[1]. And in my presentations where I explained the Linux project management model, I was always careful to say that there were other models that were used by other projects, and that it was interesting to study why certain management models worked for some projects but not others.

So by definition, if a project is successful with a single project leader ultimately responsible for making decisions to settle conflicts, then that leader is a "benevolent dictator"; if they aren't successful, then that would be an example of a dictator who wasn't able to be "benevolent", eh? :-)

In all seriousness, though, I agree with Josh when he says that the key part of an open source project being successful is gathering together a healthy community. But let's unpack that a little. In order for a community to be healthy, there needs to be general consensus on the values and cultural norms for that community. So for example, in the case of Debian, which Josh holds up as a successful community, not everyone has the ability to vote; you have to pass a fairly stringent process which tests your technical skills and your willingness to subscribe to Debian's core values as embedded to the Debian Social Contract. People who do not pass the New Membership process do not have the right to vote in Debian, even if they are the most rabid, fanatical users of Debian.

This brings up a key point about governance analogies; they all don't map cleanly into the real world for two closely related and extremely important reasons: (1) No one has a fundamental human right to be a member in some OSS project, and (2) No one is forced to stay in a particular OSS project. These are really two sides of the same coin, and what it means is that, as in Debian, "democracy" does not mean that everyone is entitled to political franchise, or the right to vote, and in a project which has a "benevolent dictator", the powers of the dictator is fundamentally limited by the simple fact that if he is too unreasonable, people are always free to leave or fork the project. These differences are extremely important when trying to reason about "democracy" or "dictatorship" as being good or bad as applied to open source projects.

Democracy is rarely the best way for an open source project to make decisions. For technical issues, you want the right answer, and majority vote isn't necessarily way to resolve a technical issue. As far as policy decisions, if the decision can not be made by consensus, then some percentage of the community will be unhappy with how the decision, and that can cause a huge amount of arguments and dissension. If the community has very clearly expressed values, then the number of times where such votes need to be made will be minimal.

For example, Debian has an explicit declaration of its values and mission in its Debian Social Contract; people who don't agree to the Social Contract aren't allowed to become Debian Developers and thus are not allowed to vote. But, one of the more controversial disputes that have threatened to tear apart Debian is the question of the non-free packages, and that's fundamentally because of an ambiguity in the Social Contract. On the one hand, Debian has declared that it is fundamentally 100% about Free Software, to the extent that it has adopted Richard Stallman's rather petulant request that the Linux operating system be called GNU/Linux (which admittedly is better than Lignux.) On the other hand, the Social Contract also explicitly specifies the existent of the non-free section in the distribution. Over time, this ambiguity have been the source of a lot of highly passionate debate and controversial votes that have satisfied neither the FSF fanatics nor the more Linux-like moderate position tolerating the use of proprietary as well as open source software. I submit that if Debian had either explicitly stated an FSF-centric position, and only accepted members that supported point of view, or explicitly subscribed to a position that users should be able to use whatever software they feel meets their needs, as Ubuntu has, it would avoided many arguments that have threatened to tear apart Debian.

More recently the fact that the Social Contract has been silent about whether or not Debian Developers should be paid or purely be 100% volunteer. This was never an issue in the Linux Kernel community, but for some reason it has been a highly divisive and disruptive issue inside Debian. Democratic votes have not really resolved the conflicts, and in fact some might claim that these votes have if anything hardened various Debian Developers' positions.

Going back to Josh's final point, "Next time you hear the words 'benevolent dictator' you probably want to think about the motives of the person saying it. Because the corollary to 'dictator good' is 'democracy bad', and is that a place you really want to go?", I am quite proud to stand by my convictions that for Open Source Projects, pure democracy is a bad idea. If a projects wants to be technically successful, it needs to be a meritocracy, and chose the best technical decisions; Linus does not decide by abiding with majority vote whether or not a patch should be accepted, and that's as it should be.

Even Debian's "raw democracy" doesn't let any random luser get to vote on who should be the Debian Project Leader. Decisions need to be made by people who are competent to make a valid choice. Even in real life, one could argue that the choice made by Americans, over of which 70% believed that Saddam Hussein was "very likely" or "somewhat likely" to be personally involved with the 9/11 attacks even though there has been absolutely no evidence to support this "gut feeling", made a rather disastrous choice in voting to re-elect George Bush to the presidency. Democracy without sufficient education, literacy, and intellectual curiosity to learn about what is happening in the world can result in truly disastrous choices being made. So certainly, democracy is not a panacea. In the real world, it may be the best choice of many bad choices, given how much people have suffered under military dictatorships. However, in the Open Source world, things are different because people are always free to leave one project and affiliate with another --- not an alternative most people living in dictatorships have historically had. Given these key differences, I think it is clear that for open source projects, democracy is indeed often a bad idea, and that if you if a project has a single person with a strong technical vision and good interpersonal skills, the benevolent dictator model can indeed be a very viable and successful.

Does that mean that other forms of governance can't work? Of course not! The rotating dictatorship used by Perl[2], the core-team-with-strongarm leader used by OpenBSD, the corporate dominated scheme of MySQL, the self-selecting core team model of Samba, all are successful models. I just have to strongly disagree with Josh's claim the single strong leader approach is somehow inferior to "democracy" as if democracy was somehow a magic bullet.


[1] For am early example of this quote, nine years ago, please see this message, where I'm pretty sure I authored the colon prefixed quote, and I know I was using the term "benevolent dictator" much earlier than 1998.

[2] In the early days of Perl, it also used the benevolent dictator model. Larry Wall is a great exemplar of how someone who has a strong vision of what he wanted to accomplish, and was humble and easy to work with, both on-line and in person, can make all the difference in the world for an up-and-coming open source project.

I looked at the Foleo and played with one while I was at Linux World a few weeks ago, but it is so restrictive in what it can do that I was completely unimpressed.

First of all, according to one of the people at the booth, it will only work with a select set of Treo's; mostly the newer ones. A colleague I was with had just gotten a Treo 650, and the person at the booth said that it wouldn't work with that model of Treo. (WTF?) Furthermore, it will only do e-mail (POP or IMAP) by linking with the Treo over bluetooth connection and letting the Treo pull down the e-mail and store it on the Treo. Given that it only has 128 megs of RAM and 256 megs of Flash, it just doesn't have enough storage apparently to run a stand-alone e-mail application, which is a little bit scary. The limited amount of memory is probably why it is using Opera as a web browser, which previous experience on the N800 has largely unimpressed me in terms of compatibility with Web 2.0 sites that aggressively use AJAX or flash.

So OK, it's not supposed to be a laptop. But the problem is, for 0.2 pounds more, I can get a laptop. Let's review the critical statistics, shall we? The Foleo costs $499, weighs 2.5 pounds, and has a size of 10.5" by 6.7" by 0.9". As stated before it has 128 megs of RAM and 256 megs of flash, with a SD slot for expansion purposes, and it has a claimed 5-6 hours of battery life. But let's compare that with my IBM X41 which I recently purchased off of eBay for $800. It has 1.5 gigs of memory, and 60 gigs of hard disk. It has two batteries; with the 4 cell battery it weighs 2.7 pounds and delivers 2 hours of battery life, and with the 8 cell battery it weighs 3.2 pounds and delivers 4 hours of battery life. So true, even with the 8 cell battery the X41 is 1.7 pounds heavier and still has slightly less battery life. But you can do a lot more with the X41! Furthermore, the Foleo's weight advantage is somewhat nullified by the fact that you have to bring the Treo around to do certain activities, and the Treo has to be powered on since the Foleo is mostly designed around being a remote large screen for the Treo.

At the end of the day it's all about tradeoffs. Perhaps if enough companies created enough killer apps that could fit in 128 meg of ram for the Foleo, it might be useful enough to justifying buying it. I hear for example, that even though the Foleo doesn't have any kind of PIM functionality, a 3rd party ISV is planning on making a product available that will provide calendar and contact functionality that can sync Palm PDA's. No word on how much it will cost or how usable it will be, but with enough applications, maybe Foleo could be useful enough to justify its size/weight. I imagine that these apps will probably be commercial ones, since open source apps like Evolution will probably have difficulty fitting in the Foleo's constrained environment. :-)

And, of course, it's a lot cheaper than my used X41 laptop, never mind a brand-new Lenovo X61s, which could run 2 or 3 kilobucks fully outfitted with the 4 gigs of memory and 160 gig/7200 rpm drive. However, as a road warrior, my priorities are not just weight, but functionality. A 2.5 pound solid-state laptop with only 128 megs of memory which massively restricts what I can do is not a good use of the space in my laptop bag. What I'm waiting for is the next generation of the Thinkpad X series which has a solid state disk --- which shouldn't be that far off --- and the elimination of the spinning magnetic media would mean that we should have something with Foleo's battery life without the Foleo's limited usefulness. Sure, maybe I will need to wait another year or two for my perfect laptop (12", 1024x768 LED display, at least 60 gigs solid state disk, at least 2-4 gigs memory, Intel core 2 or follow-on processor, < 3 pounds, > 5 hours useful lifetime) to become available, but the technology to do this exists today; it's just a matter of making it affordable. But given that kind of long-term future, I'm willing to settle for now with either 2 or 4 hour battery life, and a slightly heavier laptop, than to use something today with a desperately slow ARM processor and only 128 megs of memory. The weight/size savings and the increased battery life of the Foleo isn't a fair tradeoff given its very limited capabilities.

This is all really too bad, because if Palm is counting on the Foleo to allow it to succeed, I think the concept has some massive shortcomings, much like their claimed LifeDrive product, which didn't last very long. And I really like the Palm company; I still haven't found better PDA functionality on a decade-old Palm design compared to what is available on all of the Nokia phones I've looked at, the WinCE phones/PDA's, the N800, etc. So I want Palm as a company to stick around. But with Treo getting eclipsed by newer smart phones, and the Foleo not getting particularly good buzz by folks reviewing it --- and after I played with it, I have to agree with the majority of the reviewers that this is not the Next Big Thing --- I'm not sure how much longer Palm is going to be around.

... and you want to preserve Internet Radio, please consider calling your representative.