From Screenshots

As someone who did the same transition a while back, I remember going through that issue and then some. Memory usage, lack of integration with Google Contacts (exacerbated with the seamless contact sync of Android phones), a UI that makes it hard to manage multiple email accounts, and vertical space being taken up by the menu bar.

Hope that’s of some use to some folks!

Org-mode is really quite tightly integrated with Emacs — so under normal circumstances there’s no reason to support it elsewhere. But at work we’ve recently had to collaboratively edit Org documents, and alas, Emacs’ Rudel no longer work reliably on newer Linux releases. We’re forced to use Gobby, which is a fine collaborative editor but is simply not Emacs. What is one to do?

I’ve thus started a GitHub project to create a language definition file for GtkSourceView (and, by extension, any editors on the GNOME desktop, including gedit and Gobby. Do check it out here — I’m adding support for language elements as I need them, and my schedule is rather busy these days, but feel free to file requests and/or enhancement patches (but if you do the latter, please include a statement licensing your contribution under the same MIT license you can see in the header of org.lang

Note to self — figure out how to integrate the Python tool used by URM, cnunu, to keep track of non-Fedora software. My 0install feeds could use some release monitoring love…

Matthew has been working on native backlight control for a while, and for Intel hardware, there’s currently only one patch left to merge onto Linus’ kernel tree; it applies cleanly onto the most recent kernel release candidate (3.0-rc7).

Unfortunately, by default the ACPI subsystem will still be used if available, which is the sensible default. You do want to use the predefined backlight values whenever possible, not the raw values the graphics card let you set.

Ubuntu users have been resorting to Kamal Mostafa’s linux-kamal-mjgbacklight repository, which enables native backlight control, disable the ACPI video driver, and provide a patched GNOME Power Manager that can interface with the native backlight control.

The workaround I came up with is more lightweight — it just uses inotify-tools to monitor the brightness file, and apply an appropriate equivalent value to the native backlight control. Feel free to use this if you’re affected by a similar problem.

• the distribution package might be out of date
• legal reasons prohibiting the software from being packaged (e.g. Skype, Flash, Adobe Reader)

Note that the first point is not exactly a criticism — after all, distributors tend to be wary of introducing breaking changes in a stable release. For software in the second category, upstream often provides binary packages, but again, using a tarball requires users to deal with dependency resolution themselves, and even when Debian or RPM packages are provided, the packaging is often sub-par (upstream developers can’t be expected to be well-versed in the subtleties of each distribution’s packaging).

Enter 0install. Now installing, e.g. Thunderbird 5.0, is a simple process:


yum install zeroinstall-injector
0alias thunderbird5 http://mojo.informatik.uni-erlangen.de/interfaces/2011/thunderbird.xml


or use “Add New Program” from the application menu and provide it with the URL for the Thunderbird feed. This currently lets you easily select between Thunderbird 5.0 beta 2 and 5.0 final (for both 32-bit and 64-bit builds) as well as the distribution’s packaging (on RPM-based and Debian-based distributions as well as Gentoo), and will pull in needed dependencies (please report any problem here).

You can browse http://mojo.informatik.uni-erlangen.de/interfaces/2010/ and http://mojo.informatik.uni-erlangen.de/interfaces/2011/ to see other available feeds that I maintain (and 0install’s site for even more). Of note: Eclipse JEE, Maven 2.2.1/3.0.3, Skype and Tomcat.

Firstly acquired is the 15″ Vaio EB I use as a desktop replacement. It works fine with stock drivers — at least, no known problem until today. We’ll get back to it later. Next, the 10.2″ Vaio W netbook. With this the wireless driver would need to be cycled — unload and reload — unless it’s kept active by, say, a continuous ping session.

Now, the EB did not use to work with the experimental compat-wireless drivers — basically the wireless code not yet merged into the Linux kernel; while the W absolutely requires these to avoid the ping keep-alive workaround.

Then comes the 13″ Vaio Y I got on a closing sale — at a nice discount price. Both the EB and the Y need to be on Fedora 15 because of graphics quirks (the former Radeon 5650, the latter Intel Arrandale), but that’s for another article. Everything seems to work fine, until I realized today, attempting to transfer a large-ish (> 100 MB) tarball from the Y to the EB, that the wireless on the Y keeps freezing up if I continuously transmit! It’s not a regression, or not a recent one, since older kernels do that too.

In goes compat-wireless to the rescue, and lo and behold, the transmission now works fine. Only then I find out that now the EB also acts up when it’s downloading at maximum speed. Sigh… funnily, now compat-wireless works fine on it, and fixes the problem.

I don’t even think I want to file bugs on this — different issues on different laptops — I’m just glad the latest wireless code works uniformly well now, and am just waiting for it to land in the mainline kernel.

1. Patents — if some entity holds patents that apply to a format, your ability to distribute your files might be compromised by the need to pay patent royalties. Even if the patent holder covenants not to exercise enforcement of the patents, they or the patents could end up being bought, at which point who knows what could happen. Even Microsoft got into trouble with Alcatel Lucent; the case was later thrown out of court but only after a headline-grabbing $1.52 billion award of damages was initially awarded. And Alcatel is not even a patent troll! Best protection is to use software licensed with a retroactive patent retaliation clause (e.g. Apache license, Eclipse Public License, GNU Public License v3) and whose copyright holders and distributors are in a defensive patent pool such as the Open Invention Network
2. Format obsolescence – even NASA has had trouble reading precious sensor data from old punch cards and magnetic tapes generated by previous missions, because the documentation for the file formats have been lost!
3. DRM — Digital Rights, or Restrictions, Management, depending on which side of the coin you’re looking at it from. It’s not impossible to create a DRM policy that is flexible enough to guarantee you your fair use rights you enjoyed with older analog technologies — the printed book, the audio CD — but it’s not in the interest of (most) publishers and distributors to do so. Unless forced by regulation or unless you vote with your wallets.
4. Walled gardens — remember the pre-Internet days of AOL, CompuServe, etc.? We still have walled networks, they are just built on top of the Internet instead.

Office Documents

Reports, presentations, … you get the drift. I try and stick to LaTeX for the first two, if possible (generating read-only PDF for dissemination) and OpenDocument Format, authored on LibreOffice, if collaborating with a non-LaTeX-user. And for spreadsheets.

Why, you might ask? Well, LaTeX just typesets much more beautifully than other alternatives I’ve seen. It’s a solid, well-understood format, and have very few compatibility problems over the years. Compared to Microsoft Office formats — Word’s .DOC being the most notorious, with worms and newer versions not being able to render old files perfectly! Microsoft’s “Office OpenXML”, their new file format, only became a standard after a process as dubious as Japan’s sponsoring of landlocked countries to join the whaling commission to supplant its voting block. And the standard is not even implemented by Microsoft itself.

Podcasts

Most podcasts are published in MP3; some are available in the patent-free Ogg Vorbis (.ogg / .oga) format — sadly, mostly limited to free / libre / open source software (FLOSS). A rare few are available in MPEG 4 Audio / AAC (.m4a).

I try to subscribe to the Ogg feeds whenever possible. MP3 is patent-encumbered, and the last patent won’t expire until 2017; while users of open-source MP3 decoders have not been sued for infringement yet, the situation is legally uncertain enough that, if you look at your favorite large Linux distribution (be it Debian, Fedora, openSUSE or Ubuntu), none of them carry MP3 decoders, let alone encoders, in their main repositories. Even Microsoft has been hit by frivolous MP3 patent infringement lawsuits!

Consuming MP3 even when an Ogg feed exists for the same podcast would perpetuate the use of MP3 “because that’s what the customers want”. Buying a portable music player that cannot play any patent-free, open-standard compressed format (yes, I am talking about Apple’s iPod here) is even worse. It does not play Ogg Vorbis, it does not play FLAC; instead, apart from MP3 you get non-free formats that Apple is heavily involved with: .m4a, the DRM-locked variant .m4p, and Apple Lossless, which Apple invented for unfathomable reasons instead of using FLAC. It’s not even because they can then engineer DRM support; you can graft it on top of FLAC as well, if you want.

MP4 situation is similar to MP3, except that given that it’s a newer technology, the patents will take even longer to expire.

eBooks

I try to buy electronic, rather than paper, books whenever possible. With a semi-nomadic lifestyle, buying more physical books just makes moving more costly! And there’s the environmental aspect as well. It’s bad enough that we need to clear forests to grow enough food — and eating excessive amounts of meat makes the matter worse because of the reduced energy efficiencies involved in adding another layer of intermediary between the sun and ourselves — it’s even worse when one unnecessarily gets documents in printed form. Now, used books are another matter altogether.

For eBooks, tech publishers like (in alphabetical order) No Starch Press, O’Reilly and The Pragmatic Bookshelf gets the nod for providing DRM-free products, with errata updates, in the major formats (ePub, the eBook standard; Mobi, an older format supported because Amazon’s Kindle uses a DRM-encumbered version if it, .azw, and does not read ePub; and PDF, for faithfull reproduction of the original layout).

Outside of programming references, alas, most publishers are not as enlightened. I must confess to being a heavy Amazon Kindle user, despite its limitations — not being able to lend my books out without restraints for one, not being able to hand over ownership is another. But at least I get to read the books on all my devices, unlike Apple’s iBooks with its five devices per book limit. Kind of nice having Amazon backing up the purchased books in case I lose a device, too; they’re starting to do that for music as well, though only in the US. Hear that, Apple?

Video

For many years, the only patent-unencumbered format available is Ogg Theora (which started its life as On2′s VP3 codec). On2 has since been bought by Google, and their latest VP8 codec becoming the basis of WebM, which is roughly equivalent in quality to MPEG4.

I try to get my videos from sites that allow videos to be downloaded (if the uploader allows for it) — e.g. blip.tv. and vimeo. Revvr, another service featuring this, sadly was a commercial flop and is no longer available. These sites allow you to download videos, dating back to the time when YouTube not only does not allow that, as they still does not, but also limit videos to lengths of 10 minutes! There are workarounds to downloading YouTube videos, but officially you’re not allowed to do that.

Social Networks

i.e. Twitter, Facebook, etc. I have switched to posting mainly to identi.ca, a Twitter alternative running software from Status.net. Unlike Twitter, it supports federated social networking — you can talk to people running independent Status.net installations, much as in instant messaging, users of XMPP networks (various Jabber networks, Google Talk, LiveJournal, Facebook) can communicate with each other). identi.ca lets you push your updates to Twitter as well (even retweets and favorites) and to Facebook, so I really only post directly to Twitter if replying to a Twitter-only user (or the topic is too mundane and I want to keep identi.ca’s SNR high).

Facebook alternatives are not as mature yet; see EFF Deeplinks’ post on this topic for more information.

Online file storage

Here’s a mea culpa – I’m a happy Dropbox user (thank you, Dropbox, for supporting Linux clients — at least on x864 architecture). This is the exception that proves the rule, however: apart from Dropbox, I try to stick to online file storage solutions that at least have open-source clients for communicating with the servers and accessing your data. e.g. Google Apps (including Google Docs) are accessible using googlecl, which uses the Google Data APIs; both are open source. If you move to a platform that is unsupported, you can port the software yourself.

With googlecl, if one is paranoid, one could encrypt all one’s files (with, say, GnuPG) before storing them on Google’s cloud. Hmm, that would be an interesting project to attempt…

Dear Mr. Jefferson,

I’ve been listening to your show for a while, and I’m glad to report it’s been a fascinating, refreshing experience to have a humanities scholar presenting a historically-informed portrayal of how you would have reacted to present conditions (and humble enough to follow Wittgenstein’s dictum, „Wovon man nicht sprechen kann, darüber muss man schweigen.“).

All too often, people who claim to represent a historical figure’s tradition tend not to have a clue what they are talking about — witness the “Jeffersonian” Tea Party movement (Bachmann didn’t even know where Concord was!). Or the “strict constitutionalists” — who’d be aghast at your idea that the constitution be rewritten every generation! I was just thinking a few days ago about the need for better civics education, when discussing the lack of awareness of American and Canadian electorates of their own voting systems.

Or the bible-thumping fundamentalists (I do need to listen to your 794th episode on The Life and Morals of Jesus of Nazareth! .. but I digress.

What I’d like to ask is regarding your suggestion that California should be broken up to smaller, less unwieldy pieces. I greatly support the idea — my favorite books include The Nine Nations of North America (that suggests that north California does not belong, economically and culturally, with the southern parts anyway) and I read blogs like Lost States and Strange Maps with interest.

I do have a concern that it might be politically intractable. Not at the state level, but at the federal level! Now, you also suggest that the entire constitution be rewritten anyway, and I agree there, but isn’t there a connection between the two? Under the current system for electing the Senate, Republicans would very likely reject a partitioning of California — it’s bad enough that they have had two Democratic Senators for a while, but giving them even more? (unless the partitioning is gerrymandered and you create coastal Democratic and landlocked Republican states!).

I wonder what the citizens of Los Angeles would think as well, being cut off from their access to water resources in Northern California…

PS any chance you would have been a Linux user? Your support for the free citizenry would match well with the public commons of the free software, open source and free culture movements. You’d find the governance models of, say, the Fedora Project or Debian fascinating (disclaimer: I am a contributor with the former).

Yours sincerely,

—
Michel A. Salim

That this works was a pleasant surprise to a friend and I — he thought the Java plugin does not work with Chrome, and I did not realize the 64-bit version is out (OpenJDK has had a plugin since it was called IcedTea, but the Java applets I need to use tend to make use of the Sun/Oracle binary blobs not yet reimplemented).

Getting it to work with Chrome is straightforward — like with Flash, you symlink $JRE_HOME/lib/amd64/libnpjp2.so to /opt/google/chrome/plugins. With Firefox 4, though, SELinux comes into play. Presumably because while Chrome’s executable is already marked as requiring an executable stack, Firefox 4′s (or rather, XULRunner 2′s) is not.

No big deal, you might think. The SELInux troubleshooter pops up, just follow its suggestions and all shall be well — pipe the relevant audit log lines to audit2allow, load the generated policy modules, close Firefox and try again. Except for… no cheese.

If this happens to you, save yourself trying to delete the symlink, reload Firefox, closing, recreating the symlink, and trying again. It appears that Firefox tries to cleverly remember which plugins fail to load, and it’s this cache that you need to purge. Delete ~/.mozilla/firefox/*.default/pluginreg.dat, and you’d be set on the next restart.

Now Xmarks is owned by LastPass, and hopefully will be integrated soon with their own offerings — Xmarks’ password-sync is inferior, for instance, and if they offer a unified SSO with LastPass’ multi-factor authentication then that would be wonderful. I carry a Yubikey around these days, with the primary key set for Fedora’s Account System, and the second key set for Yubico’s public server for use with LastPass. If someone were to use a keylogger on a public computer that I use, then by all means; the password they tapped won’t be enough.

These all mean that I’m no longer trapped on Firefox 3, since the other extension without official Firefox 4 support, Greasemonkey, supports it in their nightly builds. Gmail is really annoying to use without Greasemonkey’s Gmail Fixed Font script! Makes one wish that Google integrates its Mail and Groups teams a bit better — Groups’ interface allows switching between fixed-width and propotional fonts for reading, while Gmail only offers it for composing. Sadly Groups’ new beta interface does not even have this feature implemented — ah well, there are three feature requests for it already, hopefully they’ll get it implemented before the new interface goes live.

EDIT – looks like Delicious will survive but will be spun off. Hope it gets taken over by a company that can give it the attention it needs — less laggy FFox support, and proper Chrome, iOS, Android support would be nice. Xmarks works well enough for me that I’ll probably stick with it though.

We’re now in 2015. I read books-related contents on a number of different devices: my smartphone, my high definition tablet, and even my PC some times. (I personally do not believe in TV for such products). I want spend a long weekend in Rome. Instead of buying a couple of books – one to organize my trip and another to use on location – I will buy rights to both.
…
My digital rights are also transferable: I can loan or give the book by simply transferring the rights attached to the digital files. In retrospect, this feature makes 2010 digital bookstores look primitive. For instance, in the Apple iBooks Store, I was forbidden to offer a book to anyone or even to access to a iBooks in a foreign country – thus negating key advantages of dematerialized contents.

Some of these are already possible nowadays — tech-savvy publishers such as O’Reilly and The Pragmatic Bookshelf already let you regenerate your digital content in the format of your choice, though normally limited to PDF, ePub and Mobi/Kindle only. But the ePub format should be flexible enough to display differently based on available screen size and multimedia support. There are free self-publishing options with sites such as FeedBooks. There are music sites that sell music files in multiple formats — though so far, only Magnatune (which has shifted entirely to a subscription model) allows redownloading while the others (Pristine Classical, HDtracks, iTunes) make you commit to the format of your choice at purchase time.

There are several limitations with these sites that might prevent more widespread adoption, though:

• Reliance on customers’ ethics: for example, The Pragmatic Bookshelf uses an easy-to-remove watermark; O’Reilly does not use any DRM or watermark; likewise with Magnatune, Pristine Classical and HDtracks. While admirable, and I personally find it a moral obligation not to distribute digital products entrusted to me in such a manner, many publishers would likely balk at the idea.
• Conversely, overly-restrictive DRM: the Amazon Kindle, Apple iBooks and B&N nook stores fall in this category. B&N innovated the ability to lend books but in a very restricted fashion: you can only lend to a unique user once, and only for two weeks. Why? Physical books can be passed around without constraints. Amazon, sadly, chose to copy this model without any change. It’s sad, when public libraries that offer DRMed ePub lending do not have such restrictive conditions; the problem is not technical but commercial motivation.
• Inflexible purchasing model: with the non-DRMed publishers you normally commit in advance to buying the rights to a work in digital-only form, or in a paper+digital bundle. You can’t “upgrade” from digital-only to paper+digital (though upgrading in the reverse direction is sometimes possible). You can’t order a replacement physical media if yours is damaged (though, again, software games publishers used to offer this). You can’t, of course, “upgrade” from a DVD to a Blu-Ray edition of your favorite movie without paying the full purchase price.

The solution, as I see it, is to separate right of use from the physical artifact. Publishers should be able to sell the two independently from each other (though obviously there is a dependency of the latter on the former). You should be able to permanently deactivate your rights license code on an artifact (for reselling), or temporarily deactivate it (for lending), and set a time limit on the latter. After all, in the information age content is getting increasingly Platonic: living in pure digital form, only to be instantiated into tangible physical forms — we should focus less on the latter and more on the former.

The train just left Switzerland, so I’m back on my netbook, but data coverage is still a bit spotty. For a while the phone actually picked up Austrian cellular networks!

It’s the first time I’ve been to FUDCon, and I’ll definitely be coming back. Though alas, the next one in Tempe, Arizona would be rather hard to squeeze in, so it might be a full year wait. It’s a wonderful energizing experience. The north American folks are great presenters (unfortunately I missed out on Jesse’s dist-git talk), and luckily one of my preferred Barcamp topics (Peter Robinson’s Fedora Mini Mobility) ended up on the morning slot, since I regrettably had to board the train right after lunch, missing all the afternoon sessions. Some ideas came up that I’ll try and follow-up on fedora-de{sktop,vel} as soon as I get back.

Oh, and for anyone thinking of visiting Red Hat’s Czech office — or Novell’s Czech office, for that matter, beware: they’re all party animals! The bunker was well stocked with beer and stronger stuff (including Swiss absinthe, which I was informed has the highest thujone content in the world, Switzerland happily not being part of the EU). ’80s Czech music is awesome, and MIsha regaled me with a Russian narcissus/Father Christmas tale, that surprisingly had a happy ending. Not many photos, unfortunately, but between Maírín and Nicu I think the event was well-covered.

No Internet on the train (I could have used my phone as a modem until the Swiss border, but between a paper, an eBook — Peter Hamilton’s Pandora Star), no Internet in the hotel, so I had to wait until I arrive at the conference venue for my Internet diet. Finally got the liboauth/bti EL-5 updates finished.

Adam’s just starting his talk so it’s time to sign off now.

Your editor will confess that he still feels a certain childlike joy at the prospect of reflashing an expensive device that he depends on, possibly bricking it, then painfully restoring all of the settings and discovering all of the new bugs which have been added. It’s the sort of adrenaline experience that others, perhaps, seek through horror movies, bungee jumping, investing in equities, or PHP programming

Jonathan Corbet, The end of the road for the Nexus One, Linux Weekly News

Well, the feeds are still there, but hidden from plain sight — turns out, though, that if you pretend to be iTunes, you can actually trick the iTMS server into giving you the raw data. And with Python 2.6′s built-in support for Apple’s property lists, extracting the feed is a trivial matter.

#!/usr/bin/env python

import plistlib
import urllib2
import sys

ITUNES_VER = '7.4.1'

USER_AGENT = 'iTunes/' + ITUNES_VER

def get_props(url):
    request = urllib2.Request(url)
    request.add_header('User-Agent', USER_AGENT)
    response = urllib2.urlopen(request)
    return plistlib.readPlistFromString(response.read())

def get_feed(url):
    next_url = get_props(url)['action']['url']
    props = get_props(next_url)
    return props['items'][0]['feedURL']
    
if __name__ == '__main__':
    for url in sys.argv[1:]:
        print get_feed(url)

update (2010-08-27)
Apple removed one level of indirection, and the code above can be simplified. For historicity, I’m not editing the displayed code, but apply this patch, either by hand or using ‘patch’:

--- a/itms_extract.py
+++ b/itms_extract.py
@@ -15,8 +15,7 @@ def get_props(url):
     return plistlib.readPlistFromString(response.read())
 
 def get_feed(url):
-    next_url = get_props(url)['action']['url']
-    props = get_props(next_url)
+    props = get_props(url)
     return props['items'][0]['feedURL']
     
 if __name__ == '__main__':

update (2011-04-05)
The latest code, for your convenience, at gist.github.com

(as an aside, the only way to flash an image to a partition, instead of wiping the entire disk, is using Fedora’s tool, and by extension MeeGo’s — but with the latter, only if one used the Fedora-derived command-line tool, mic-livecd-iso-to-disk instead of the recommended ones)

There are several reasons for this, in no particular order:

Storage

Moblin supports ext3 but not ext4; MeeGo adds btrfs to this mix but there’s still no ext4 support. I buy the rationale that ext4 is the SVN of file systems, and that we’ll eventually all migrate to btrfs anyway. But on the other hand, btrfs is not quite there yet — I switched back to ext4 after the SSD failure, when I realized that btrfsck does not yet handle bad sectors properly, unlike e2fsck. With other Linux distributions increasingly switching to ext4 — which can be easily migrated to btrfs later on — the trade-off (slight disk usage increase vs easy access to other Linux partitions) is surely in favour of supporting ext4.

Moblin/MeeGo also does not support LVM, which is used by default by Fedora’s installer, but this is a minor issue — someone deciding to use MeeGo and Fedora can just partition the disk without using LVM.

connman vs NetworkManager

I used to be agnostic as to how my network connections are configured — as long as it just works. This is why my new netbook is a Sony — because unlike other vendors (shame on you, especially, Dell) it does not use a Broadcom WLAN chip with a proprietary, badly-documented, buggy Linux driver (shame on you, Canonical, for helping develop it if only had Canonical put its considerable weight to getting specs and firmware released). Instead, it has a nice Atheros chip.

The same is true when it comes to the software stack. At the beginning I did not pay much attention to the connman vs NetworkManager controversy — prior to version 0.7, NetworkManager had its warts, and connman happened to work just fine on the home wireless networks I tried. This changed when I try connecting to a work network that uses 802.1X security. The MeeGo GUI does not support this, and the developers don’t consider this important at all. The command-line tool is badly documented — documentation is non-existent — and throw cryptic error messages. Makes one wish Java-style checked exception is more widely used; that way, at least developers have to *think* about the exception propagation, instead of just exposing them by neglect to the users.

GNOME 3.0, Fedora MeeGo stack

Fedora 13 already comes with a preview of GNOME 3.0′s shell, and work is in progress on having  MeeGo available. There are some annoying integration issues — the network-manager-netbook applet that integrates the Moblin/MeeGo desktop with NetworkManager is not as well supported as the default connman-based applet, and MeeGo’s window manager has some incompatible modifications that has not been merged back upstream. But, like NetworkManager’s Dan Williams, I believe that the way to improve the Linux desktop is to improve the existing tools that work across all sorts of devices, rather than reinventing them to solely target mobile devices, and in the process having to solve the same technical issues over and over again.

Open source drivers for AMD (née ATi) and nVidia cards are improving — and one is grateful for AMD to actually cooperate with open-source developers with documentation and technical help, but at the moment one is caught in a three-way bind: buy Intel and be stuck on the low end (or very limited vendor choices), buy nVidia and get great proprietary drivers and good open source drivers, but supporting the company with the most FLOSS-unfriendly business practices, or buy AMD/ATi and have good-ish proprietary drivers (provided one downgrades one’s Linux install or at least the X components) and so-so (but improving) open source drivers. Being stuck in the latter camp, I was running the open-source Radeon driver, which currently has no DRI support for the Radeon HD 5400 series (no gnome-shell. Not even gthumb, nowadays!) — but then noticed that an older problem might be resurfacing itself — that my graphics card is not being throttled down, contributing to the awful (~ 1 hr) battery life on Linux. That’s about the last straw one can take: my old netbook has snazzier graphics and better battery life[2] than my new notebook!

Going to try the newly-updated proprietary Catalyst driver, coupled with a downgraded X installation from Fedora 12, and see how it goes. Will report my experience here — and recuse myself from submitting X and kernel bug reports until the next version of X comes out and hopefully make the situation less painful.

First time in many, many years using proprietary graphics drivers, but I’m not killing my battery and my hearing (the fans are rather loud) over this.

[1] ports to new APIs tend to introduce periods of instability and performance regressions, but overall the impression is positive
[2] after close to two years, battery capacity just dropped precipitously to ~ 30% of the original, so it’s now getting a new — and higher-capacity — replacement. This is probably the last upgrade — there better be a dual-core, 2 GHz+ netbook out by the time the new battery fails (or a well-supported, affordable ARM smartbook), and it better has SSD options (Dell, what happened to your great Mini 9 SSD deal?)

FSFE is certainly not a Christian organization. One could argue that it is a by-product of a traditionally Christian civilization, but one could equally argue that it traces its heritage to Greco-Roman philosophy! The green cross, with a slit on the bottom so that the entire shape looks like an icon representing a person standing with open arms, is probably closer to the Red Cross in iconography than to Christianity. That being said, being a fellow does have some similarities to being a committed Christian.

• The cross we bear: Joining has a price, whether financial, in time commitment, or other means
• A mission: we bear this price gladly because we believe in what the respective organizations stand for. In case of the FSFE, it’s freedom. Freedom to learn. Freedom to innovate. Freedom from unreasonable restrictions imposed on you by software patents (at this point, I’d like to extend a special welcome to any budding cinematographer who just discovered that by recording your video in H.264, the MPEG-LA consortium owns your soul — er, I mean your work)
• Diverse voices: just as Christianity is represented by a myriad denominations, some with higher profile than others, some with a more tarnished reputation than others, yet all based on the same foundation — no matter how garbled in the transmission (we are all humans!), the same is true of the Free, Libre and Open Source Software (FLOSS) community. There are differences between the free and open source camps, between the copyleft and liberal-license camps, etc. But deep down we believe in sharing our works with others, whichever way we justify it to ourselves and others.

The struggle against the anti-commons nature of overly-restrictive intellectual property will be with us for a long time. We have made huge technical leaps — FLOSS software is competitive in diverse fields including server operating system (Linux, the BSDs, OpenSolaris), instant messaging (Jabber, standardized as XMPP), audio codecs (FLAC, Vorbis, Speex), and are catching up in video (Theora, Dirac, and thanks to Google, WebM, née VP8). Even users still locked into proprietary systems can thank FLOSS, and open standards, for the Web they surf (served mostly by Apache), their web applications (often built on top of the Java platform), and further down, the network protocols they use, all developed in collaboration instead of in proprietary isolation.

Yet the road ahead is a long and winding one. Flash is still omnipresent on the Web, Apple is proving a huge disappointment (after contributing to, and sponsoring, so many open source projects, now they’re starting to shrilly attack any competitor to their iPhone/iPad lines — be it Android, Flash, Theora, or WebM). To quote Benjamin Franklin,

Those who would give up Essential Liberty to purchase a little Temporary Safety, deserve neither Liberty nor Safety

I’m not giving up my liberty, and neither should you. It’s true that it is hard to completely give up proprietary software — don’t be discouraged, many free software advocates are not there yet either. But you can start by following these simple guidelines:

• Favour open standards — does your calendaring solution support the ICAL format? does your mail provider provide IMAP and POP3 access? is your instant messaging platform XMPP-based (e.g. Google Talk), or are you locked onto a proprietary protocol?
• Vote with your wallet — if a company has a history of abusive behavior (sadly, Apple is now there), attempt to discourage this kind of behavior. Don’t buy the products they’re trying to protect by this behavior, tell them why you’re not buying, and tell other people why too.
• Be aware of your rights — you have the right to make a personal copy of your music and movie collections. Yet the RIAA and MPAA tries their hardest to make this impossible — in case of DVDs and Blu-ray, to the point of making it illegal

I highly recommend reading Against Intellectual Monopoly and Gridlock Economy; both are accessible and highly illuminating accounts of the damage our current legal IP regime is doing to our societies. The solution is not anarchy — copyleft licenses *are* legal copyright documents — but to work for reform; if you agree, consider donating your time — or money — to organizations such as the Free Software Foundation, its affiliates — including FSFE; the Open Invention Network; your favourite free/open source project (whether in code, documentation, useful bug reports or donation); or projects that enrich our cultural commons by making public domain information more accessible — e.g. Wikipedia and Project Gutenberg.

I thank you. Future generations will thank us too, for whatever little we can do for them today.

You’d want to install the latest version, rather than relying on the version bundled with Emacs (since version 22) — otherwise some features described in the documentation simply won’t work (e.g. quick-selecting the initial state of a TODO item).

It is necessary for substitutions to behave as values — if you take a substitution, and add a binding, you better get a new substitution, rather than do the modification in place, because that substitution might be used by other goals. In Lisp, the traditional data structure for this is the association list: a list of lists, where the cars (first elements) of each inner list are the keys. Clojure takes Phil Bagwell’s Hash Array Mapped Trie and adds persistency on top of it, to the effect that you have a copy-on-write trie map, with logarithmic rather than linear lookup times.

Technorati Tags: functional, logic, programming, clojure, scala, jvm, data-structures, tries

Using association lists, my Scala code was performing about 3x slower than Petite Chez Scheme. Horrendous results — interestingly, using linked case objects optimized for storing keys and values reduce the gap, to the point that it was only about 10-15% slower. Still not great, but acceptable. Since my Mini Kanren implementation is written in Scala, not Clojure, I tried using Scala’s own immutable maps, reasoning that with Bagwell tries, the boost in look-up performance (O(log n) vs O(n)) ought to outweigh the increased cost of extending (O(log n) vs O(1)). Alas, Scala’s maps are not Bagwell tries. One simply gets an OOM error; I’d have to dig deeper to find out exactly why.

The solution, of course, is to use Clojure’s maps in Scala. Doable, after patching its Java interface a bit — Clojure being a Lisp, there is a penchant for short names, and therefore an IMapEntry has a getter called val(), instead of value() — but val is a keyword in Scala (denoting values, i.e. names, like variables but immutable). Patched code lives in a branch on my Clojure fork; hopefully this can get merged in as a stop-gap until Scala gets its own tries.

And with Clojure maps? Scala Kanren is now 2.5 times *faster* than Mini Kanren on Petite Chez. Mind-boggling. Oh, and you’d want the raw numbers, naturally. Bear in mind that this is used on a desktop system that is running Firefox, etc. at the same time. I’d need to do longer runs (e.g. 100x) from runlevel S to get better results.

Next speed-up is probably going to be obtained by reimplementing this back in Clojure — it has a built-in parallel map that works on lazy sequences, unlike Scala.

After you get the code, there’s still the matter of setting things up. And SVN is a rather messy interface to use if one does not have commit access…

I’ve decided to take the plunge, though. I used git-svn to make a clone of the official repository, committed some usability improvements to a bugfix branch, and pushed the code to GitHub. Get it here (the master branch is Jon’s latest SVN code; the bugfix branch has a Makefile added that let you run Gnoetry by simply running ‘make’ — it does all the setting-up for you. ‘make clean’ cleans up the generated files).

Now, time to fix that unreleased-mutex-on-exit bug…

Technorati Tags: gnoetry, linguistics, python, poetry, statistics

So far, I’ve been using it to brush up on my Clojure, Pure and Scala fu. I’ve just added the Pure category, so the examples are a bit sparse still. If you want to see the solution to any particular problem in it, just drop me a note.

Also check out the stair climbing exercise I borrowed from Chung-chieh Shan’s Lambda the Ultimate post.

• During the package review process for pure, we discovered that a dlsym-ed strcmp does not produce the right result on F-12
• The same problem turned out to be the case with avahi-sharp; its dlsym-ed strlen produces incorrect results, causing segmentation faults in Banshee (which uses it through mono-zeroconf)
• Switching zeroconf to use avahi through DBUS would fix things, but when poking through the sources, it is discovered that its developer has been bundling a private copy of NDesk.DBus, against Fedora guidelines and the wishes of NDesk.DBus upstream
• Said upstream has not released a stable release since January 2008, though the code is still actively developed in Git, and there is a (quiet) bugtracker on Launchpad. Nobody appears to be using the upstream code; even openSUSE Factory is still using 0.6.1 (not even the latest 0.6.1a).
• Meanwhile, Fedora’s ndesk-dbus carries a patch, at the request of sugar-sharp, that changes the public API by adding some public methods. The API version, however, is unchanged.

I’ve notified upstream, as well as the zeroconf developers. The Fedora bugs affected are linked from the upstream bug report as well. Meanwhile, let’s hope the glibc bug gets fixed soon too. It’s nice having a seconder, especially as the test case is much smaller than the test case I submitted for pure.

Syntax
Implicit conversions now allow you to write the more idiomatic x === y to mean mkEqual(x,y) . Likewise, you can use x =/= y to mean neverEqual(x,y) (see below for explanation).

Semantics
The =/= (neverEqual) goal has been added, specifying that x and y should never be unified with each other. This requires reworking the substitution system; previously, a substitution is simply a list of var -> x mappings, together with assorted functions.

Supporting neverEqual involves adding more checks to unify; in addition, functions like lookup need different implementations depending on whether they operate on simple or extended substitutions.

The implementation is now more object-oriented, using a pattern similar to Haskell typeclasses:

• Subst specifies the operations available to any substitution, together with default implementations for a simple substitution, whenever possible
• ConstraintSubst extends Subst. It does not actually add any more methods, but it overrides the implementation of some methods (such as unify)

There is a reason both of these provide the same public interface: it allows the same code to run, with either a simple or a constrained substitution, with no change at all (neverEqual-related operations just become no-ops if a simple substitution is used).

Performance
Surprisingly, the change to the substitution system actually results in a performance boost. Previously, the palprod_o benchmark takes around 110 seconds to run on a Core 2 Duo (single thread), compared with around 40 seconds for Petite Chez Scheme. The new code, using linked case classes instead of linked lists, come in at around 44 seconds. (fastest time with constraints disabled is around 43.5 seconds; with it enabled (but no constraints) the overhead is around a couple of seconds).

More systematic benchmark numbers will be forthcoming.

Misc
The standard prelude and examples are now better documented. The examples now have their own namespace (info.hircus.kanren.examples), and a second example, SendMoreMoney, has been implemented.

Technorati Tags: functional, logic, programming, kanren, mini-kanren, scala, jvm

It has an almost-complete numerical stack — the missing arithmetic relations are not coded yet because I was documenting and/or getting larger test cases to work — and likewise with list-processing support. The years-old stack overflow problem I initially attributed (when taking a programming language course, and having no time to debug thoroughly) to Java’s lack of TCO turns out to be fixable by some judicious call-by-name optimizations.

Having just discovered metalua and its macro goodness, I’d say that after bringing Kanren to the JVM, the next step would be to have a C-embeddable Kanren written in metalua. There might be some performance snags, though — the Scala port is currently about 3x slower than Mini Kanren running on Petite Chez Scheme.

Technorati Tags: kanren, mini-kanren, functional, logic, programming, lua, scala

Being more familiar with the former, over the weekend I decided to try and write an equivalent module. Two problems came to mind:

• Python uses an @abstractmethod decorator to mark an otherwise-normal function definition. While a decorator pattern can be used in Lua (though not a standard practice), this Python decorator works by setting an attribute in the function it decorates:
  

  funcobj.__isabstractmethod__ = True

  This is not an option in Lua, because functions don’t have individual metatables.

• Python has a standard way of doing OOP, and the enforcement that an instantiated class does not contain any abstract method is done in the __new__ method of the ABCMeta metaclass. Contrast to Lua, where OOP is normally done in an ad-hoc manner — and even the LOOP library provides several OOP inheritance mechanisms.

The solution I adopted is delightfully simple: the abc module provides two functions: an amethod function that throws an exception no matter what arguments it is passed, and a verify function that takes a classname, and reflects on the members (using loop.cached.allmembers if possible, falling back to pairs if loop is not installed) and check if any of them is equal to amethod.

This works even on simple tables, though it shifts the responsibility a bit to the programmer: unit tests should be used to verify each created class. As a fallback, attempts to use the abstract methods will fail at runtime, but that’s taking dynamism a bit too far…

Still working out how to get the Pythonic behavior (instantiating a class with some abstract members should fail) when using LOOP. Meanwhile, try it out for yourself and make sure to file bug reports!

Technorati Tags: lua, python, oop, abc, abstract-methods, virtual-functions, interfaces, traits

While looking if this has been done before, I came across a rejected Python Enhancement Proposal:

The common case for list operations is on small lists. The current array-based list implementation excels at small lists due to the strong locality of reference and infrequency of memory allocation operations. However, an array takes O(n) time to insert and delete elements, which can become problematic as the list gets large.

This PEP introduces a new data type, the BList, that has array-like and tree-like aspects. It enjoys the same good performance on small lists as the existing array-based implementation, but offers superior asymptotic performance for most operations. This PEP proposes replacing the makes two mutually exclusive proposals for including the BList type in Python:

1. Add it to the collections module, or
2. Replace the existing list type

It is currently rejected, but could be added to the collections module if there is sufficient outside interest. This is not quite the immutable vector from Clojure, but close enough: one merely needs to subclass it, and modify the setters (__setitem__, __setslice__, etc.) to first copy the collection and then operate on the copy. Copy-on-write ensures that only the changed leaf is actually copied, plus the internal nodes on the path leading to the leaf.

I’ve packaged this for Fedora (review request). Anyone cares to review it?

Technorati Tags: fedora, linux, python, data-structures

The question is: short of totally overhauling the system, is there any way security could be improved?

My (preliminary) ruminations on the topic yields the following:

Pre-authentication
For customers who opt-in, require texting the transaction amount (optionally +/- a given amount) for transaction (esp. above a certain amount) to be approved. Problem: SMS is not a secure medium. Using a smartphone, one could get around this (but then the proportion of customers covered will be much smaller), but given that data connectivity is not common (esp. in US and Asia), we’d still be limited to SMS.

If we care only about authentication, the cleartext plus its PGP signature would fit inside 160 chars, but if one wants to encrypt the content as well, it’s not possible.

e.g. for the cleartext

2009-10-06|11:30:07|12.34

(the timestamp is needed to prevent replay attacks. Yes, this is obviously not secure enough still, but the example is to illustrate the transmission size problem).

Signed, the signature takes 104 bytes. Plus the 26 bytes of the message, and a token separator, we get 131 bytes, within the limits. But what if the message is to stay private as well? Using GnuPG, I get a message size of 625 bytes. This could be split into multiple SMSes, but it’s not convenient.

Post-authentication
Have the card issuer send an SMS *after* an authorization request is received. We still have the transmission size problem above, but the issuer can choose to transmit less sensitive information — e.g. rather than the amount, transmit the merchant identifier. Still a privacy problem, and this will obviously not be popular on a busy check-out line. Also has the problem that to make it secure, you’d need a smartphone (to either decode the message, or verify the signature).

Secret code
There already are security mechanisms asking for the CVV/CVC code. Make it ask for a secret number instead, one that is settable by the customer.

But above all…
Raise the base level of authentication required! Some online vendors like Amazon still do not even verify the billing address (which is convenient, if one’s card is issued in a country like Indonesia, where for some reason address verification *never* works, but scary. Though, funnily, I haven’t gotten any card misused on Amazon. I *did* have one stolen card used on iTunes, so Apple’s authentication is obviously comparably weak).

Any other idea I’m missing, or any problem with the three schemes above that I have not noticed, let me know (comment or trackback) and I’ll update the post. Thanks!

Technorati Tags: banking, cryptography, security

Technorati Tags: visualization words

If you use a non-HP cartridge, an HP printer prints with less quality, to make the user think the HP cartridges are better. The same happens with batteries in mobile phones: some phones raise their antenna power to the maximum if you use a battery from another company, just to drain the battery and make you think the phone’s own batteries are better.

How ironic. Other cases spring to mind — the infamous Windows 3.1 beta, and Intel’s icc compiler choosing unoptimized code paths on AMD CPUs, but neither of those involve cryptography, unlike, apparently, the cases above.

[1] He of Rijndael / AES fameTechnorati

Tags: anticompetitive, authentication, brucon, lwn, security

There’s Google Voice, and other forwarding services, but that would require giving people a different number; one whose voice quality is probably not as good. And if the new number is only needed temporarily, that seems like an overkill.

Enter ContactBot. It provides a Twitter bot; you can tell it your phone number and location, and your friends (people you follow, not people who follow you) can then query the bot for your whereabouts.

The test bot is @hircus_contact. It is rate-limited at the basic 150 messages per hour, so if you want to heavily test it, or do not want to give me your phone number (I run the bot, after all!), feel free to check out the code and run your own bot!

Note that the bot does not currently add you back, so you’d have to wait until I personally add you before you can send direct messages. Should take less than a day.

TODO: timezone support, so the bot can warn your tactless friends not to call you at 3 a.m., documentation, and test cases. And online help.

Current command set:

• D hircus_contact set phone 555-5555
• D hircus_contact set location New York, NY
• D hircus_contact phone ma_cherie
• D hircus_contact location ma_cherie

Technorati Tags: bots, contacts, python, software, twitter

It’s particularly bitter-sweet when that happens when one — ok, I — was in the middle of moving the files to what I figured, as it turns out correctly, is a more stable solution: Btrfs. I might not want to have it on my root partition just yet — from past experience, running an RPM transaction on it can be rather tediously slow — but for a relatively small number of large files, it should do OK. And it did — though during one of the move operations, NTFS-3G gave up the ghost. Ah well.

Sufficiently impressed by Btrfs — seems to be holding well so far, while I’m catching up on some of the lost torrents — the next step is, of course, to run a home directory under it. Not crazy enough to put my main laptop to the test, I volunteered my netbook. Some recent Rawhide update means compositing on it is currently rather buggy — unusably flickering display with KMS, very slow without — so while I’m back on a plain GNOME desktop after having fun with the new GNOME Shell, I figured the netbook will serve as a Btrfs testbed. btrfs-convert happily converted 7G-worth of files from an ext4 partition in a couple of minutes. The conversion is even reversible, though after making sure everything is still there, I nuked the old image file. The important files are synced to the main laptop anyway (thanks, Unison!)

One small niggle — at the beginning, logging in through the console yields a bizarre error

No directory /home/michel!
Logging in with home = "/".

ls --lcontext shows that /home/michel is properly labeled — nice conversion job –  so this was at first puzzling. As it turns out, though, /home was not. Restoring it to the proper context fixes matters.

Will update if there are any further issues, but so far, Btrfs looks like it’s going to stay. Back to tinkering with GNUstep and Étoile…

Technorati Tags: fedora, linux, filesystem, btrfs

Secondly, between its support for a myriad of language front-ends, and its venerable age, GCC is not as easily extendable as it could have been. Hard to integrate it with a modern IDE, as Apple is noticing.

And lastly, LLVM IR is a *beautiful* assembly language to write to. Arbitrary-precision integer arithmetic? Check. Functions? Check. Support for various calling conventions? Check. Infinite SSA registers? Check.

Actually, that’s not even the last point. Clang also comes with a static analyzer, that lets you hook it into a GCC compilation process — amusingly, it does not seem to work yet when using Clang as the compiler. I’d be happy to stand corrected on this, though. It provides readable warnings in HTML!

And now all this is yours to test! If you are on Rawhide, install as such:

$ yum install llvm   # to install LLVM itself
$ yum install clang  # to install the clang compiler

Or get clang and clang-analyzer together by using

<br />$ yum install clang-analyzer<br />

Fedora 11 users will have to wait until slightly after 2.6 final is out. We are trying to get OpenGTL packaged, and at this point it does not build against LLVM 2.6 due to API changes.Technorati Tags: compiler, fedora, linux, llvm, programming

It’s time to call a spade a spade. OLPC was a failure. Businessweek called it two years ago. Now, Timothy Ogden, editor-in-chief of Philanthropy Action has made a compelling argument to give up on OLPC. He points out that supporting de-worming programs has more impact on child learning than the OLPC laptops.  The laptops were designed without end-user input, they cost too much both to produce and to run, and they’re now being outcompeted by commercial laptops. Only about a million OLPCs have shipped so far.

Meanwhile, a commenter, Gansler, notes Shaikh’s rhetorical tendency:

Alanna Shaikh and I worked together. I found then, as I do now, that her rhetoric is a bit thick, a bit black and white – all is doom and gloom (though sometimes it’s awesome and bright). I find her “Dream is Over” comment very much in line with that.

While I do consider the OLPC laptop project itself rather a failure — its relentless pursuit of volume, its alienation of the teaching profession (the two are related, in a chicken-and-egg way: OLPC has to sell in spades to achieve low per-unit price; third-world educational budgets are limited; ergo some money has to be diverted from traditional, human-resource-heavy teaching approaches. Rather unethical at the very least, using thirld world students and teachers as human lab rats.

Whether it would have worked better had it targeted educational niches in the developed world — home-schooled kids, isolated rural kids wanting to collaborate with others across geographical distances, or as a supplement in general for public school education, since after all, studies /do/ show that a high proportion of US teachers are not qualified to teach their subject — is an exercise is uchronia. We’d simply never know. We do know certain things, though:

• OLPC single-handedly removes the linkage between ultra-portability and ruggedness on one hand, and price on the other. Remember the pre-netbook days of super-expensive ultra-portables? Intel responded by introducing the Classmate, which begat the netbook, and the rest is history.
• The OLPC user interface, Sugar, lives on in Sugar on a Stick. It’s bootable from a USB stick, can be installed, and runs on commodity hardware which are aplenty even in underdeveloped countries.
• The amazing LCD from OLPC is being commercialized by Pixel Qi, and is coming to a laptop near you within months. Obligatory plug: I maintain a Facebook group that (attempts to) keep track of Pixel Qi announcements. Some of the videos are just simply delicious.

So I guess this is turning into a eulogy. OLPC has failures in abundance, but it is a radical, category-creating product. Negroponte could have managed it better, but he should still be able to take pride of what the project managed to accomplish. You can’t make an omelette without breaking eggs, after all.

For those of you steeped in Judeo-Christian history, think of OLPC as the John the Baptist to netbooks’ Christ: loud, insistent, un-PC, and martyred, but a success nonetheless.

• mono-core contains some documentation tool, so those pull in monodoc. monodoc alone is 7MB, so plus the tools that are erroneously in mono-core, more than that is wasted at this stage
• mono-core also pulls in mono-extras and mono-web (how could mono-extras be extra if you always need to install it, you might ask? Good question! See bug 434709). These are not so bad, about 1.5MB in total
• *but* mono-web (Mono’s ASP.NET) is closely coupled with mono-winforms. And that’s 2MB.

This Gordian knot has now be unknotted. As evidenced from the following transcript, below the cut:

$ sudo yum install f-spot
...
Dependencies Resolved

================================================================================
 Package            Arch     Version                            Repository
                                                                           Size
================================================================================
Installing:
 f-spot             x86_64   0.6.0.0-3.fc12                     rawhide   2.0 M
Installing for dependencies:
 gnome-sharp        x86_64   2.24.0-6.fc12                      rawhide   260 k
 gtk-sharp2         x86_64   2.12.7-6.fc12                      rawhide   593 k
 mono-addins        x86_64   0.4-8.20091702svn127062.1.fc12     rawhide   365 k
 mono-core          x86_64   2.4.2.3-1.fc12                     rawhide   8.7 M
 mono-data          x86_64   2.4.2.3-1.fc12                     rawhide   912 k
 mono-data-sqlite   x86_64   2.4.2.3-1.fc12                     rawhide   104 k
 mono-extras        x86_64   2.4.2.3-1.fc12                     rawhide   886 k
 mono-web           x86_64   2.4.2.3-1.fc12                     rawhide   416 k
 mono-winforms      x86_64   2.4.2.3-1.fc12                     rawhide   1.8 M
 monodoc            x86_64   2.4.2.3-1.fc12                     rawhide   6.9 M
 ndesk-dbus         x86_64   0.6.1a-7.fc12                      rawhide    46 k
 ndesk-dbus-glib    x86_64   0.4.1-7.fc12                       rawhide    11 k

Transaction Summary
================================================================================
Install      13 Package(s)
Upgrade       0 Package(s)

Total download size: 23 M
Is this ok [y/N]: n

$ echo Now these are the only packages that are actually needed...
$ du -chs mono*.rpm
6.9M    mono-core-2.4.2.3-2.fc12.x86_64.rpm
912K    mono-data-2.4.2.3-2.fc12.x86_64.rpm
104K    mono-data-sqlite-2.4.2.3-2.fc12.x86_64.rpm
2.0M    mono-web-2.4.2.3-2.fc12.x86_64.rpm
1.9M    mono-winforms-2.4.2.3-2.fc12.x86_64.rpm
12M     total

$ sudo yum install f-spot
...
Dependencies Resolved

================================================================================
 Package           Arch     Version                             Repository
                                                                           Size
================================================================================
Installing:
 f-spot            x86_64   0.6.0.0-3.fc12                      rawhide   2.0 M
Installing for dependencies:
 gnome-sharp       x86_64   2.24.0-6.fc12                       rawhide   260 k
 gtk-sharp2        x86_64   2.12.7-6.fc12                       rawhide   593 k
 mono-addins       x86_64   0.4-8.20091702svn127062.1.fc12      rawhide   365 k
 ndesk-dbus        x86_64   0.6.1a-7.fc12                       rawhide    46 k
 ndesk-dbus-glib   x86_64   0.4.1-7.fc12                        rawhide    11 k

Transaction Summary
================================================================================
Install       6 Package(s)
Upgrade       0 Package(s)

Total download size: 3.2 M
Is this ok [y/N]: y

Download size before: 23 MB
Download size after: 15 MB

That’s roughly a one-third decrease in size — and that’s for an application that makes heavy use of the Mono framework. You will get better luck with anything that does not touch mono-web.

These should show up in Rawhide real soon now — and F-11 after the Mono updates currently percolating their way there have settled down.

I have several Aardvark invitations available. Reply if you’re interested!

This was one day after I came down with a really bad cold — still shrugging it off now. Productivity plummets to no end, but setting up a new computer is a more fun way to while away illness than reading a book (sorry, Orhan Pamuk; I think My Name is Red is your one masterpiece. The others are too meandering, especially when one can’t concentrate well).

The initial plan was to install Rawhide (ambitious!) over the CS department’s gigabit network, using the btrfs file system. This turns out to be unworkable — the r8169 driver mistakes the netbook’s 8101E as a gigabit adapter, whereas it’s only Fast Ethernet. I managed to get a DHCP lease once, on Dell’s Ubuntu installation.

Several network install attempts follow, over a 100 mbps link, using F11 alpha’s boot.iso. These all mysteriously fail, sometimes maddeningly close to completion. Even with ‘maxcpus=0 selinux=0′.

Giving up on this approach, I opted next for a hard drive install: use livecd-tools to put boot.iso into a thumb drive, and copy the ISO image to the drive’s root. It appears that this is insufficient — images/install.img has to be on the drive as well. This allows installation to succeed.

And then disappointment comes.

• btrfs checksumming makes any RPM transaction mind-numbingly slow
• Kernel panics. Even with maxcpus=0. This might actually explain the network install failures
• Kernel panic at boot using newer kernels

So off with btrfs and on with ext4. Too bad; the SSD optimizations in btrfs look nice. One wonders if it’s the early production status, or if it does actually impose a certain amount of computational overhead making it unsuitable to netbooks (or any single-core computers, for that matter. No, hyperthreading does not count).

I’ve had a fully up-to-date Rawhide all of Sunday and it’s a joy to use. 512 MB is rather usable, albeit forcing a certain discipline when it comes to browser tabs. Will put in the 2GB upgrade when the new wireless card arrives, so I don’t have to open the hood twice. Ironically, that card shipped promptly but has since been wandering the limbo of Chinese customs… why is it that the cool electronics products inevitably come from Chinese eBay sellers…

Remaining bugs:

• Cannot be used on Gigabit networks
• Broadcom WL causing hard lock-up with PEAP authentication
• Internal mic not working. Same problem on my other Dell laptop. ALSA can be such a nightmare..

The advantage of bundles is that it contains more metadata than a simple tarball.
Compatibility problems might arise, but at least the application launcher could provide more
meaningful feedback to the user.

Or even integrate with the OS’ version control system, especially now that Linux has
PackageKit, to say “hey, the user wants to run this new bundle that needs libfoo-x.y, install
whatever package is necessary to provide that”.

You’d need a package management system that automatically scans built packages for what
libraries they provide; RPM does that, not sure whether DPKG does.

To provide the context: the discussion was on how application vendors can easily target LSB 4 with a single binary image. My post was in direct response to an argument by another reader, that complicated schemes do not add much real value over tarballs, to which I begged to differ, arguing that the metadata available in bundles make the integration of binary applications much easier.

The issue of binary distribution triggers an allergic reaction from some people in the FLOSS community, a reaction that is, in my opinion, rather unwarranted. Even Debian provides, in their non-free repositories, stub packages that will download binary packages and create a standard .DEB package out of them. There are clear advantages to making binary-only applications more well-behaved, in fact the same argument for having package management systems with graph-based dependency tracking in the first place: dependency, dependency, dependency. When installing/upgrading a package, you’d want all its dependencies to be pulled in automatically. When upgrading a library, you want to make sure that all its dependents will still work. When there is a security vulnerability, you want a non-techie end-user to be notified, preferably within a fixed period of the vulnerability being made public (through periodic updates), or the next time the user launches the application concerned.

There have been attempts to create a one-size-fits-all universal package format, that’s distribution-independent and vendor-friendly. This is a red herring, IMHO, for the same reason that the Unix market splintered in the ’70s and ’80s, and that we have a proliferation of Linux distributions — and multiple independent BSD operating systems, each of them with their own ports tree (DragonFly being an exception in that they share NetBSD’s pkgsrc system). It’s nice to control your own packaging format, or if it’s a shared format (like RPM is), to control the naming conventions, etc.

What application bundles can do is provide the best of both worlds: vendors can ship binary-only bundles that declare dependencies in a least-common denominator format that the LSB can standardize, for example:

<br /><Provides>
  <lib>libbaz-a.b</lib>
</Provides>
<Requires>
  <lsb-version>4.0</lsb-version>
  <bin>convert</bin>
  <lib>libfoo-x.y</lib>
  <lib>libbar-z.w</lib>
</Requires>

The first time the bundle is launched, the launcher can add it to its index of available bundles. If any of the dependencies are missing, the system-native package management (or a meta management infrastructure such as PackageKit) is triggered to install the missing dependencies. The bundles themselves can be placed anywhere (though library bundles — in NextStep/OpenStep/OS X parlance, “frameworks” should probably be placed in pre-determined paths, e.g. /Library/Frameworks, /System/Library/Frameworks and ~/Library/Frameworks).

The only problem is that the system-provided libraries might not be ABI-compliant with the specified LSB standard, for example, libraries written in C++ after a compiler ABI change. There would probably be a need for the native packages to declare their compliance, or non-compliance, with LSB standards.

And one last nice thing about bundles: fat binaries. It’s easy to provide multi-arch bundles, and stripping away unwanted architectures is a simple rm operation.

This seems like a good argument in favour of adopting Debian-style package naming, when it comes to libraries: append the ABI version to the (sub)package name containing libraries that are linked to from other packages. So the old Firefox 1.5 would have a libgtkmozembed18 subpackage, that can be shipped with RHEL 5.2 without shipping the rest of Firefox 1.5 (technically speaking, the ABI version is 1.8.x.y, as prior to XULRunner 1.9 the ABI is ever-changing, but packages already handle this by depending on the specific version and release number anyway)

This is already done from time to time in the RHEL/Fedora world, in the form of compatibility packages, but making it the default would avoid this kind of breakage, where a package /has/ to be updated (due to upstream EOL) but parts of it are needed downstream.

This will be rather interesting to watch. The idea of writing an OS kernel in a strongly-typed language makes sense — witness House and Singularity. The effort is not expected to be ready for years — this is not Windows 7, and I’d guess there will still be a traditional Windows 8, even if Midori is ready by then.

Incidentally, the OSS community already has a strongly-typed virtual machine designed for efficient native code generation: LLVM. If one takes a Unix kernel (or, more practically, microkernel) and get it to compile using LLVM’s C front-end, one then has the opportunity to gradually rewrite it one module at a time in any language with LLVM front-ends. In the time it will take for Midori to get ready, would there perhaps be an ML-like front-end to LLVM?

clipped from arstechnica.com Every once in a while, an article gets posted somewhere in the blogosphere about how Microsoft needs to release a complete Windows rewrite, something along the lines of what Apple did with Mac OS X. Most people realize that Microsoft is in no position to pull a stunt like that at the moment; it’s hard to see Microsoft phasing out support for a billion-Windows-PC-strong user base, but that day may one day come, perhaps thanks to robust virtualization technology. But in the meantime, Microsoft has settled on rewriting bits and parts of the Windows operating system as it sees fit, with Vista being one of the biggest rewrites (a fact which partially explains the many hardware and software compatibility issues XP’s successor experienced at launch). Speculation around a non-Windows operating system in the works at Microsoft has been present for years, but recent trustworthy tidbits of information have found their way into the hands of Microsoft ZDNet bloggers Mary Jo Foley and Ed Bott.

]]> http://hircus.wordpress.com/2008/07/08/midori-ntvista-nt-win95/feed/ 3 Michel S. blog it http://hircus.wordpress.com/2008/06/21/python-tail-call-optimization-done-right/ http://hircus.wordpress.com/2008/06/21/python-tail-call-optimization-done-right/#comments Sat, 21 Jun 2008 18:08:11 +0000 Michel S. http://hircus.wordpress.com/2008/06/21/python-tail-call-optimization-done-right/ ]]> It occured to me this morning to revisit the issue of tail-call-optmization using function decorators in Python. Last time I checked, the working trick involves stack inspection (works only in CPython) and throwing an exception whenever a tail call is detected. In short: non-portable and slow. I posted an enhancement here that allows for mutual recursion (function A tail-calling function B tail-calling function A …), but it did not occur to me that the stack inspection hack, clever as it is, could be improved on.

Improved on it has: Miguel Perez is reporting that his solution runs pretty much as fast as normal looping. Supports mutual recursion and is completely portable too.

clipped from groups.google.com Please critique this tail call optimizing decorator I’ve written. I’ve tried to fix the pitfalls of other proposed decorators, and the result is this one that supports mutual recursion, does not use exceptions, stack inspection or any implementation-dependent hack, and is pretty short and fast – the fastest out of the ones I could find and try. In fact, in tail-recursive environments I tested the impact of using the decorator is difficult to even measure, as the extra time the decorator takes to run is probably saved by the better use of cache memory. The only caveat is that if used in a function that’s not called in a tail-recursive fashion, bad things will happen.

What
The goal is to evaluate the performance of middle-of-the-road, not embarrassingly parralelizable tasks on modern-day multi-core hardware. Such as the Sun T2000 servers. Fittingly, the task is to parse a multi-gigabyte web server log file and compile some aggregate statistics.

Design
The solution I came up with for the earlier iteration of the contest, coded in different versions (C++, OCaml and JoCaml) is fundamentally sound, though rather unoptimized (picking up two-and-a-half different languages in one weekend is a good way to find out how much there is to know about, say, C++ stream buffering). With the benefit of hindsight, and given that we are several weeks into the project and there are strong implementations already [wikis.sun.com], the idea is to find an unexplored niche.

Short recap of the main implementations:

• OCaml: Fernandez is ahead of the pack again, the only solution in the 7 minutes
• make+C+awk+sh: Perl is dead, but shell scripting is enjoying a renaissance with parallelizable tasks. 8 minutes
• Java-based solutions: in the 13-17 minutes range are the various JVM solutions, from Java, Groovy and Scala to Fan, an interesting Ruby-like language for the JVM. Reminds me of .NET’s Boo.
• Python, Ruby: in the 20+ minutes range. Python multiprocessing is not that efficient yet; I believe an improved Stackless Python solution might be forthcoming

I dabbled with a Common Lisp solution; it works and appears to be competitive, when tried on a partial log file. Exploring the available options for parallelism, however, revealed the disconcerting fact: no freely-available Common Lisp compilers have good multi-threading, or even multi-processing (without shared memory) on Solaris! Even worse, the SBCL incompatibility with GCC 4.3 means that even the Linux version on my Fedora machine is several months old, and does not have the threading library.

So it’s back to Java. Perusing the blogs of the Java and Scala programmers, it appears that the common complain is .. regular expressions. So the hunt was on for a good regular expression library. Joni, a port of the Ruby Oniguruma regex library to the JVM used by the JRuby project, appears ideal: low-level and supposedly very fast. Until one hits the total lack of documentation. So that’s off the table. Ended up using dk.brics.automaton, which appears to perform well enough, even when parsing Unicode strings.

The nice thing about using Java is that, if you hit a performance brick wall, chances are that many other people have been there before you. The problem I have, the need to have random access within a file (so different threads can start at different offsets (Java’s RandomAccessFile is good for this) combined with the need for buffered I/O (BufferedReader is good, but there is no RandomAccessReader !) is solved by the nice folks at Biojava.com. Great!

On my system (2 GHz Core 2, 2 GB RAM, 5400 rpm HDD, Fedora 9 x64), Ruby takes about 2.2 seconds, while my Java implementation running on OpenJDK 1.6 (64-bit) takes about 1.6-1.7 seconds with 1 thread and 1.4-1.5 seconds with 2 threads. Close to the 1.2+ seconds time that Alex reported for Python, but hey, we’re paying the Java start-up cost here.

Will update when I get an account on the test server. 40GB dataset, here I come! In the meantime, time to look for opportunities to use a JVM-based language better suited for the task. The Java code is a tad bit verbose.

Every OS upgrade entails several decision: do you do a clean install, or upgrade your current installation? Do you start with a clean home directory, or re-use your previous one? Any combination works fine, though my personal preference is to do a clean install and use a clean home directory, having archived the older directory. I think of it as house-cleaning — and it’s nice to experience the desktop as it ships out of the box, before customization (naturally, I then restore my music database, my address books, browser and e-mail client profiles, etc. This is not Memento!)

So, now that Fedora 9 has been released, what needs to be added to / changed from the base setup? As it turns out, not that many:

Compositing
Some people swear by Compiz; I personally find Metacity much more usable (Compiz does not support cycling through all windows of a given application — Ctrl+F6 in Metacity; Cmd+~ in OS X). Metacity now has a compositing manager that’s turned off by default; turning it on involves either using gconftool-2 (only for advanced users) or gconf-editor, and setting the /apps/metacity/general/compositing_manager key to true.

The support in the stable version is a bit flaky still; the metacity package in Rawhide is much better behaved and appears quite stable. Upgrade by issuing yum --enablerepo=rawhide update metacity. As of the moment it does not pull in any other Rawhide package so you can rest easy.

Try pressing the volume up/down/mute keys on your keyboard (if you don’t have a multimedia keyboard, change the bindings in System->Preferences->Personal->Keyboard Shortcuts) and be amazed at the translucency coolness (no, this is not bling). The brightness pop-up windows have not been changed yet, alas.

Firefox
Ever cursed Firefox’s font rendering in silence? Type about:config in the address bar, and add the following boolean keys:

font.FreeType2.autohinted = true
font.FreeType2.enable = true

Keyboard
For the English-speakers among us specifically, and those who use the US keyboard layout in general (it’s the standard layout in Indonesia, for instance), the occasional times when one has to type an accented character is rather annoying.

There are various work-arounds — launch the character map (under Accessories), add the Character Palette applet to the panel (so that it consumes RAM even when you don’t use it!)…*or* you can just fix your keyboard layout. The die-hard command-line junkie would be able to tell you what option to pass to setxkbmap to achieve this. The rest of us can just use System->Preferences->Hardware->Keyboard. In the “Layouts” tab, select “Layout Options”. The option you want is “Compose key position”; I use Right Alt, but Caps Lock haters will rejoice to know that, yes, you can use that dreaded key as your compose key as well. To type an accented character, now the only thing you need to do is hit the Compose+accent followed by the letter you want to accent (using shift as necessary, e.g. for ^).

While you’re here, you might want to change the Alt/Win key behavior, and map either Meta, Super or Hyper to one of your Win-keys. The GNOME default is inexplicably for the Win-key to be a normal key and not a modifier (so it cannot be combined with other keys).

Coming up: Applications
Et voilà! You should have a nice-looking, and more importantly, functional desktop right now. In the next instalment, I’ll comment on the applications I use. Until then, à bientôt!

Posting this from the new N810 using its built-in keyboard. Typing those tags was painful, and as a result, more languages will be added to the table tomorrow — from the workstation.

• It looks gorgeous
• It runs Linux, and showcases what can be done with more vertical integration
• Nokia has been improving their interaction with the developer community
• Video camera and Skype (no Skype video support yet, though)
• Rhapsody subscription service
• New: Now with GPS, spacious internal storage, and sliding keyboard built-in!
• New: More video codecs, Flash 9, Mozilla-based browser

So commercial software providers (Skype, Real Networks) will provide Linux ports if they judge that the userbase is big enough. Which is good news.

The same thing applies to Nokia itself, naturally, and sadly in this case, they do not think there is demand for Ogg Vorbis playback.

So if, like me, you find the product attractive, but have a personal collection of Ogg Vorbis files (or FLAC, which transcodes seamlessly to Vorbis), then this is what you can do:

• E-mail Nokia about it
• Inform outlets that stock the tablet (e.g. Best Buy, CompUSA)
• Sign this pledge and pass it around

All the software for the new device (minus GPS — though perhaps it’s the same software that comes with the GPS kit for N800? Oh, and the ambient light sensor) will run on the N800, so holding back won’t be that painful.

The JoCaml version does the file-partitioning trick used in the C++ implementation, with each finder workers being run inside a JoCaml channel; the channels share a single lock so they can update the hash table serially.

Interestingly, current implementation does not get a speed-up from the input file being cached (Ilmari’s wf.ml does). Will have to peruse his to see what’s slowing things down.

Lesson: not all techniques for processing a file line-wise are equally good!

• Boost regular expression is not compiled?
• C++ strings have higher overhead than null-terminated c_str

First point can be ruled out: Boost compiles regular expressions when you assign them. Second point — well, reading in the file using std::getline turns out to consume the bulk of time.

I’ve reorganized the code a bit, using a multimap rather than a vector to rank the URLs by count, with no effect on speed. With two and four threads on a dual-core Intel notebook, the performance is at least on par with Ruby.

Alastair Rankine has a C++ implementation that is slightly faster, but uses Boost memory-mapped IO that I avoided for the same reason he put as caveat: that it will not scale to files that are too large. Which Tim’s log file might well be. Again, that is not significantly faster than the Ruby code.

Moral of the question: Perl and Ruby can be faster than C++! The C implementations out there are blindingly fast, but the way they do regular expression handling are really painful.

Will turn my (limited) spare time to doing a clean JoCaml implementation — it might not be faster but it definitely will look cleaner!

Requirements:

• Functional
• Good support for threading
• If possible, support for distributed computing

As it turns out, JoCaml fits the bill perfectly. It’s an extension of Ocaml, so it combines a rich library with a familiar syntax (not to me, but having used both Scheme and Haskell, how different can it be) — and a very nice process calculus!

Example: this is a concurrent stack that blocks if there is no input available

let new_stack () =
def state (s) & push (v) = state (v::s) & reply to push
or state (x::s) & pop () = state s & reply x to pop in
spawn state([]);
pop, push
;;

This defines a private state channel, and then export the pop and push synchronous channels (that to the user behave just like ordinary functions)

and this is how you use it:

let pop, push = new_stack ();;
spawn echo(pop());;
push(1);;

Note that the echo channel will block, since pop can’t return a value until the stack contains something! This value is then pushed into the stack and ’1′ printed.

More of this at the JoCaml site. And, as it turns out, there already is a JoCaml implementation of the Wide Finder, by Ilmari Heikkinen. Will have to grok the finer details from him.

It occurred to me a while back that this is a perfect job for a C multi-process program, taking advantage of Linux’s cheap copy-on-write fork, if not for the need to merge the result. So it would probably be easier to write it using the pthreads library instead. One would want to reduce inter-thread communication as much as possible, though.

I did not have time to touch the code until today, but now it’s done. Two key insights:

• The input file can be partitioned cleanly into multiple chunks of roughly identical sizes without communication, as long as each thread follows the same protocol. i.e. a common chunk size is used, and all but the first thread needs to check if the character before their starting offset is the end-of-line. If not, they need to skip until the end of line, yielding that line to the previous thread.
• When merging the hash maps, the first thread to acquire the lock does not need to insert its items one by one, but can just set the main hash map to point to its own

On a data set size of 200MB, with a single thread, performs slightly slower as Ruby by wall clock:

Ruby timing

2.99user 1.88system 0:05.26elapsed 92%CPU (0avgtext+0avgdata 0maxresident)k
152inputs+0outputs (3major+853minor)pagefaults 0swaps
3.04user 1.84system 0:05.33elapsed 91%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+856minor)pagefaults 0swaps
2.84user 2.01system 0:05.19elapsed 93%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+856minor)pagefaults 0swaps


C++ timing, one thread

2.70user 5.21system 0:07.95elapsed 99%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+655minor)pagefaults 0swaps
2.78user 5.14system 0:07.96elapsed 99%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+655minor)pagefaults 0swaps
2.72user 5.12system 0:07.90elapsed 99%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+654minor)pagefaults 0swaps


With two threads, though, it runs faster, almost at Ruby speed.

C++ timing, two threads

3.33user 5.40system 0:05.51elapsed 158%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+675minor)pagefaults 0swaps
3.40user 5.33system 0:05.86elapsed 148%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+672minor)pagefaults 0swaps
3.41user 5.36system 0:05.60elapsed 156%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+674minor)pagefaults 0swaps


So, as Tim observed, the problem is not entirely IO-bound. More testing is needed, but will probably need to be done on a machine with faster IO (and more CPUs). Like Tim’s new Niagara T2 testbed.

Will edit this post later — need to run now, and wanted to get this out as soon as possible. Code is available here… now I just need to find Tim’s email, since his blog is down. Argh!

Update: switched to the new std::tr1::unsorted_map. Performance seems identical with a single thread, but slightly higher (within margin of error) on two threads — which makes sense: a multithreaded run exercises the map more, because each thread’s map has to be merged into the final map.

An interesting development is that they are also working with Novell to bring the technology to Linux (to be known as Moonlight). Moonlight supposedly will be GPL’ed, with the exception of the binary codecs needed for playback of proprietary Windows Media formats.

So Linux is still a second-hand citizen (version 1.0 is out, and development on the Linux port is just announced?), but this might be interesting for those of us (read: most Linux desktop users) that have been waiting for a decent Flash playback (forget authoring) solutions. Still no x86_64 support after all these years, forcing people to stick to 32-bit browsers (or use nspluginwrapper). And unless Adobe has changed things since acquiring Macromedia, anyone who has merely used the Flash plugin is tainted and can’t work on Flash reimplementations!

(From this interview [lwn.net] with principal developers of Swfdec and Gnash)

Rob: The Adobe EULA for Flash forbids anyone who has installed their Flash tools or plugin from working on Flash technologies. This has had a chilling effect on the development of free Flash players, since a developer must either choose to decide that Adobe won’t sue them over this, or to do what Gnash does, which is a slow and inefficient, clean room, reverse engineering project.

As I argued, Novell would most likely require copyright assignments on contributions to Moonlight, and thus the use of GPL is not going to hinder Microsoft benefiting from community contribution (interesting, from a once anti-GPL company). The framework is conveniently written in C# as well, so it’s not going to directly benefit the Gnash and Swfdec projects (Flash), but depending on how clearly Novell marks up their code for patent-encumbrance, it could still either benefit or act as a time bomb for Flash projects..

Naturally, I’d still advise anyone to refrain from using Silverlight/Moonlight unless it’s being used as a container for standard (H.263/4) codecs. WMV is an abomination dating back to the AVI days — note how most HD- and BD-DVDs are encoded in H.264, not VC-1 which is basically WMV.

The archives, thankfully, are still (largely) available. But how, pray, would you download hundreds of songs scattered over 26 pages? (one per each letter in the alphabet)

If you use Firefox (and if not, you should. Not because it’s the best browser, but becaue of the add-ons), then it’s simple — look up FlashGot or DownThemAll!. The former is more configurable, I believe (you get to use your own download manager), but the latter is probably simpler to use.

Caveat lector: the (rough guess) 70% or so of the songs that still have functional links total up to more than 2GB’s worth. Will probably post a Torrent link on Mininova once I tidy them up a bit.

Conveniently left out is the one-sidedness of this smear campaign. Microsoft could take all their time to look at open-sourced code, but on the other hand, Microsoft’s code is sacrosanct. One wonders if Microsoft’s “Shared Source” licenses and EULAs allows the licensee to sue Microsoft for patent violation? One would suspect otherwise.

The patent system is broken. And it’s a matter of incentives. As I wrote in response, the US Patent and Trademark Office grants overly-broad patents without checking carefully for prior art. Thus Microsoft’s claim is probably true in a very twisted way: nobody can write anything meaningful without violating some patent that one of the big tech companies (IBM, Sun, Microsoft, etc.) holds. Any company with a legal team worth their salt would play the system and file for as many patents as they can, no matter how dubious. Some even boast about it (Steve Jobs, about Apple’s iPhone). The culprit is thus the patent system, not the owners of the dubious patents (who are ethically responsible to their shareholders only. Corporate law is also very flawed, but that’s the subject of another post), and certainly not the authors of the infringing software.

Regardless of whether software patents is a good idea or not (I personally think it’s not), the USPTO and Congress have a serious conflict-of-interest problem here. Granting more patents earn them more money, and examining patent applications carefully cost them money. Plus, they don’t lose anything for patents that are overturned. The Justice department should charge USPTO for the time wasted by patent challenges, if the patent ends up overturned because of USPTO’s fault. That way they’d have an incentive to be more careful.

According to the New York Post, Microsoft is asking Yahoo to re-enter formal talks in order to finalize a merger of the two companies. This move was attempted in the past just shortly after Google emerged, but in the end went nowhere. After the huge loss last month to Google over the DoubleClick purchase, Microsoft is intensifying their plans to acquire Yahoo.

This is disturbing. Yahoo has traditionally been quite open-source friendly (Yahoo! Toolbar is available for Firefox, their AJAX library is open-sourced, and they even produced a Linux client of Yahoo Messenger, long time ago — it still works, though not updated), or at any rate, more platform-agnostic.

Of the three companies, Yahoo Messenger supports OS X on par with Windows, with Linux being an afterthought, while Microsoft’s Messenger client is a joke on a Mac, and GTalk .. is currently Windows-only (Google employs a key Pidgin developer, so now that the dispute with AOL over their old name, Gaim, has been settled, hopefully audio chat support will be coming to the free Unices (and OS X) soon.

What will a Microsoft buy-out entail? First of all, presumably the downgrading of Mac support (look at Virtual PC, or IE for Mac). Microsoft would cancel a product that’s making them money (VPC), and forbid the use of home editions of Vista under virtualization, to sabotage Mac interoperability, while at the same time offer (read: promise) a free version of their Flash competitor Silverlight for the platform (not authoring tools, naturally. Everyone knows digital artists <prefers> working in Windows). Imagine the fate of Yahoo Messenger, what with Apple aggressively pushing iChat into the corporate market. Microsoft will probably calculate that they are better served undermining OS X’s viability as a business platform, at the expense of a miniscule loss of marketshare.

So we in the FLOSS world are in this unconfortable situation of hitching our wagons to Google as the lesser of the two evils (despite privacy concerns). I’m personally using GTalk (through Pidgin) at least until Jabber becomes the established IM protocol standard, but if anyone has a suggestion for a good non-big-3 search engine, I’m definitely considering putting my eggs in more baskets.

They have a point there. I have a problem with their assertion that the $3 Windows edition is another admission of failure. True, the stronger anti-piracy controls on Vista means that Microsoft’s closing one eye on piracy no longer reduces Linux’s price advantage as much. But there’s the point of government tie-in: to be eligible for the cheap Windows licenses, governments have to step in and subsidize the hardware!

So Microsoft might have a weakened hold on its current OEMs, but it is hoping to gain entire countries as new clients. In which case the price drop cannot be compared against the full price of Windows, but what Dell, HP etc. gets charged (less than $50 a piece, I’m sure). An order of magnitude less. Which is a side issue — Microsoft is not currently gaining anything from those pirated copies anyway (apart from mindshare). The main worry is that another generation will be locked into Microsoft products, and want to bet there’s a clause in the fine print promoting the so-called Open XML over Open Doc?

In the case of the woman mentioned in the story, Linux is clearly not at fault — the keyboard was just broken, and the tech support person was too clueless (or the support regulations structured in such a way) to notice. However, there could be valid cases where the combination of crappy hardware and running Linux *would* ruin your hardware. I know, because it just happened to me.

The recipe is as such:

1. Broken ACPI implementation (hello, HP! To be fair, the most recent dv2000, dv6000 and equivalent Compaq lines are probably fixed; by the lack of complaints), so that Linux’s ACPI subsystem cannot control the fan and CPU voltage properly
2. A CPU that runs hot (first-generation “mobile” Turion 64)
3. 2 years of frequent use, tolerating occasional crashes due to overheating
4. OMG!!! LINUKS KILLZ ME PC!!

I guess the writing is on the wall when I had to remove my second memory module to keep the laptop running at all. The poor fans must be getting to their end of life. And today, after a freeze while the laptop was on battery power (previously almost unheard-of, it’s normally stable if running at 800 MHz, which it does when unplugged), it just never woke up again. The power indicators would be on, the optical drive would operate properly, and the AC indicator too, but apart from that, zilch. Nothing displayed to the screen, no error beep, nothing.

One begins to wonder whether the buggy Microsoft ACPI AML compiler was actually intentionally buggy. After all, if you load Windows onto your machine ”as it is intended”, you’ll be using a Microsoft-written AML interpreter, and everything would be just fine. If you use the Intel reference AML interpreter, well, … Very disturbing indeed. The same way Microsoft abused its IE market share by intentionally not making it standards-compliant, making life a nightmare for web designers and thus the easy way out is to only support IE ..

RIP, HP L2000 “Lance Armstrong special edition”. It’s ironic that a record winner of an endurance cycling race, who also happens to be a cancer survivor, would lend his name to a buggy laptop with poor noise insulation and heat issues that didn’t even last two years. I was planning to replace it with an HP dv2000t once the Santa Rosa models are out (late April — early May), but after this, I wonder. I’ve never had a laptop dying on me before; my previous write-offs are either due to inexpert disassembly (Vaio C1) or drop damage (Vaio C140). Perhaps I’ll get a Macbook after all, though my Apple purchasing track record is notoriously bad (I’d buy just before a new model is launched – I barely managed to cancel a Dell laptop purchase when the Powerbook G4 12″ was released, and I jumped the gun and bought the Sony C140 — now with a broken LCD — and before it even arrived, the Core 2 Macbooks came out).

Could I wait this time for the don’t-know-when Santa Rosa? Come on, Apple, update the line before Leopard comes out!

Update – 2007/04/16 – OSnews has an article on a Bill Gates memo showing Microsoft’s intention to cripple ACPI. You’re not paranoid if people really are out to get you! And shame on that abusive monopolist — this is straight out of The Corporate, folks. It figures that after screwing over DR-DOS in a very similar way, they’d try the same trick again.

Case in point: France 24, the French answer to CNN and BBC. While most Anglo-American site would at least offer you a choice between Real and Windows Media (I don’t see QuickTime used much), most French sites, including the aforementioned, serves only WMV. This, at a time where the BBC is even sponsoring the development of an open-source video codec, Dirac [wikipedia.org].

The site’s design also feels rather sluggish, and the aspect ratio is wrong unless you go full-screen. Alors…

If you think of an application as an object, a thing, then you want to be able to treat it like any other object (say, a document): you can copy it by drag-and-drop, move it to another location, run it by double-clicking, and when you don’t need it anymore, drag it to the trash. This works if you’re on the aforementioned system, but not for everyone else:

- Windows applications write to system directories, and the default Registry setting is normally set up only once, during installation
- Unix applications are installed all over the file system

Even with package management (e.g. RPM+yum, DEB+apt, Smart, emerge) the situation is far from ideal:
- application still cannot be dragged-and-dropped
- can’t run an application without installing (for trying it out, for example)
- if you don’t have the original package you can’t recover it from the installed files (I don’t believe the package database holds the pre-/post-install scripts, only the pre- and post-uninstall ones)

And even OS X’s implementation is not ideal: there’s no way to declare dependencies, so a lot of applications just bundle all the libraries they can’t expect the user to have installed already. ROX is ahead here with their integration of the 0install system: you first download an almost-empty application bundle, and the first time you run it it triggers 0install, which can pull in the main body of the application, and install the dependencies too.

For some reason ROX’s installation system has not caught on yet with the major desktops (neither Nautilus nor Konqueror, the two main desktops’ file managers, handle launching ROX AppDirs, the most basic prerequisite for interoperability; for some reason the talks with the XFCE project did not go anywhere either). But now at least another project is picking up the idea, and were it to be successful, would see application bundles available to millions of users. Who? Why, the One Laptop per Child project.

(J5 blogged about it several weeks ago, but I didn’t get to that post until now. Mea culpa.)

I don’t know how it’s implemented there, but as I commented, it could possibly go like this:

App.xo
|– deps.xml
|– info.xml
|– binary
|– icons/
|– libs/
|– share/

with the application directory monitored for changes; drop a new application in and Nautilus will scan deps.xml for any prerequisites that’s not yet installed and offer to install it.

That way we can have OS X-style ease-of-use without their application bloat (since a lot of the .app bundles there just ship all the libraries they can’t assume the use.

2006-11-25 19:09 EST Update: Here is the specification for the bundles. No dependency tracking yet, unfortunately.

Why a company with shady political connections and a disreputable security track record continues to be contracted to supply voting machines, I don’t know.

Technology is neutral. But belief in the infallibility of it is irresponsible, misguided, and .. perhaps actively encouraged by people with vested interest. Big Brother is watching YOU!

No more squinting when looking at small text, or finding your desktop space taken up by extra-large application menus!

“Be not afraid of greatness: some are born great, some achieve greatness and some have greatness thrust upon them”.
- (Shakespeare, W., Twelfth Night, Quote Act II, Scene V).

“Some men are born mediocre, some men achieve mediocrity, and some men have mediocrity thrust upon them. With Major Major it had been all three”.
- (Heller, J., Catch-22, Quote Page 94 (paperback)).

“Some languages are born lazy, some languages achieve laziness, and some languages have laziness thrust upon them”.
- (Salim, M., unpublished)

So you can probably guess what this is about. A stream library for Lua, that allows you to do things like running the Sieve of Eratosthenes without specifying an upper bound, lazily filtering the set of all positive integers as you go along.

Witness the use of the library:

require("stream")

function ints(n)
   return {
      head = function () return n end,
      tail = function () return ints(n+1) end }
end

function primes (sints)
   local curprime = false
   local tailthunk = false
   local fhead =
      function ()
	 if curprime then return curprime
	 else
	    curprime = sints.head()
	    tailthunk = sints.tail
	    return curprime
	 end
      end
   local ftail =
      function ()
	 if (not tailthunk) then
	    fhead()
	 end
	 return primes(
		 stream.filter(function(n) return not (n%curprime == 0) end,
			       tailthunk() ))
      end
   return {head=fhead, tail=ftail}
end

-- print the first 100 primes
stream.print(100, primes(ints(2)))

A stream is a table containing a head and a tail; both are thunks, so you call them with no arguments to obtain the value. Note the caching that avoids recomputation, similar caching is done in the functions provided by the stream library (like stream.filter)

Code available on hircus.org.

And some benchmarks:

	Lua	Haskell

The Haskell code was run using GHC’s ‘runhaskell’, which does not precompile the code, to be fair to Lua which is interpreted. Not bad when put up against a language that is lazy by design (and thus optimized for it)! Timing taken using the ‘time’ utility on an Intel Core 2 Duo 1.66 GHz laptop which was mostly idle.

Alas, the Nokia announcement is still vaporware (I’d happily beta test the new firmware, since the latest update actually broke timezone support, and the full-screen virtual keyboard sounds like a nice improvement over the half-sized version one has to carefully peck with a stylus right now). And to make matter worse my tablet’s wireless actually died yesterday! Thankfully it’s still under warranty, and Nokia didn’t actually announce a hardware upgrade, as was speculated.

The Apple announcement, on the other hand, is welcome. There are concerns about the integrated video (can’t be worse than the also-integrated, 700-fps-in-glxgears ATi in my HP right now), and the price premium charged for the black case, but so far it’s looking good. I’ll let the early adopters work out all the issues with this and get the next model with OS X 10.5 and 64-bit Merom CPUs..

There are some HOWTOs and forum discussions out there, but most of the ones that refer to Ubuntu refer to earlier versions, and a lot of the steps mentioned are probably not needed anymore.

As it turns out, these are all you need:

apt-get install multisync libmultisync-plugin-irmc-bluetooth libmultisync-plugin-evolution

Modify /etc/bluetooth/hid.conf . This is the crucial step! In /etc/bluetooth/hcid.conf, change the local device class from

class 0x3e0100;

to

class 0x100100;

Now set up multisync and follow the instructions – should be quite straightforward. The only thing to note is that by default, Multisync does not preselect any calendar, address books etc., since you can have more than one. Make sure to pick the ones you want to sync.

Given two binary trees, and a depth-first-search traversal ordering of the leaf nodes, produce two new trees with the same structures as the originals, but with the leaf nodes swapped. If there are more leaf nodes in one tree than the other, the remaining leaf nodes are copied unchanged.

So (swap-leaves ‘(1 . (2 . 3)) ‘(a . ((b . c) . d)))
=> ‘((a . (b . c)) ‘(1 . ((2 . 3) . d))

Waiving the requirement that the program be completely functional, my original thought was I could probably just modify my tree walker to do this. This treewalk procedure takes a tree and produces a stream of leaf node values. Changing it so that it returns a pair of the value, and a setter procedure that can modify the value in place, is trivial: the recursive auxiliary procedure for treewalk is given the parent node, so it can use set-car! or set-cdr! (depending on which side of the parent node this leaf node is) to update the parent:

(define treewalk
  (lambda (t)
    (if (not (pair? t))
        (stream-cons
          `(t . (lambda (v)
                  (error "Cannot modify singleton")))
          empty-stream)
        (stream-append (treewalk^ (car t) #t t)
                       (treewalk^ (cdr t) #f t)))))

(define treewalk^
  (lambda (t left? parent)
    (if (not (pair? t))
        (stream-cons `(,t . ,(if left?
                               (lambda (v)
                                 (set-car! parent v))
                               (lambda (v)
                                 (set-cdr! parent v))))
                     empty-stream)
        (stream-append (treewalk^ (car t) #t t)
                       (treewalk^ (cdr t) #f t)))))

The swapping is then just a matter of calling the setter for node1 with the value of node2, and vice-versa; this generalizes to swapping between multiple trees as well.

(define treewalk
  (lambda (t)
    (if (not (pair? t))
        (stream-cons `(t . (lambda (v)
                             (error "Cannot modify singleton")))
                     empty-stream)
        (stream-append (treewalk^ (car t) #t t)
                       (treewalk^ (cdr t) #f t)))))

(define treewalk^
  (lambda (t left? parent)
    (if (not (pair? t))
        (stream-cons `(,t . ,(if left?
                               (lambda (v)
                                 (set-car! parent v))
                               (lambda (v)
                                 (set-cdr! parent v))))
                     empty-stream)
        (stream-append (treewalk^ (car t) #t t)
                       (treewalk^ (cdr t) #f t)))))

But this modifies existing trees in place, a big no-no in functional programming land (though imperative programmers live with this every day). Is there a way to do this in, if not a fully functional way, then in a way that the side-effects are confined? (I mentioned to Roshan that this is a dual of Haskell’s monads: it behaves functionally even though it is imperative; monads gave the illusion of imperative behaviour even though they are fully functional. He pointed out that internally the state monad is optimized into imperative calls as well, so the comparison is even more interesting)

Instead of modifying existing trees in place, we want to copy the trees. But copying, naively done, would involve traversing the trees again, and that would be inefficient. The copying should be done Lazily as needed, and with streams, that means doing it while you’re walking the tree and collecting the nodes!

In short, at every internal node,
- create a new node, which has as its children the children of the original node;
- update the clone of the parent, instead of the parent, so that its car or cdr points to this new node
- the setter procedures for each leaf nodes modify these cloned nodes instead of the originals

...
(let ([copy (cons (car t) (cdr t))])
  (if left?
    (set-car! copy-parent copy)
    (set-cdr! copy-parent copy))
  (stream-append (treewalk-copy^ (car t) #t copy)
                 (treewalk-copy^ (cdr t) #f copy))
...

The main treewalk procedure now has to return the cloned tree as well, since at each subsequent stage of the walk only the direct parent is known, not the entire tree.

...
(let ([copy (cons (car t) (cdr t))])
       (if left?
           (set-car! copy-parent copy)
           (set-cdr! copy-parent copy))
  (stream-append (treewalk-copy^ (car t) #t copy)
                 (treewalk-copy^ (cdr t) #f copy))
...

Code, including stream implementation, available here

That time was spent reading Lambda the Ultimate, which was how I noticed the LiteratePrograms wiki. Based on Wikipedia’s MediaWiki, with added feature from the noweb literate programming system, it is to collaborative programming what other wikis are to collaborative writing. You can declare code blocks, and if the language is supported get syntax highlighting for free; the code is automatically packaged into a zip archive everytime someone hits the download link.

Check it out. But if you’re teaching a programming course, don’t mention it to your students

This is the original implementation:

#!/usr/bin/env python2.4
# This program shows off a python decorator(
# which implements tail call optimization. It
# does this by throwing an exception if it is
# it's own grandparent, and catching such
# exceptions to recall the stack.

import sys

class TailRecurseException:
  def __init__(self, args, kwargs):
    self.args = args
    self.kwargs = kwargs

def tail_call_optimized(g):
  """
  This function decorates a function with tail call
  optimization. It does this by throwing an exception
  if it is it's own grandparent, and catching such
  exceptions to fake the tail call optimization.

  This function fails if the decorated
  function recurses in a non-tail context.
  """
  def func(*args, **kwargs):
    f = sys._getframe()
    if f.f_back and f.f_back.f_back         and f.f_back.f_back.f_code == f.f_code:
      raise TailRecurseException(args, kwargs)
    else:
      while 1:
        try:
          return g(*args, **kwargs)
        except TailRecurseException, e:
          args = e.args
          kwargs = e.kwargs
  func.__doc__ = g.__doc__
  return func

@tail_call_optimized
def factorial(n, acc=1):
  "calculate a factorial"
  if n == 0:
    return acc
  return factorial(n-1, n*acc)

print factorial(10000)
# prints a big, big number,
# but doesn't hit the recursion limit.

@tail_call_optimized
def fib(i, current = 0, next = 1):
  if i == 0:
    return current
  else:
    return fib(i - 1, next, current + next)

print fib(10000)
# also prints a big number,
# but doesn't hit the recursion limit.

There is a subtle bug: after handling the exception, func then calls its own saved version of g, not the g of the func that throws the exception. Modifying func and the TailRecurseException class so that the function to be called can be passed in the exception solves this nicely:

class TailRecurseException:
  def __init__(self, g, args, kwargs):
    self.g = g
    self.args = args
    self.kwargs = kwargs

  ...
  def func(*args, **kwargs):
    f = sys._getframe()
    if f.f_back and f.f_back.f_back         and f.f_back.f_back.f_code == f.f_code:
      raise TailRecurseException(g, args, kwargs)
    else:
      newg = g
      while 1:
        try:
          return newg(*args, **kwargs)
        except TailRecurseException, e:
          newg = e.g
          args = e.args
          kwargs = e.kwargs
  ...

Throwing an exception just to handle the required stack manipulation because the interpreter won’t handle it is grossly inefficient, granted, and requiring the programmer to declare functions as optimizable is not be ideal, but still a really neat trick.

As a side note, if a function can be called in tail and non-tail position, a non-decorated version should be made available, otherwise bad things can happen:

@tail_call_optimized
def add1(n):
  return (n+1)

@tail_call_optimized
def add2(n):
  return add1(add1(n))

This returns 41.

Modified version, and test cases, available here [IUCS] (backup copy here).

Update: added isTailCall by jorend. Now it is safe to optimize every function (though for efficiency reason you’d only optimize the ones that make tail calls).

Which makes languages like Scala exciting. It’s fully functional, and also fully object-oriented.Integers are objects. Functions? Also objects. The List ADT familiar to functional programmer is now an Object, with methods such as fold and filter defined on it (which makes the syntax a bit confusing, but not if you come from Ruby. Or your first language is japanese. But I digress)

Best of both worlds? Well, not quite. The new Scala compiler looks promising, but it still trips up on this simple mutual-recursion example:

object RecEvenOdd {
  def rec_oddp(n: Int): Boolean =
    { if (n == 0) false else rec_evenp(n-1) }
  def rec_evenp(n: Int): Boolean =
    { if (n == 0 ) true else rec_oddp(n-1) }
  def rec_oddevenp(flag: Boolean, n: Int): Boolean = {
    if (n == 0) flag else rec_oddevenp(!flag, n-1)
  }

  def main(args: Array[String]): unit = {
    val n: Int = Integer.parseInt(args(0));
    System.out.println("recursive: " + n + ": " +       rec_oddevenp(true, n));
    System.out.println("mutual rec: " + n + ": " +       rec_evenp(n));
  }
}

The former gets detected as a tail call and optimized. The second? No such luck.. yet.

Of further note is the implementation of streams: Scala implements the “odd” style of lazy programming (Wadler et al., How to add laziness to a strict language without even being odd); the provided implementation is very similar to the reference implementation of streams for Scheme in SICP. The following program will thus unnecessarily compute the square root of -1:

object WadlerTest {
  def countdown(n: Int): Stream[Int] =
    Stream.cons(n, countdown(n-1));
  def sqrt(n: Double): Double = {
    System.out.println("sqrt: called with " + n);
    Math.sqrt(n);
  }
  def main(args: Array[String]): unit = {
    countdown(4) map { n => sqrt(n) } take(5) print
  }
}

The syntax can also be bizarre at times, a motley of Java-isms and proper FP style. For example,

{ n => sqrt(n) }

is η-reducible to sqrt, but Scala would not compile it with this definition of sqrt. Rewriting sqrt thus:

  def sqrt = { n: Double =>
  ...

now sqrt can be passed to map – but the automatic cast from Int to Double is no longer performed. Eek.

Moral of the story? I’m learning Haskell.

First impression? It’s smaller than expected – about the same dimension as a PDA, rotated 90° and stretched slightly along the width. And the screen is crystal-sharp!

On the downside, application load times are on the sluggish side, and the OOM killer kicks in a bit too often, wspecially when browsing Flash-heavy sites.

Some third-party applications are on the unstable side – the WebCore-based mini browser crashes whenever one types into a password field; the ScummVM game Beneath a Steel Sky does not display an icon in the task bar, making it impossible to kill without first installing X Terminal. Hail killall!

Need to set up my Linux box for Maemo development over the break, so I could start porting Quarry. Chess is great, but sometimes one feels like Go or Reversi…

This post typed on a 770

Provided Eric gets the H211 Introduction to Programming (Honors) class, which is in Python, and I get the C211 Introduction to Programming (Scheme), our discussion should stand us in good steed, though funnily today I played the Python guy and he played the Scheme one.

I’m going to show the examples, both in Scheme and Python; the first one in each section would appear to show that the language in question features dynamic scoping, which is incorrect as both actually do lexical scoping.

Scheme:
Bad:

(let ((pi 3.1415))
 (define area
  (lambda (r)
   (* pi r r)))
 (display (area 10))
 (newline)
 (set! pi 3)
 (display (area 10)))

Good:

(let ((pi 3.1415))
 (define area
  (lambda (r)
   (* pi r r)))
 (display (area 10))
 (newline)
 (let ((pi 3))
  (display (area 10))))

Python:
Bad:

pi = 3.1415
def area(r):
 return pi*r*r
area(10)
pi = 3
area(10)

Good:

pi = 3.1415
pi_holder = 10
def create_area():
 pi_holder = pi  # local pi_holder, different from pi_holder outside
 def area(r):
  return pi_holder*r*r
 return area
area = create_area()
area(10)    # 314.15
pi_holder    # Still 10
pi_holder = 3
area(10)    # Still 314.15

The above works, but is a bit problematic. I introduced pi_holder=10 to show that, (1), pi_holder inside of create_area() is a local variable; (2), that this local pi_holder is the one that is in area‘s scope, and thus changing the value of pi_holder does not affect it.

Isn’t it easier to just do pi = pi ? Well, that does not work. My initial hunch was that Python reads the LHS of the expression, decided pi has been redeclared as a local variable, and thus since it’s not been initialized it got confused trying to assign it the value of itself. But it’s actually worse; this code does not work either:

x = 42
def local_var_test():
 temp = x
 print temp  # 42
 x = temp
 print x   # 42?

Surprise! Python won’t let you do that either. Take out the last two lines and the code works, though. Basically, if in the block a variable is declared anywhere, it is a local variable everywhere in that block, and trying to refer to a variable declared in the surrounding scope, even before the local declaration, will fail.

But this is where default parameters come in handy. A better way to rewrite the clunky code above is as follows:

pi = 3.1415
def create_area(pi = pi):
 def area(r):
  return pi*r*r
 return area
area = create_area()
area(10)    #314.15
pi = 3
area(10)    #314.15

So Python has static scoping after all. The thing to bear in mind is that Scheme functions are named closures, while Python functions inherit the surrounding scope, so to freeze the variables you depend on you have to wrap your function definition inside another function that copies in the values you need into its local variables.

References:

• Python programming language (Wikipedia): Closures
• Scope (Wikipedia)

And the funny thing is, I started the day trying to find good dynamic languages that run on the Java platform (platform envy, I guess, since .NET more prominently touts its language neutrality). Sun’s finally catching up, though – Tim Bray wrote a few months back about the Coyote project to support dynamic languages in Sun’s open source IDE, NetBeans, and pointed to an interesting Sun-developed scripting language, Pnuts. Which reminded me of Groovy and Boo.

Googling for groovy boo .net – Groovy being a Ruby-like scripting language for Java that received a lot of attention a few months ago, and then taken some flak over its development model, and Boo being the Python-like language for .NET – yields this very interesting Slashdot discussion that led me to such intriguing functional OO languages as Scala and Nice. .NET fans do not get to have all the fun!

Groovy, on the other hand, seems rather disappointing. Oh well. Scala looks more like Haskell, but with dynamic type inference (like Boo).. yay!

Update – 2005/06/06

Realized a few days ago, but haven’t gotten round to posting about it, that I was unfairly comparing Scheme and Python, and that Python methods are closures in themselves. Note:

pi = 3.1415
def area(r):
 return pi*r*r
print area(10)  # 314.15
def test():
 pi = 3
 print area(10) # 314.15
test()

In the earlier example, overriding the value of pi with pi = 3 is the equivalent of doing (set! pi 3) in Scheme, i.e. it will change the value of the variable that both the top-level pi, which is the one that area knows. In a dynamic scope, which uses a stack to figure out which value assignment should apply, pi = 3 would affect the call to area just after it.

Software installs/updates and power management.. the latter is probably an eye-popper to a veteran Linux laptop user, and the former, even Red Hat never managed to get right out of the box (while Debian never managed to get pretty-looking). The thing that prompted me to write this, though, is removable media support. Plug in a USB drive (in my case, the TunePlug from Magnatune, with 10 MP3 rock albums bundled) and up it pops on your desktop, with the correct volume label. Naturally suspicious, I did a quick directory listing of /media and, to my pleasant surprise, the mount-point was dynamically created with the same name.

I was impressed enough to start looking at fixing my desktop’s loud fans (to no avail, but I chanced upon Shuttle’s new BTX line, the XPC i-series. Gorgeous, alas, it’s for Intel CPUs.. and I’m not a fan (pun intended) of the Pentium 4′s design, even with the bolt-on 64-bit support). And looking at the notebook situation in Intel land.. which led me to humbly revise my opinion of Dell. Their Inspiron 700m is.. ooh!

Pity about the battery life though. Let’s see what Fujitsu could do with the new AMD Turion..