Planet PLUG

yonkeltron: Rails + Merb = Rails 3

Wed, 17 Mar 2010 16:39:49 +0000

I’m rather excited about this. Saw this talk right here.

Oh, and then there’s this.

Alex Launi: March record scores!

Mon, 15 Mar 2010 15:42:39 +0000

It’s the 14th already, and I actually blew through my record budget within the first week. I would have made this post earlier but

I was a little embarrassed I blew through my record budget within the first week
I was waiting for a record to come in from the Czech Republic

So here they are, in all of their beautiful waxy glory.

Eternal – Lucifer’s Children: Electric Wizard side project band!! X/500!
Anima Morte/Hooden Menace split picture disc! X/500
Leather – Ancorite. I don’t know how many of these there are, but I got a test press!
Killing Joke s/t
Sisters of Mercy – Floodland
Bauhaus – Press Eject and Give Me the Tape

Not bad for $50 right? The Eternal record is killer, I saw the artwork in the record store and it struck me as Electric Wizard like, so I picked it up, brought it home, and found out a) it’s rare b) it’s pre-electric wizard c) IT’S HEAVY

Alex Launi: gudev-sharp is born!

Mon, 15 Mar 2010 04:57:22 +0000

In some of my spare time I’m working on modernizing Banshee’s hardware support (all the while lamenting the death of HAL). With HAL we had a nice (although monolithic) toolkit-agnostic hardware layer that was used through dbus. HAL was deprecated in favor of using udev, which is the the Kernel’s hardware event system. I’m sure udev is much nicer from an engineering point of view, but it’s certainly not as nice for application developers as a big old dbus-interface where any information about hardware could be retrieved was. Regardless of how I feel about HAL, HAL is still gone, so Banshee still needs updated to udev. Since udev is so new, no one had made C# bindings for libgudev yet, so I decided to take a stab at using GAPI to generate C# bindings. After hours of talking to myself in irc, trying to trick RAOF into doing my work for me, gudev-sharp is born! It just successfully ran the (incredibly simple) test code I wrote just to make sure it wouldn’t segfault. If you’re interested in modern hardware support via C#, help write unit tests!

I know some people are going to say “bzr? launchpad? And you want this adopted?”. It can move whenever it needs to, LP and bzr are my tools of choice, but if upstream Mono wants to move this into svn, I’d be happy to see it go.

mikegrb: Task Management with Hiveminder and Perl

Sat, 13 Mar 2010 23:08:25 +0000

This is how I manage my tasks with Hiveminder on a weekly basis and the Perl script that helps me do it. I don’t really expect the Perl script to be useful to anyone as-is but portions of it may be useful to others, as well as the general work flow, so I’ve decided to share them.

At Linode we have a wiki page where we list 5 or more tasks we wish to accomplish during the week. There is a heading for each employee and below the heading we place our tasks. Throughout the week we can add additional tasks or mark existing ones as done.

I’ve used Hiveminder for some time. When we started the weekly task lists at Linode I found that taking a few minutes to figure out which tasks I wish to complete in the coming week works quite well for me. I started marking these tasks with the ‘week’ tag in Hiveminder. I quickly ended up writing a perl script, week.pl, to help me manage hem.

My Weekly Workflow

First thing Monday morning I run:

$ week.pl report

This prints a report with two sections. The first section lists tasks that currently have the ‘week’ tag with a line through the task ID if it is completed. This gives me a nice summary of what I planned on accomplishing the previous week and how I did. The second section lists all of my tasks currently visible in Hiveminder. I hide tasks that I know I’m not going to work on in the next few weeks so this list is usually no more than 20 or 30 items.

I take this report into the Monday morning meeting with me. During the meeting, I’ll glance over this list and select items for the upcoming week. I also use this page to take notes on during the meeting, writing down any new tasks that come up in the meeting.

After the meeting, I add any new tasks generated in the meeting that I won’t be working on this week to Hiveminder.

$ todo.pl braindump

todo.pl comes from App::Todo, a command line Hiveminder interface. The braindump command launches $EDITOR where I add new tasks, one per line. The braindump syntax allows for specifying tags, setting priorities, and other things as well.

Next, I prepare the task list for the upcoming week:

$ week.pl edit
Carry over the following tasks?
bring about world peace (y or n) [default y] y
write some awesome pre
do some other cool stuff
create practical cold fusion (y or n) [default y] n
return library book

The edit command iterates over each task tagged with ‘week’ if the task is not marked completed. It prompts whether or not I wish to carry the task over to this week (leave the tag). Any tasks marked completed have the tag removed automatically.

$ week.pl add
Created:
#YRVK write an awesome report for Tom [week dev]
#YRVL test some new stuff for deployment [week admin]

The add command works the same as todo.pl’s braindump command except the ‘week’ tag is automatically applied to the newly created tasks. The full braindump syntax is available for specifying other properties of the tasks.

$ week.pl update https://path.to.trac/wiki/Tasks/2010-03-15
Sup dawg, I heard you like tasks so I did ur shit for you

The update command grabs my tasks tagged with ‘week’ and formats them one per line started with ‘ * ‘, a wiki bullet list. It grabs the current wiki page, finds my heading, substitutes the formatted task list under the heading, and submits the change. It also stores the path given in the YAML config file.

$week.pl go

This opens the stored URL for this week’s tasks wiki page in my default browser, allowing me to confirm week.pl did what it’s supposed to.

Later in the week once I’ve done something:

$ week.pl done tom
#YH7T bring about world peace [dev week]
#YRVL test some new stuff for deployment [week admin]
[DONE] #YRVK write an awesome report for Tom [week dev]

This retrieves the task(s) tagged ‘week’ that are not marked completed containing the given string. If there was only one match, the script marks it as done and then outputs the current state of tasks tagged ‘week’.

$ week.pl update
Sup dawg, I heard you like tasks so I did ur shit for you

Same as update before, except when no URL is given, the URL is read from the configuration file. This way I only need to worry about the URL once per week, the first time I update for the week.

Conclusion

So there you have it. If you are also using Hiveminder, maybe aspects of my work flow will make sense for you and pieces of the Perl script may be useful. If you aren’t using Hiveminder, maybe you will be inspired to check it out. I use michael@thegrebs.com on Hiveminder in case you feel the need to assign me a task or wish to gift me another year of Hiveminder Pro ;)

week.pl

yonkeltron: Enthusiasts talk shop casually

Mon, 08 Mar 2010 18:22:15 +0000

I spent some time with a few colleagues yesterday and came to the conclusion that true enthusiasts gravitate toward talking shop with one another. This is true even of casual situations. I’ve had plenty of “getting to know you” and “let’s just hang” social situations with other geeks where we might start out talking about any old thing but end up having an enthusiastic exchange about computers. I don’t view this is as negative. On the other hand, plenty of mailing lists and other communication media devoted to discussions of specific topics end up going “off-topic” as people socialize more generally. Fine, so it goes both ways.

It’s good to love what you love and even better to share it.

Alex Launi: Docky has been removed from Do!

Mon, 08 Mar 2010 03:00:10 +0000

Tonight I finally pushed the revisions to bzr that removed Docky from the Do source tree. For a while now Docky has been being developed as a stand alone application separate from Do. The two projects were limiting one another, so a decision was made to split them out. In the future there will be some of docky <–> do integration, but for now Docky is going to be the most kick ass Dock you’ve ever seen, and Do is going to be the most kickass launcher/everything-else-do-does you’ve ever seen.

In Ubuntu Lucid you can install Docky from the software center, otherwise the source is available at http://launchpad.net/docky. Today is the start of a new day in Do, development is going to start rolling again, and there should be a new kick ass release in the coming months. We’ve let things slide, but we’re kicking it back into gear.

Alex Launi: Anything but the buttons, or how I learned to stop clicking and love Do

Sun, 07 Mar 2010 20:00:34 +0000

I know this is a controversial opinion, but I want to be one of the few to publicly announce that they love the window controls on the left side of the window. They’re so slick looking! It takes about 10 minutes to adjust to the change, but the beauty is worth it. A very small price to pay for a major improvement of the look and feel of the desktop.

Now, for those who don’t want to adjust (which is fine, really!) I have a prescription for what ails you. Yes, this is a shameless plug, no this is not snake oil. This is real jawn which will make your left-side-window-controlled hell-of-a-life into the garden of eden you never dreamed it could be. 40 virgins? No.. Jimmy Hendrix and Neil Peart (yeah, I know he’s not dead, just bear with me) jam sessions? No… This isn’t religion, magic, mysticism, mass hysteria, or Minnesota swap gas. This the Do window manager plugin.

No video support? Try youtube.

And here’s a youtube version, if you’re not down with HTML5.

You can install it from apt://gnome-do and apt://gnome-do-plugins. Thw window manager plugin is enabled by default in Lucid (and maybe Karmic?)!

Paul Snyder: An Emacs Bisection

Fri, 05 Mar 2010 02:52:48 +0000

A problem has been following me around for a couple of years. Emacs, in which I have been spending more than a small portion of my time, has had a problem that would manifest in any non-X mode. It persisted across multiple terminals: putty on Windows back to a Linux box, emacs -nw in a GNOME Terminal , even a bog-standard Linux console. Specifically, hitting ‘Page Up’ and would result in the string ‘5~‘ being printed and ‘Page Down’ would result in '6~‘. Not deadly, but just enough to throw me off my stride when moving around files. I’d never gotten around to sorting in out, since it worked just fine when running as an X server.

This came up after the PLUG meeting last night, over pizza and beer at the Best House last night. As Walt pointed out, there was almost certainly a very good reason for this, and it would be living somewhere in the years of messy accretions that comprise my .emacs file and its satellites.

I knew this was probably the case. Like most .emacs files, mine was not planned. It grew from bits and useful snippets, mode hooks, random configurations, pieces that crept in as distribution cruft, and pastings of now-unknown provenance. The whole morass needs a good mucking out, but that’s not on the to-do list. Perhaps, though, I could finally get the nagging problem resolved.

The first test was an obvious one: remove all the startup files and see if the problem went away.

Now, at some point I acquired a set of XEmacs startup files. The reason for this is hazy…I haven’t used XEmacs in years. Nonetheless, there it is: an .xemacs folder with an init.el and custom.el. At some point, something broke when I tried to take my .emacs file and .emacs.d folder without bringing .xemacs along for the ride, so the path of least resistance was to let it follow me from machine to machine ever since.

After renaming all the startup files, the problem was indeed gone and I could happily page up and down. From there on, the process is a mechanical one: add files back in until it fails, then comment everything out in the file and add chunks back in until the problem returns. Once it does, you can be pretty sure you’ve isolated the problem.

Indeed, the problem was lurking in .xemacs/init.el, in the following pair of lines:

(global-set-key "\e[" 'enlarge-window) (global-set-key "\e]" 'shrink-window)

It seems to me like a slightly odd manifestion, but there you have it. With those lines removed, my Emacs bliss is unimpeded.

I really ought to get around to getting rid of that XEmacs cruft.

(CC-licensed image of a cricket from TaranRampersad’s flickr photostream.)

Paul Snyder: Extract n largest values from a Python dict

Thu, 04 Mar 2010 05:49:17 +0000

Having stashed a bunch of data in a python dict, one often needs a quick way to extract the n largest. Inconveniently for the particular problem I was working on, the key value for the dict was an object and the associated value was a floating-point weight. A quick-and-dirty method is to use insert all the dict-keys into a heap, using the dict-value as the heap-key Fortuitously, Python has both a heap module and convenient functions for using it in exactly this way.

import heapq

def dict_nlargest(d,n):
    return heapq.nlargest(n ,d, key = lambda k: d[k])

d = { 'a':10.0, 'b':2.0, 'c':5.0, 'd':3.0, 'e':15.0 }
print dict_nlargest(d,3)

Which neatly prints a list of the keys associated with the three largest values:

['e', 'a', 'c']

The heapq.nlargest() function can also use arbitrary object attributes as keys, as well, by supplying a one-argument function to the named parameter key. (This is the role of the lambda in the examples above and below.)

import heapq
class Toy:
    def __init__Toy(self, s, i, f):
        self.s = s
        self.i = i
        self.f = f

l = [ Toy('a', 10, 4.4), Toy('b', 4, 12.0),
      Toy('c', 2, 20.0), Toy('d', 1, 5.0) ]

# largest by the 'f' attribute
print heapq.nlargest(2, l, key = lambda o: o.f)

# largest by the 'i' attribute
print heapq.nlargest(2, l, key = lambda o: o.i)

This is fine for a one-off, but you’ll likely want to maintain this information in a data structure if this you’ll be performing this operation multiple times on a large collection.

yonkeltron: Neato article about NoSQL on Ars Technica

Wed, 24 Feb 2010 15:46:28 +0000

I am very excited by some of the NoSQL stuff out there and enjoyed a recent Ars Technica article. The first page is some neat background and the second page has a good overview of some NoSQL options out there.

http://arstechnica.com/business/data-centers/2010/02/-since-the-rise-of.ars/

JonathanD: Lower Merion laptop spying

Tue, 23 Feb 2010 15:53:05 +0000

I'm sure most of you have heard about the rather big story of Lower Merion School District allegedly spying on students using laptops in their school issued machines. As more news comes out about this, I'm personally getting more and more disturbed by it.

I think my views are somewhat unpopular, but I have a real problem with the basic premise that high school students (or any students) are untrustworthy and need anything approaching the sort of monitoring that went on. Even if we assume for a moment that they only ever used it as claimed, to recover stolen laptops, it seems to me even this was aimed at keeping students honest by not allowing them to steal these machines and sell them. While I don't question that such a thing may happen, I believe that protecting against it in this manner is not ideal, at best.

The issue isn't just a few laptops, and even if we find this school district was in fact using this software to spy on other occasions, I think it goes much further than just this story. In the grand scheme, this isn't even important. What IS important is the slow erosion of rights of our students. Students today are told they essentially have no rights beyond the right to an education. This incident is just an implementation of this policy, of showing these kids that they really don't have any rights, in or out of school. Month after month, new stories surface of how students are being punished or at least spoken to for things they are doing outside of school, for posts they are posting on line, for pictures they are taking and sending to friends. Why is this tolerated still?

Our rights are not something to just let slip away, and teaching our youth that they are mutable and unimportant is no way to raise the next generation in this nation. This is a deep issue with national, and even worldwide implications, not just a few pictures from a students laptop. Amidst all the chaos around this issue, lets not forget the big picture.

These kids are tomorrows adults. Start treating them like it.

JonathanD: fosscon continues, building a workshop, and more.

Tue, 23 Feb 2010 14:22:38 +0000

We're continuing to finalize details on fosscon, with registration now open and sponsorship opportunities posted. I'm terrifically excited about the prospect of this event and can't wait to see it realized.

We've also just moved, which is adding to the chaos... of course, by just moved, I mean we moved nearly a month ago, and we're still settling in, but the best part of this for me is clearly setting up my new basement workshop. There is TONS more space in this new house, and this means I have plenty of space to setup my equipment and have a little fun... finally. The only bad news is it looks like my o-scope gave up the ghost shortly after the move.

All in all, love the new place, hated the move, and hopefully we can stick here a decade at least.

Alex Launi: Making my bed

Sat, 20 Feb 2010 19:13:21 +0000

I’ve made a resolution to make my bed every day. This post is for public accountability of this resolution. I know that most people reading this will never be near my bed, but if we’re chatting at some point please ask me how it’s going and give me a hard time if I say I’ve been slacking.

No that’s not my bed.

Paul Snyder: Eliminating a nasty, loud ‘crack’ before playback in Ubuntu Karmic

Fri, 19 Feb 2010 23:00:59 +0000

Ever since installing Karmic on my main box, audio has been an intermittent annoyance. While I like the idea behind pulseaudio, its presence in Karmic has certainly has contributed to a few headaches (made painfully worse when I imprudently tried exploring the network multicast features). I’ve had most of the bugs worked out for a while, except for one: before a sound is played back, a loud, unpleasant, sharp crack is emitted by the speakers. After that, sound playback proceeds normally…music, further OS alert beeps, no problem. However, after a period of idleness without any sound activity (ten seconds, in fact), the next speaker access will result in the same loud report. Even though annoying, this problem has not been critical.

The box is custom-built system based around an Asus P6T motherboard, which has has an onboard RealTek ALC1200 for sound. This shows up in Linux as

00:1b.0 Audio device: Intel Corporation 82801JI (ICH10 Family)
HD Audio Controller

and thus uses the snd_hda_intel kernel module (Intel High-Definition Audio).

Having just had occasion to look into my /etc/modprobe.d/alsa-base.conf, I found this exceedingly suspicious pair of lines:

# Power down HDA controllers after 10 idle seconds
options snd-hda-intel power_save=10 power_save_controller=N

The 10-second time-out surely could not be a coincidence.

Sure enough, commenting it out has removed this lingering audio annoyance (which turns out to be completely unrelated to pulse). It appears this issue is already known: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/417302. As one commenter observed: “I was also affected by this. I think this is a bug – speakers should not produce unpleasant sounds for [no] apparent reason.” This is a sentiment with which I can only agree.

Alex Launi: Jan-Feb 2010 Record scores

Fri, 19 Feb 2010 20:56:45 +0000

I’ve been spending too much money on records recently so I’ve decided to give myself a soft-limit of $50/month to spend on records. At the end of each month I’ll do a post of what I got with my $50 that month. I say soft limit because if I come across a copy of SUNN White 1 for less than $100, I might buy it despite my $50 limit. I’ve already gone over that limit for February, so I won’t be buying any new records this month except for $1 porch sale/thrift store records.

2010 has been a good year for record hunting. Here are my 2010 record scores for January and February.

The Facedowninshit record is on yellow, too. Unfortunately, my record player broke the other night! I guess I’ll be scouring craigslist this weekend trying to find a new turntable.

JonathanD: We're having a fosscon!

Thu, 18 Feb 2010 12:40:14 +0000

This June, in Rochester. Fosscon is a free and open source software conference, assembled by the foss community and for the foss community. You can be a part of fosscon. All of us involved are very excited as this great project moves forward! Take a look at http://fosscon.org, registration will open soon. Besides checking out the website feel free to drop in #fosscon on chat.freenode.net to talk to others involved in fosscon.

yonkeltron: The zen of backup theory

Thu, 11 Feb 2010 17:33:42 +0000

When stuck inside due to all the snow, there is no better time to consider the topic of backup. Seriously, backup is important and the issue is fascinating. The area of backup brings together so many topics in computing. Think about it! To do backup successfully, you must deal with data transfer, data integrity validation, networks, distributed systems, compression and, if you’re doing it right, cryptography. It is not just computer science however, but also workflow management and that somewhat-nebulous-yet-often-referred-to thing of systems thinking.

I got to thinking about the topic of backups and was curious as to the state of research as well as backup tools. A great intro to so-called “backup theory” is available on the ‘Backup’ Wikipedia article and others have written on the subject (Google will verify that this is true). As it turns out, advances in storage have recently offered many new opportunities for improving the way that a given backup process might work.

Several distributed fault-tolerant filesystems are coming along quite nicely. Though the Google File System and the equivalent HDFS project which is part of Apache’s Hadoop have gotten much attention, there are other options. GlusterFS (follow GlusterFS on Twitter, perhaps?) and Ceph are two excellent examples of Free/Open Source Software projects which offer the compelling combo of fault tolerance and distributed storage. They each employ replication and have similar architectures insofar as they abstract individual machines into “chunks” or “blocks” and manage replication automatically. One interesting difference is that while GlusterFS exports filesystems “as-is” (see the docs for an explanation), Ceph exports entire block devices.

So then what about the theory and goals of backup? In my opinion, backing up is not enough. Having a backup plan and executing it perfectly doesn’t mean a thing if the data can’t be recovered. (In fact this issue relates strongly to a larger discussion of reliability. Rather than focusing exclusively on total time spent in a failure state, individuals concerned with reliability would also do well to consider how fast a system can recover from those failures. If a system can be up and running again after only 5 minutes, then that system can go down 12 times before reaching an hour of downtime. If another system takes 20 minutes to rebound from a failure, then that system can only go down 3 times!) I haven’t yet gotten to thinking about the problem of restoration following failure for anything other than plain old files. For example, I backup my personal data to servers in Philadelphia and California in addition to an external hard drive in my apartment. I make careful use of old stand-byes like tar, gzip and rsync along with a checksum utility like md5sum or, more recently, something in the SHA family. Plus I use the git revision control system for code and etckeeper for config. By backing up all of my configuration data in addition to my personal files, I make it so that I can easily return a given system to a usable state, if not restoring it perfectly. I have successfully restored systems by doing little more than a reverse rsync.

To be fair, I just spoke about about the few systems under my personal control which constitute a small and limited case. I back up to different locations which is good practice but my local copy is certainly not sufficient for anything industrial. If the disk breaks, I’m out of luck. Considering a larger-scale system is when GlusterFS, Ceph and others would come in handy. Obviously there are a great many books written about this topic but, for the purposes of discussion, if I were to build a platform for the reliable storage of huge amounts of data, my project would look something like this…

First, I would round up spare computers with room for extra disks. They machines would not need to be particularly fast or possess large amounts of memory. I’m not sure of the exact hardware requirements for either GlusterFS (vague wiki entry) or Ceph but it’s hard to imagine that they’d require huge amounts of anything but disk space. Anyway, if my organization had old desktops or something which were being replaced then they might be perfect candidates.

Second comes storage. It seems that at the time of this writing, one can purchase a 1TB hard disk for around US$85. Let us assume that 20 desktop machines could be procured and each had two spare disk slots. For around US$3500 (figuring 2 US$85 disks per machine and a little more for tax+shipping, etc.) one could buy 20TB of storage. Now, it’s not quite that simple as the fault-tolerance scheme in both GlusterFS and Ceph relies on replication. Assuming the accepted replication factor of 3 (a norm adhered to by the Google File System), that would reduce the 20TB storage block by a third leaving around 6 and two thirds terabytes of fault-tolerant storage. Filesystems of this type usually require cluster control processes which (ideally, I think) reside on dedicated machines so an extra machine or two would also be required. I got a good explanation of how metadata servers work in GFS/HDFS by reading the HDFS design document, actually. Metadata servers and other control processes serve similar functions. Ceph documents are very explicit about not having a single point of failure whereas GlusterFS is not quite so adamant. I need some help figuring out if GlusterFS is as fault-tolerant in that respect.

For under US$4000, one could theoretically build over 6TB of fault-tolerant distributed storage (provided that spare machines are plentiful, something which shouldn’t be a problem for organizations with a semi-regular hardware replacement cycle). Now, as for the usage of that storage system for backup, it’s a different piece of the discussion. I’ve seen quite a few setups where a single-but-very-large server provides networked storage to a large number of users via something like samba or NFS. In this case, the big server is usually blessed with some complicated RAID arrangement which people put entirely too much faith in. No one ever listens that RAID is not a backup strategy. Personally, I don’t believe in hardware reliability because it seems silly to spend money trying to prevent hardware from failing when you know it’s going to break (eventually) anyway. I’m not saying that RAID isn’t useful because it *is* useful and has a place. What I am saying is that it’s not backup. So if an organization has a big network storage server then where does that get backed up to? It’s hard to backup 6 TB off site but it can (certainly) be done. However, if an organization is lucky enough to have multiple buildings then a distributed storage cluster like I described earlier would be an excellent addition to the overall backup infrastructure. Putting a few machines in different buildings and using it as a place to shadow the main file server and whatever else needs to be backed up would grant an added measure of security.

Snow days are a good time for thinking and my backup jobs went smoothly. However, I feel as if I have begun a track of study which might yield some good results. Granted, it’s not just about the technology (humans screw things up) but the ability to build reliable and fault-tolerant storage systems for massive amounts of data using only commodity hardware is a huge boon to users everywhere. This concludes my backup rant.

Paul Snyder: Moonlight just works

Tue, 26 Jan 2010 03:39:43 +0000

On the way into Philadelphia today, my train was delayed. While the rain seemed rather pleasant as I walked to the station, the winds had been strong enough to disrupt service. An earlier train had been disabled on the line, blocking the tracks. In all, it made for a thoroughly tedious experience, and not one to encourage me in my attempts to use public transit nor bolster my (non-existent) love for SEPTA.

Eventually, a “rescue train” came along to extract we poor passengers from our misery. The extraction was not timely, and rather than arriving forty-five minutes early for the lecture on a multi-robot planning, I arrived forty-five minutes late: just in time to miss the end.

You can imagine I was quite pleased, then, when a friend told me the video would be available online. Unfortunately, the link he sent me was for a lecture earlier in the month. Even more distressingly, the recording was only provided via Microsoft Silverlight. I simply do not understand why a computer science department that runs mostly on Linux keeps returning to suckle at a Windows-only, proprietary teat. Disgruntled, I made a note to check out Moonlight at some point.

Somewhat later, I ran an apt-cache search moonlight on my main research box (currently on Karmic), and sure enough, moonlight-plugin-mozilla looked plenty promising. A simple sudo aptitude install moonlight-plugin-mozilla, and it was on its way.

I returned to writing email. A few minutes later, I heard the sound of talking: a video was playing somewhere. Irritated, I tried to figure out if a video one of my open tabs had launched itself. I was somewhat gobsmacked when I found it was the lecture mentioned above: unless I somehow managed to restart Firefox in a memoryless stupor, it seems that the Moonlight plugin insinuated itself into a running Firefox 3.5.7 instance, loaded the player, and kicked off the playback without any intervention. Impressively smooth and simple.

This may partially make up for my annoyance with Karmic’s compiz, which has an annoying habit of dying out from under X every day or two.

AndyML: 24 hours with the Google Nexus One

Sat, 09 Jan 2010 02:20:08 +0000

Day 1 down, we will see how the weekend goes. So far I am very happy with the size/shape, performance, and software, I just hope something can be done about the battery life.

mjd: A monad for probability and provenance

Fri, 08 Jan 2010 02:48:00 +0000

I don't quite remember how I arrived at this, but it occurred to me last week that probability distributions form a monad. This is the first time I've invented a new monad that I hadn't seen before; then I implemented it and it behaved pretty much the way I thought it would. So I feel like I've finally arrived, monadwise.

Suppose a monad value represents all the possible outcomes of an event, each with a probability of occurrence. For concreteness, let's suppose all our probability distributions are discrete. Then we might have:

	data ProbDist p a = ProbDist [(a,p)] deriving (Eq, Show)
	unpd (ProbDist ps) = ps

Each a is an outcome, and each p is the probability of that outcome occurring. For example, biased and unbiased coins:

    unbiasedCoin = ProbDist [ ("heads", 0.5),
                              ("tails", 0.5) ];

    biasedCoin   = ProbDist [ ("heads", 0.6),
                              ("tails", 0.4) ];

Or a couple of simple functions for making dice:

    import Data.Ratio

    d sides = ProbDist [(i, 1 % sides) | i - [1 .. sides]]
    die = d 6

d n is an n-sided die.

The Functor instance is straightforward:

    instance Functor (ProbDist p) where
      fmap f (ProbDist pas) = ProbDist $ map (\(a,p) -> (f a, p)) pas

The Monad instance requires return and >>=. The return function merely takes an event and turns it into a distribution where that event occurs with probability 1. I find join easier to think about than >>=. The join function takes a nested distribution, where each outcome of the outer distribution specifies an inner distribution for the actual events, and collapses it into a regular, overall distribution. For example, suppose you put a biased coin and an unbiased coin in a bag, then pull one out and flip it:

	  bag :: ProbDist Double (ProbDist Double String)
	  bag = ProbDist [ (biasedCoin, 0.5),
                           (unbiasedCoin, 0.5) ]

The join operator collapses this into a single ProbDist Double String:

	ProbDist [("heads",0.3),
                  ("tails",0.2),
                  ("heads",0.25),
                  ("tails",0.25)]

It would be nice if join could combine the duplicate heads into a single ("heads", 0.55) entry. But that would force an Eq a constraint on the event type, which isn't allowed, because (>>=) must work for all data types, not just for instances of Eq. This is a problem with Haskell, not with the monad itself. It's the same problem that prevents one from making a good set monad in Haskell, even though categorially sets are a perfectly good monad. (The return function constructs singletons, and the join function is simply set union.) Maybe in the next language.

Perhaps someone else will find the >>= operator easier to understand than join? I don't know. Anyway, it's simple enough to derive once you understand join; here's the code:

	instance (Num p) => Monad (ProbDist p) where
	  return a = ProbDist [(a, 1)]
	  (ProbDist pas) >>= f = ProbDist $ do
				   (a, p) <- pas
				   let (ProbDist pbs) = f a
				   (b, q) <- pbs
				   return (b, p*q)

So now we can do some straightforward experiments:

	liftM2 (+) (d 6) (d 6)

	ProbDist [(2,1 % 36),(3,1 % 36),(4,1 % 36),(5,1 % 36),(6,1 %
	36),(7,1 % 36),(3,1 % 36),(4,1 % 36),(5,1 % 36),(6,1 %
	36),(7,1 % 36),(8,1 % 36),(4,1 % 36),(5,1 % 36),(6,1 %
	36),(7,1 % 36),(8,1 % 36),(9,1 % 36),(5,1 % 36),(6,1 %
	36),(7,1 % 36),(8,1 % 36),(9,1 % 36),(10,1 % 36),(6,1 %
	36),(7,1 % 36),(8,1 % 36),(9,1 % 36),(10,1 % 36),(11,1 %
	36),(7,1 % 36),(8,1 % 36),(9,1 % 36),(10,1 % 36),(11,1 %
	36),(12,1 % 36)]

This is nasty-looking; we really need to merge the multiple listings of the same event. Here is a function to do that:

        agglomerate :: (Num p, Eq b) => (a -> b) -> ProbDist p a -> ProbDist p b
        agglomerate f pd = ProbDist $ foldr insert [] (unpd (fmap f pd)) where
          insert (k, p) [] = [(k, p)]
          insert (k, p) ((k', p'):kps) | k == k' = (k, p+p'):kps
                                       | otherwise = (k', p'):(insert (k,p) kps)


        agg :: (Num p, Eq a) => ProbDist p a -> ProbDist p a
        agg = agglomerate id

Then agg $ liftM2 (+) (d 6) (d 6) produces:

        ProbDist [(12,1 % 36),(11,1 % 18),(10,1 % 12),(9,1 % 9),
                  (8,5 % 36),(7,1 % 6),(6,5 % 36),(5,1 % 9),
                  (4,1 % 12),(3,1 % 18),(2,1 % 36)]

Hey, that's correct.

There must be a shorter way to write insert. It really bothers me, because it looks look it should be possible to do it as a fold. But I couldn't make it look any better.

You are not limited to calculating probabilities. The monad actually will count things. For example, let us throw three dice and count how many ways there are to throw various numbers of sixes:

        eq6 n = if n == 6 then 1 else 0
        agg $ liftM3 (\a b c -> eq6 a + eq6 b + eq6 c) die die die

      ProbDist [(3,1),(2,15),(1,75),(0,125)]

There is one way to throw three sixes, 15 ways to throw two sixes, 75 ways to throw one six, and 125 ways to throw no sixes. So ProbDist is a misnomer.

It's easy to convert counts to probabilities:

	probMap :: (p -> q) -> ProbDist p a -> ProbDist q a
	probMap f (ProbDist pds) = ProbDist $ (map (\(a,p) -> (a, f p))) pds

	normalize :: (Fractional p) => ProbDist p a -> ProbDist p a
	normalize pd@(ProbDist pas) = probMap (/ total) pd where
	    total = sum . (map snd) $ pas

        normalize $ agg $ probMap toRational $ 
               liftM3 (\a b c -> eq6 a + eq6 b + eq6 c) die die die

      ProbDist [(3,1 % 216),(2,5 % 72),(1,25 % 72),(0,125 % 216)]

I think this is the first time I've gotten to write die die die in a computer program.

The do notation is very nice. Here we calculate the distribution where we roll four dice and discard the smallest:

        stat = do
                 a <- d 6
                 b <- d 6
                 c <- d 6
                 d <- d 6
                 return (a+b+c+d - minimum [a,b,c,d])

        probMap fromRational $ agg stat

	ProbDist [(18,1.6203703703703703e-2),
                  (17,4.1666666666666664e-2), (16,7.253086419753087e-2),
                  (15,0.10108024691358025),   (14,0.12345679012345678),
                  (13,0.13271604938271606),   (12,0.12885802469135801),
                  (11,0.11419753086419752),   (10,9.41358024691358e-2),
                   (9,7.021604938271606e-2),   (8,4.7839506172839504e-2),
                   (7,2.9320987654320986e-2),  (6,1.6203703703703703e-2),
                   (5,7.716049382716049e-3),   (4,3.0864197530864196e-3),
                   (3,7.716049382716049e-4)]

One thing I was hoping to get didn't work out. I had this idea that I'd be able to calculate the outcome of a game of craps like this:

	dice = liftM2 (+) (d 6) (d 6)

	point n = do
	  roll <- dice
	  case roll of 7 -> return "lose"
                       _ | roll == n  = "win"
                       _ | otherwise  = point n

        craps = do
          roll <- dice
          case roll of 2 -> return "lose"
                       3 -> return "lose"
                       4 -> point 4
                       5 -> point 5
                       6 -> point 6
                       7 -> return "win"
                       8 -> point 8
                       9 -> point 9
                       10 -> point 10
                       11 -> return "win"
                       12 -> return "lose"

This doesn't work at all; point is an infinite loop because the first value of dice, namely 2, causes a recursive call. I might be able to do something about this, but I'll have to think about it more.

It also occurred to me that the use of * in the definition of >>= / join could be generalized. A couple of years back I mentioned a paper of Green, Karvounarakis, and Tannen that discusses "provenance semirings". The idea is that each item in a database is annotated with some "provenance" information about why it is there, and you want to calculate the provenance for items in tables that are computed from table joins. My earlier explanation is here.

One special case of provenance information is that the provenances are probabilities that the database information is correct, and then the probabilities are calculated correctly for the joins, by multiplication and addition of probabilities. But in the general case the provenances are opaque symbols, and the multiplication and addition construct regular expressions over these symbols. One could generalize ProbDist similarly, and the ProbDist monad (even more of a misnomer this time) would calculate the provenance automatically. It occurs to me now that there's probably a natural way to view a database table join as a sort of Kleisli composition, but this article has gone on too long already.

Happy new year, everyone.

[ Addendum 20100103: unsurprisingly, this is not a new idea. Several readers wrote in with references to previous discussion of this monad, and related monads. It turns out that the idea goes back at least to 1981. ]

My thanks to Graham Hunter for his donation.

AndyML: How to: Mount an ISO image under Linux

Thu, 07 Jan 2010 17:04:57 +0000

I had an iso I needed to get files off real quick – not worth burning the CD. I used this method to mount it in linux. http://www.cyberciti.biz/tips/how-to-mount-iso-image-under-linux.html

AndyML: How can I disable the User Account Control (UAC) feature on my Windows Vista computer?

Thu, 07 Jan 2010 16:59:59 +0000

Came across this great how-to on disabling that annoying Administrative Rights popup in Vista (called uac). Thanks! http://www.petri.co.il/disable_uac_in_windows_vista.htm

AndyML: Hack your old wireless router into switch-only mode.

Thu, 07 Jan 2010 16:53:29 +0000

I got an e-mail from a friend last night. Hey Andy, I wonder if I can bother you for some network troubleshooting? Now, I have my FIOS wireless router set up to talk to a linksys wireless game adapter wired to my xbox. works well (most of the time). I am getting a tv and blu-ray player that also [...]

mjd: A short bibliography of probability monads

Mon, 04 Jan 2010 15:03:00 +0000

Several people helpfully wrote to me to provide references to earlier work on probability distribution monads. Here is a summary:

Material related to Martin Erwig and Steve Kollmansberger's probability library:
- Probabilistic Functional Programming in Haskell, their 2006 paper
- the package, described in the paper, and specifically its probability distribution monad
- Some commentary on this library, by someone who doesn't put their name on their blog.
Some stuff from Dan Piponi's blog:
- "Monads, Vector Spaces and Quantum Mechanics, part I"
- and part II
Eric Kidd's blog: "What would a programming language look like if Bayes’ rule were as simple as an if statement?"

My thanks to Stephen Tetley, Gaal Yahas, and Like Palmer for these.

I did not imagine that my idea was a new one. I arrived at it by thinking about List as a representation of non-deterministic computation. But if you think of it that way, the natural interpretation is that every list element represents an equally likely outcome, and so annotating the list elements with probabilities is the obvious next step. So the existence of the Erwig library was not a big surprise.

A little more surprising though, were the references in the Erwig paper. Specifically, the idea dates back to at least 1981; Erwig cites a paper that describes the probability monad in a pure-mathematics context.

Nobody responded to my taunting complaint about Haskell's failure to provide support a good monad of sets. It may be that this is because they all agree with me. (For example, the documentation of the Erwig package says "Unfortunately we cannot use a more efficient data structure because the key type must be of class Ord, but the Monad class does not allow constraints for result types.") But a number of years ago I said that the C++ macro processor blows goat dick. I would not have put it so strongly had I not naïvely believed that this was a universally-held opinion. But no, plenty of hapless C++ programmers wrote me indignant messages defending their macro system. So my being right is no guarantee that language partisans will not dispute with me, and the Haskell community's failure to do so in this case reflects well on them, I think.

yonkeltron: Cool Free/Open Source Software from Google

Mon, 28 Dec 2009 21:59:10 +0000

As has been spoken about endlessly (OStatic, OSnews), there is a great blog post from 0×1fff with many (started at 35, is now many more) open source projects from Google. In fact and indeed, there is some cool stuff on there. I knew about Caja and Protocol Buffers (wish there was a JS port of protocol buffers) but did not know about CRUSH and skia. Honestly, there are plenty of cool projects out there and my already-positive opinion of Google is only bolstered by the fact that they give back so willingly. Gotta love it.

mjd: Gray code at the pediatrician's office

Wed, 16 Dec 2009 17:03:00 +0000

Last week we took Iris to the pediatrician for a checkup, during which they weighed, measured, and inoculated her. The measuring device, which I later learned is called a stadiometer, had a bracket on a slider that went up and down on a post. Iris stood against the post and the nurse adjusted the bracket to exactly the top of her head. Then she read off Iris's height from an attached display.

How did the bracket know exactly what height to report? This was done in a way I hadn't seen before. It had a photosensor looking at the post, which was printed with this pattern:

(Click to view the other pictures I took of the post.)

The pattern is binary numerals. Each numeral is a certain fraction of a centimeter high, say 1/4 centimeter. If the sensor reads the number 433, that means that the bracket is 433/4 = 108.25 cm off the ground, and so that Iris is 108.25 cm tall.

The patterned strip in the left margin of this article is a straightforward translation of binary numerals to black and white boxes, with black representing 1 and white representing 0:

0000000000
0000000001
0000000010
0000000011
0000000100
0000000101
0000000101
...
1111101000
1111101001
...
1111111111

If you are paying attention, you will notice that although the strip at left is similar to the pattern in the doctor's office, it is not the same. That is because the numbers on the post are Gray-coded.

Gray codes solve the following problem with raw binary numbers. Suppose Iris is close to 104 = 416/4 cm tall, so that the photosensor is in the following region of the post:

...
0110100001 (417)
0110100000 (416)
0110011111 (415)
0110011110 (414)
...

But suppose that the sensor (or the post) is slightly mis-aligned, so that instead of properly reading the (416) row, it reads the first half of the (416) row and last half of the (415) row. That makes 0110111111, which is 447 = 111.75 cm, an error of almost 7.5%. (That's three inches, for my American and Burmese readers.) Or the error could go the other way: if the sensor reads the first half of the (415) and the second half of the (416) row, it will see 0110000000 = 384 = 96 cm.

Gray code is a method for encoding numbers in binary so that each numeral differs from the adjacent ones in only one position:

0000000000
0000000001
0000000011
0000000010
0000000110
0000000111
0000000101
0000000100
0000001100
...
1000011100
1000011101
...
1000000000

This is the pattern from the post, which you can also see at the right of this article.

Now suppose that the mis-aligned sensor reads part of the (416) line and part of the (417) line. With ordinary binary coding, this could result in an error of up to 7.75 cm. (And worse errors for children of other heights.) But with Gray coding no error results from the misreading:

...
0101110000 (417)
0101010000 (416)
0101010001 (415)
0101010011 (414)
...

No matter what parts of 0101110000 and 0101110001 are stitched together, the result is always either 416 or 417.

Converting from Gray code to standard binary is easy: take the binary expansion, and invert every bit that is immediately to the right of a 1 bit. For example, in 1111101000, each red bit is to the right of a 1, and so is inverted to obtain the Gray code 1000011100.

Converting back is also easy: of the Gray code. Replace every sequence of the form 1000...01 with 1111...10; also replace 1000... with 1111... if it appears at the end of the code. For example, Gray code 1000011100 contains two such sequences, 100001 and 11, which are replaced with 111110 and 10, to give 1111101000.

yonkeltron: ES5 is finally here, JavaScript geeks rejoice!

Wed, 16 Dec 2009 01:03:56 +0000

So, earlier this week, it was announced that ECMAScript 5 has finally been released. This is a good thing and I caught the highlights on InfoQ. The full draft is a 252 page PDF beast of a document which covers basically about everything there is to cover. The things which strike me as interesting are the improved Array functions (like map, filter and reduce), some (finally) ways to harden Objects (in the form of freeze and seal) and JSON in the language. The other big deal which has me excited is the availability of a strict mode which has been spoken about by Douglas Crockford in his Google Tech Talk as well in his book JavaScript: The Good Parts, which you should buy. Honestly, it makes you appreciate JavaScript so very, very much as D-Crock highlights the best and worst features of JavaScript.

In reality, this has been a big month for JavaScript with Google open sourcing its internal JS toolkit, Closure along with much attention being paid to to projects like NodeJS (for network stuff) and CommonJS (for everything else).

mjd: Monads are like burritos

Tue, 15 Dec 2009 06:26:00 +0000

A few months ago Brent Yorgey complained about a certain class of tutorials which present monads by explaining how monads are like burritos.

At first I thought the choice of burritos was only a facetious reference to the peculiar and sometimes strained analogies these tutorials make. But then I realized that monads are like burritos.

I will explain.

A monad is a special kind of a functor. A functor F takes each type T and maps it to a new type FT. A burrito is like a functor: it takes a type, like meat or beans, and turns it into a new type, like beef burrito or bean burrito.

A functor must also be equipped with a map function that lifts functions over the original type into functions over the new type. For example, you can add chopped jalapeños or shredded cheese to any type, like meat or beans; the lifted version of this function adds chopped jalapeños or shredded cheese to the corresponding burrito.

A monad must also possess a unit function that takes a regular value, such as a particular batch of meat, and turns it into a burrito. The unit function for burritos is obviously a tortilla.

Finally, a monad must possess a join function that takes a ridiculous burrito of burritos and turns them into a regular burrito. Here the obvious join function is to remove the outer tortilla, then unwrap the inner burritos and transfer their fillings into the outer tortilla, and throw away the inner wrappings.

The map, join, and unit functions must satisfy certain laws. For example, if B is already a burrito, and not merely a filling for a burrito, then join(unit(B)) must be the same as B. This means says that if you have a burrito, and you wrap it in a second tortilla, and then unwrap the contents into the outer tortilla, the result is the same as what you started with.

This is true because tortillas are indistinguishable.

I know you are going to point out that some tortillas have the face of Jesus. But those have been toasted, and so are unsuitable for burrito-making, and do not concern us here.

So monads are indeed like burritos.

I asked Brent if this was actually what he had in mind when he first suggested the idea of tutorials explaining monads in terms of burritos, and if everyone else had understood this right away.

But he said no, I was the lone genius.

mjd: World's shortest explanation of Gödel's theorem

Tue, 15 Dec 2009 02:58:00 +0000

A while back I started writing up an article titled "World's shortest explanation of Gödel's theorem". But I didn't finish it, and later I encountered Raymond Smullyan's version, which is much shorter anyway. So here, shamelessly stolen from Smullyan, is the World's shortest explanation of Gödel's theorem.

We have some sort of machine that prints out statements in some sort of language. It needn't be a statement-printing machine exactly; it could be some sort of technique for taking statements and deciding if they are true. But let's think of it as a machine that prints out statements.

In particular, some of the statements that the machine might (or might not) print look like these:

`P*`x	(which means that	the machine will print x)
`NP*`x	(which means that	the machine will never print x)
`PR*`x	(which means that	the machine will print xx)
`NPR*`x	(which means that	the machine will never print xx)

For example, NPR*FOO means that the machine will never print FOOFOO. NP*FOOFOO means the same thing. So far, so good.

Now, let's consider the statement NPR*NPR*. This statement asserts that the machine will never print NPR*NPR*.

Either the machine prints NPR*NPR*, or it never prints NPR*NPR*.

If the machine prints NPR*NPR*, it has printed a false statement. But if the machine never prints NPR*NPR*, then NPR*NPR* is a true statement that the machine never prints.

So either the machine sometimes prints false statements, or there are true statements that it never prints.

So any machine that prints only true statements must fail to print some true statements.

Or conversely, any machine that prints every possible true statement must print some false statements too.

Order
5000 B.C. and Other Philosophical Fantasies

with kickback
no kickback

The proof of Gödel's theorem shows that there are statements of pure arithmetic that essentially express NPR*NPR*; the trick is to find some way to express NPR*NPR* as a statement about arithmetic, and most of the technical details (and cleverness!) of Gödel's theorem are concerned with this trick. But once the trick is done, the argument can be applied to any machine or other method for producing statements about arithmetic.

The conclusion then translates directly: any machine or method that produces statements about arithmetic either sometimes produces false statements, or else there are true statements about arithmetic that it never produces. Because if it produces something like NPR*NPR* then it is wrong, but if it fails to produce NPR*NPR*, then that is a true statement that it has failed to produce.

So any machine or other method that produces only true statements about arithmetic must fail to produce some true statements.

Hope this helps!

(This explanation appears in Smullyan's books 5000 BC and Other Philosophical Fantasies, chapter 3, section 65 (which is where I saw it) and also in The Mystery of Scheherezade.)

I gratefully acknowledge Charles Colht for his generous donation to this blog.

[ Addendum 20091214: Another article on the same topic. ]

mjd: Another short explanation of Gödel's theorem

Mon, 14 Dec 2009 19:30:00 +0000

In yesterday's article, I said:

A while back I started writing up an article titled "World's shortest explanation of Gödel's theorem". But I didn't finish it...

I went and had a look to see what was wrong with it, and to my surprise, there seemed to be hardly anything wrong with it. Perhaps I just forgot to post it. So if you disliked yesterday's brief explanation of Gödel's theorem—and many people did—you'll probably dislike this one even more. Enjoy!

A reader wrote to question my characterization of Gödel's theorem in the previous article. But I think I characterized it correctly; I said:

The only systems of mathematical axioms strong enough to prove all true statements of arithmetic, are those that are so strong that they also prove all the false statements of arithmetic.

I'm going to explain how this works.

You start by choosing some system of mathematics that has some machinery in it for making statements about things like numbers and for constructing proofs of theorems like 1+1=2. Many such systems exist. Let's call the one we have chosen M, for "mathematics".

Gödel shows that if M has enough mathematical machinery in it to actually do arithmetic, then it is possible to produce a statement S whose meaning is essentially "Statement S cannot be proved in system M."

It is not at all obvious that this is possible, or how it can be done, and I am not going to get into the details here. Gödel's contribution was seeing that it was possible to do this.

So here's S again:

S: Statement S cannot be proved in system M.

Now there are two possibilities. Either S is in fact provable in system M, or it is not. One of these must hold.

If S is provable in system M, then it is false, and so it is a false statement that can be proved in system M. M therefore proves some false statements of arithmetic.

If S is not provable in system M, then it is true, and so it is a true statement that cannot be proved in system M. M therefore fails to prove some true statements of arithmetic.

So something goes wrong with M: either it fails to prove some true statements, or else it succeeds in proving some false statements.

List of topics I deliberately omitted from this article, that mathematicians should not write to me about with corrections: Presburger arithmetic. Dialetheism. Inexhaustibility. ω-incompleteness. Non-RE sets of axioms.

Order
Godel's Theorem: An Incomplete Guide to Its Use and Abuse

with kickback
no kickback

Order
Inexhaustibility: A Non-Exhaustive Treatment

with kickback
no kickback

Well, I see now that left out the step where I go from "M proves a false statement" to "M proves all false statements". Oh well, another topic for another post.

If you liked this post, you may enjoy Torkel Franzén's books Godel's Theorem: An Incomplete Guide to Its Use and Abuse and Inexhaustibility: A Non-Exhaustive Treatment. If you disliked this post, you are even more likely to enjoy them.

Many thanks to Robert Bond for his contribution.

mjd: On failing open

Sat, 12 Dec 2009 06:11:00 +0000

An axiom of security analysis is that nearly all security mechanisms must fail closed. What this means is that if there is an uncertainty about whether to grant or to deny access, the right choice is nearly always to deny access.

For example, consider a login component that accepts a username and a password and then queries a remote authentication server to find out if the password is correct. If the connection to the authentication server fails, or if the authentication server is down, the login component must decide whether to grant or deny access, in the absence of correct information from the server. The correct design is almost certainly to "fail closed", and to deny access.

I used to teach security classes, and I would point out that programs sometimes have bugs, and do the wrong thing. If your program has failed closed, and if this is a bug, then you have an irate user. The user might call you up and chew you out, or might chew you out to your boss, and they might even miss a crucial deadline because your software denied them access when it should have granted access. But these are relatively small problems. If your program has failed open, and if this is a bug, then the user might abscond with the entire payroll and flee to Brazil.

(I was once teaching one of these classes in Lisbon, and I reached the "flee to Brazil" example without having realized ahead of time that this had greater potential to offend the Portuguese than many other people. So I apologized. But my hosts very kindly told me that they would have put it the same way, and that in fact the Mayor of Lisbon had done precisely what I described a few years before. The moral of the story is to read over the slides ahead of time before giving the talk.)

But I digress. One can find many examples in the history of security that failed the wrong way.

However, the issue is on my mind because I was at a job interview a few weeks ago with giant media corporation XYZ. At the interview, we spent about an hour talking about an architectural problem they were trying to solve. XYZ operates a web site where people can watch movies and TV programs online. Thy would like to extend the service so that people who subscribe to premium cable services, such as HBO, can authenticate themselves to the web site and watch HBO programs there; HBO non-subscribers should get only free TV content. The problem in this case was that the authentication data was held on an underpowered legacy system that could serve only a small fraction of the requests that came in.

The solution was to cache the authentication data on a better system, and gather and merge change information from the slow legacy system as possible.

I observed during the discussion that this was a striking example of the rare situation in which one wants the authentication system to fail open instead of closed. For suppose one grants access that should not be granted. Then someone on the Internet gets to watch a movie or an episode of The Sopranos for free, which is not worth getting excited about and which happens a zillion times a day anyhow.

But suppose the software denies access that should have been granted. Then there is a legitimate paying customer who has paid to watch The Sopranos, and we told them no. Now they are a legitimately irate customer, which is bad, and they may call the support desk, costing XYZ Corp a significant amount of money, which is also bad. So all other things being equal, one should err on the side of lenity: when in doubt, grant access.

I would like to thank Andrew Lenards for his gift.

mjd: Dijkstra was not insane

Sat, 12 Dec 2009 05:53:00 +0000

Recently, a reader on the Higher-Order Perl discussion mailing list made a remark about Edsger Dijkstra and his well-known opposition to the break construction (in Perl, last) that escapes prematurely from a loop. People often use this as an example to show that Dijkstra was excessively doctrinaire, and out of touch with the reality of programming^[1], but usually it's because they don't know what his argument was.

I wrote a response, explaining where Dijkstra was coming from, and I am very happy with how it came out, so I'm reposting it here.

The list subscriber said, in part:

On a side note, I never read anything by Dijkstra that wasn't noticeably out of touch with the reality of programming, which qualifies them as screeds to me.
And I say that as a former Pascal programmer, and as one who has read, and bought into, things like Kernighan's "Why Pascal is Not My Favorite Programming Language" and the valid rants about how some form of breaking out of a loop without having to proceed to the end is very useful, without destroying structure (except by Dijkstra's definition of structure)...

A lot of people bring up the premature-loop-exit prohibition without understanding why Dijkstra suggested it; it wasn't just that he was a tightassed Dutchman.

Dijkstra's idea was this: suppose you want to prove, mathematically, that your program does what it is supposed to do. Please, everyone, suspend your judgment of this issue for a few paragraphs, and bear with me. Let's really suppose that we want to do this.

Dijkstra's idea is that the program is essentially a concatenation of blocks, each of which is trying to accomplish something or other, and each of which does not make sense to run unless some part of the program state is set up for it ahead of time. For example, the program might be to print a sorted list of links from a web page. Then the obvious blocks are:

A: get the web page and store it in a variable
B: extract the links from the text in the variable into an array
C: sort the array
D: print out the array contents

Section C is trying to sort the array; if it is correct then the array will be sorted by the time step D commences. But it doesn't make sense to commence step C unless the array is populated. Garbage in, garbage out, as they used to say when I was in elementary school.

We say that the "precondition" for C is that the array be populated with URLs, and the "postcondition" is that the array be in sorted order. What you would want to prove about C is that if the precondition holds—that is, if the array is properly populated before C begins—then the postcondition will hold too—that is, the array will be in sorted order when C completes.

It occurs to me that calling this a "proof" is probably biasing everyone's thinking. Let's forget about mathematical proofs and just think about ordinary programmers trying to understand if the program is correct. If the intern in the next cubicle handed you his code for this program, and you were looking it over, you would probably think in very much this way: you would identify block C (maybe it's a subroutine, or maybe not) and then you would try to understand if C, given an array of URLs, would produce a properly sorted array by the time it was done.

C itself might depend on some sub-blocks or subroutines that performed sub-parts of the task; you could try to understand them similarly.

Having proved (or convinced yourself) that C will produce the postcondition "array contains sorted list of URLs", you are in an excellent position to prove (or convince yourself) that block D prints out a sorted array of URLs, which is what you want. Without that belief about C, you are building on sand; you have almost nothing to go on, and you can conclude hardly anything useful about the behavior of D.

Now consider a more complex block, one of the form:

        if (q) { E; }
        else { F; }

Suppose you believe that code E, given precondition x, is guaranteed to produce postcondition y. And suppose you believe the same thing about F. Then you can conclude the same thing about the entire if-else block: if x was true before it began executing, then y will be true when it is done.^[2] So you can build up proofs (or beliefs) about small bits of code into proofs (or beliefs) about larger ones.

We can understand while loops similarly. Suppose we know that condition p is true prior to the commencement of some loop, and that if p is true before G executes, then p will also be true when G finishes. Then what can we say about this loop?

        while (q) { G; }

We can conclude that if p was true before the loop began, then p will still be true, and q will be false, when the loop ends.

BUT BUT BUT BUT if your language has break, then that guarantee goes out the window and you can conclude nothing. Or at the very least your conclusions will become much more difficult. You can no longer treat G atomically; you have to understand its contents in detail.

So this is where Dijkstra is coming from: features like break^[3] tend to sabotage the benefits of structured programming, and prevent the programmer from understanding the program as a composition of independent units. The other subscriber made a seemingly disparaging reference to "Dijkstra's idea of structure", but I hope it is clear that it was not an arbitrary idea. Dijkstra's idea of structure is what will allow you to understand a large program as a collection of modules.

Regardless of your opinion about formal verification methods, or correctness proofs, or the practicality of omitting break from your language, it should at least be clear that Dijkstra was not being doctrinaire just for the sake of doctrine.

Some additional notes

Here are some interesting peripheral points that I left out of my main discussion because I wanted to stick to the main point, which was: "Dijkstra was not insane".

I said in an earlier post that "I often find Dijkstra's innumerable screeds very tiresome in their unkind, unforgiving, and unrealistic attitudes toward programmers." But despite this, I believe he was a brilliant thinker, and almost every time he opened his mouth it was to make a carefully-considered argument. You may not like him, and you may not agree with him, but you'll be better off listening to him.

An archive of Dijkstra's miscellaneous notes and essays (a pre-blogging blog, if you like) is maintained at the University of Texas. I recommend it.
I said:
```
                if (q) { E; }
                else { F; }
```
Suppose you believe that code E, given precondition x, is guaranteed to produce postcondition y. And suppose you believe the same thing about F. Then you can conclude the same thing about the entire if-else block.
Actually, your job is slightly easier. Let's write this:

[x] E [y]
to mean that code E, given precondition x, produces postcondition y. That is, if we know that x is true when E begins execution, then we know that y is true when E finishes. Then my quoted paragraph above says that from these:

[x] E [y]
[x] F [y]
we can conclude this:

[x] if (q) {E} else {F} [y]
But actually we can make a somewhat stronger statement. We can make the same conclusion from weaker assumptions. If we believe these:
[x and q] E [y]
[x and not q] F [y]
then we can conclude this:

[x] if (q) {E} else {F} [y]
In fact this precisely expresses the complete semantics of the if-else construction. Why do we use if-else blocks anyway? This is the reason: we want to be able to write code to guarantee something like this:

[x] BLAH [y]
but we only know how to guarantee
[x and q] FOO [y]
and
[x and not q] BAR [y]
for some q. So we write two blocks of code, each of which accomplishes y under some circumstances, and use if-else to make sure that the right one is selected under the right circumstances.
Similar to break (but worse), in the presence of goto you are on very shaky ground in trying to conclude anything about whether the program is correct. Suppose you know that C is correct if its precondition (an array of URLs) is satisfied. And you know that B will set up that precondition (that is, the array) if its precondition is satisfied, so it seems like you are all right. But no, because block W somewhere else might have goto C; and transfer control to C without setting up the precondition, and then C could cause winged demons to fly out of your nose.

yonkeltron: Messing with OpenStreetMap

Thu, 26 Nov 2009 03:31:34 +0000

Some people might not see the reason for a project like OpenStreetMap when there are plenty of good mapping products and services laying around. I am not one of them. Whenever I use a GPS, I think quite a bit about it’s inner workings. How does it figure out which route is best? How does it calculate things on the fly? All of these questions usually lead me to think, at one point or another, that it depends very much on the data. While most mapping services and individual GPS devices use various algorithms for calculating routes, etc. (prob based on some weighted graph or something), they also rely on different sets of map data. A GPS can only tell you where on the planet you are, not what road you are on. For that, it needs map data. The only issue is that all of the map data used by popular services is proprietary!

Enter, OpenStreetMap. Seeded with the geographical data made publicly available by various governments and public universities, OpenStreetMap provides Free (as in freedom, licensed under CC-BY-SA) map data to anyone who wants it. The data is usually in pretty good shape because the initial measurements are in good shape. However, things aren’t perfect. Lucky for the web, OpenStreetMap.org allows users to help improve the data in a number of ways.

First, users can upload GPS traces to help improve the quality of unmapped regions such as seriously-rural areas along with bike and hiking trails. Second, users are able to tweak the mapping data to correct errors. There are a number of ways to do this but OpenStreetMap.org has an online editor which lets you overlay OSM data onto sattelite imagery so you can move those roads, landmarks and the like into the right location. In about an hour, I had cleaned up much of my hometown and began to add local landmarks, parks and buildings. It’s quite easy.

The project itself seems off to a great start and the planet shows a fair bit of activity. In particular, I like the idea of mapping parties where people get together and work on a given area. This seems like a great way to give back to the community and I plan to float the idea at the next SCOSUG meeting.

yonkeltron: Data visualization on a web page

Sun, 01 Nov 2009 21:06:07 +0000

Quickie: Two of my favorite ways to get data visualized on a web page are the Google Chart API and flot, the amazing canvas-based plotting library built on top of jQuery.

The Google Chart API provides a rediculously clever way to get high-quality information graphics which are generated on the back of the clearly-amazing Google infrastructure. You just use the URL layout provided and it sort of just works. All types of charts can be created. It’s very nice if you’re willing to take the time to piece together the URLs in the proper format. There are some abstractions, though.

Then there’s flot (which I’m told is Swedish for “pretty”). Flot is a library written in JavaScript on top of jQuery which produces very nice charts inside a canvas element. The demos are quite gorgeous and it’s operation seems straightforward enough. As a side note, Lift has a built-in flot widget.

JonathanD: CPOSC

Fri, 16 Oct 2009 21:04:13 +0000

Tomorrow I'll be heading to CPOSC, and speaking there too, it seems. It's been quite a while since I've spoken in front of a group larger than perhaps a dozen, so here's to not fainting.

The event looks like lots of fun, and I'll know at least a few people there (lyz and waltman) so well, away we go. Heading to the train.

yonkeltron: Good talk about DSL construction in JS

Mon, 12 Oct 2009 01:25:36 +0000

Neat talk, should remind everyone about the joys of doing cool stuff in the browser. Plus, it made me think about DSLs in general which is good because of all the Scala goodness I’ve been messing around with recently. The talk is on InfoQ.

yonkeltron: Thoughts on different types of data

Mon, 05 Oct 2009 18:41:32 +0000

At work, my title is the Data Warehouse Manager. Mostly, I work with numbers. Lots of test scores, checklist data, compositve variables and the like. Numbers are cool and they can often yield plenty of insight. However, numbers are not the only story that can be told! So much of business intelligence right now focuses on natural-language processing (NLP). I once heard someone say that getting to 80% in NLP was almost comically easy but getting to 90% has proven to be amazingly difficult! Obviously, I’m not content to wait around until the singularity (though I do believe it is near) to be able to utilize textual data for informing educational decisions. So then where is the merit in utilizing textual data if we can be only 90% sure of something? Numeric metrics like test scores are perfectly clear and definite, right?

Wrong! Any good educator knows that test scores (especially from a single test) are not the only way to gain additional insight. While the strengths and weaknesses of the standardized-testing model are a different discussion entirely, suffice it to say that the community needs to remember that, when it comes to education, all numbers are fuzzy. The learning process is not discrete, definite or clean-cut so why should we expect a signle number to show singular accuracy with any great fidelity? In fact, we cannot and it is for this reason that an automatic evaluation of textual data might help educators make better decisions. Perhaps an example is in order.

Let us say that every so often in a math class, teachers write a brief one-paragraph essay about each student summing up their competency and general performance. Then let us also say that students are able to write a sefl-evaluation each time their teacher writes something. So after, a period of time, each student has a bit of text in addition to their test scores and general grades. I propose using some sort of automated classification mechanism to evaluate the textual data and produce best guesses as to the nature of the content. Afterall, if there are 25 students in a class and there are 10 classes taught each day, then that’s a lot of evaluation data to be read! So what about using something like a naive bayesian classifier to sort through things?

But wait, doesn’t this just give us another mechanism to boil down the real-world intuition of the educational system? Well, no. If we let go of the idea that these numbers are perfectly discrete (which we know they aren’t), then we can use them to inform our descisions rather than letting the data decide for us. Probabilities are ok so long as we use them correctly. Few people would wear shorts in the winter just because the weatherman said there’s a chance of sun. So how might such a system be used?

Well, the point of a naive classifier is that it needs to be trained. First, teachers would need to assmble a training data set of some sort. In younger grades, this might be quite easy because students have a more limited vocabulary and might not construct sentences with as much nuance as they learn to do in later grades. As for teacher evaluations, there is a somewhat-finite set of words in the vocabulary of an educator to usefully describe educational progress. Relying on the consistency of such a set nomeclature might very well yield consistent results. Though, all of this would need to be tested anyway, right? This is just a thought experiment…

So the system would be able to chew through these evaluations and produce a guess about whether they reflect positively on student learning or not. Perhaps they could even be used to demonstrate growth. Getting this right would take work but it might lead to a useful way to model evaluative-data. Obviously, relying too much on this might very well to a negation in the positive nature of narrative evaluations but it’s worth a try!

Alex Launi: Tictactoe 0.3.2! How did I get to 0.3?

Wed, 30 Sep 2009 04:39:30 +0000

I really have no Idea how I got to 0.3, I think I chose it arbitrarily to start with. Anyway, New release! This release closes all but one of the open bugs in launchpad. The last open bug is to move the code to use the telepathy mission control 5 api, but the python bindings haven’t been updated to make this easy yet so it will have to wait. What does this release bring?

Now we’re netbook friendly! I had initially locked the size at 600×600, which didn’t fit on netbooks with the window border, but now we resize properly, and should automatically fit onto your screen.
Instead of a Square we draw a real X! In light of this change, the winner screen now says “Cross wins” instead of square wins, so translators, please update your translations.
I also changed the color of the game type descriptions to make it easier to read. It should be a darker shade of gray now.

That’s all! I know it’s not much, but it’s a simple game and the resize thing was the only serious issue with 0.3.1. I’ve got 0.3.2 proposed for uploading into Debian, and although it’s too late for Karmic, I’ve got packages building in my ppa, and Lucid will get tictactoe-ng synced from Debian when the time comes.

So here’s a screenshot of a tiny game of tictactoe!

AndyML: how to REALLY speed up FreePBX

Tue, 15 Sep 2009 16:31:14 +0000

This is a real winner. Philippe from FreePBX mentioned this in our Open Telephony Training Seminar last year in regards to embedded systems but I’ve seen excellent performance enhancements on all the systems I’ve installed it in. yum install php-eaccelerator That’s it! You can do a before and after comparison by setting “DEVEL=yes” in your /etc/amportal.conf and [...]

AndyML: HowTo: Avoid ‘losing’ shared folders on iSCSI targets on reboot.

Tue, 15 Sep 2009 16:17:38 +0000

We have a Windows 2003 fileserver that uses storage on an iSCSI array that seemed to “lose” it’s shares every time it rebooted. I suspected it was trying to initialize teh shares before the iSCSI target was connected and not finding them, so went looking for a way to force Windows to wait for the [...]

AndyML: Making Linux hosts play nice with Windows DHCP/DNS

Thu, 10 Sep 2009 15:40:43 +0000

Sure is nice how windows hosts automatically update their DNS records in your AD integrated DNS servers, huh? Sure would be nice if your Linux hosts did the same thing, huh? This is actually easier than it sounds. First make sure your (Microsoft) DHCP server is configured to “Dynamically update DNS for hosts that do not request [...]

mjd: You think you're All That, but you're not!

Thu, 10 Sep 2009 03:25:00 +0000

I have long been interested in term rewriting systems, and one of my long-term goals is to implement the Knuth-Bendix completion algorithm, described by Knuth and Bendix in their famous paper "Word Problems in Universal Algebras". This paper grabbed my attention around 1988; I found it in an anthology edited by John Leech (of Leech lattice fame) that I was probably looking into because it also contained an enumeration by J.H. Conway of all knots with at most eleven crossings. I found the Knuth-Bendix paper very hard to read, but the examples at the end were extremely compelling. I still find the paper very hard to read, but fortunately better explanations are now available. (For example, this one by A.J.J. Dick.) One of the also-ran topics for Higher-Order Perl was a structured drawing system based on Wm Leler's "Bertrand" term-rewriting system.

So I was delighted to discover that there was a new book out called Term Rewriting and All That. The "...and All That" suffix is a reference to the tongue-in-cheek classic of British history, 1066 and All That, and promised a casual, accessible, and possibly humorous treatment.

Unfortunately the promise was not kept. The book is very good, but it is not casual or humorous. Nor is it especially accessible. It is a solid slab of term rewriting, one of those books that make me think "I would not want to drop it on my foot." That is not a bad thing; the Barendregt book is superb, an enormous superb slab of lambda calculus that you would not want to drop on your foot. But it is not titled "Lambda Calculus and All That".

So why the title? I don't know. The authors are Germans, so perhaps they don't understand the joke?

[ Addendum: There's exactly one review of this book on Amazon, and it says the same thing I do. It begins: "My main criticism of this book is its title." ]

yonkeltron: Podcasts I now listen to

Mon, 07 Sep 2009 00:47:44 +0000

I commute to work. Unfortunately, there is no way to carpool so I end up driving about 80 miles a day alone. Still, I drive a very fuel-efficient Honda Fit and try not to feel too guilty about the fuel consumption considering that I use very little. So, with all that time spent on the Merritt, what should I be listening to in the car? Well, I now have a few podcasts that I listen to.

First and foremost, I enjoy the LKML Summary podcast put out by Jon Masters. He does an exceptional job of summarizing things and his side commentary is usually rather hilarious, actually. I do wish he’d provide a touch more background on certain things as it can be hard to jump right into the high-level technical discussion on the mailing list. Masters’ radio-compatible voice is a pleasure to listen to in the car, even if each episode is only just under nine minutes long.

Next, I have Software Engineering Radio, which has some really neat discussions. The latest one is on APIs and then I’m going to listen to the one where they interview Odersky. The commentary on topics is quite lucid and I greatly appreciate the candor of guests when discussing specific issues. After all is said and done, the software community at large can be a political group (myself included) so it’s nice to have some really great technical discussions as well.

So far, that’s it. I want to also sample the OSNews podcast but haven’t gotten around to it yet. So with about 2 hours in the car every day, anyone know of anything else I should be listening to now that LugRadio is over? I’d be interested in some podcasts on the topics of emacs, Debian or Scala stuff.

Alex Launi: Dont tie your tubes, telepathy can use them for mega fun

Thu, 27 Aug 2009 01:44:31 +0000

This weekend I was supposed to go to Richmond, VA for Best friends day 2009 with a couple of my buddies. I got sick and having just gotten pneumonia not long ago I didn’t want to chance camping in a hurricane while already sick; so I lamed out and stayed in bed all weekend drinking tea, eating soup, and trying to get better. I’m happy to say I’m back on my bike, and feeling pretty good, but while I was sick (plus another day or two) I decided that it’s too hard to play games online with your friends. My girlfriend and I were trying to play online scrabble one night since we were both at home sick, and after 30 minutes of fail just called it quits and went to bed.

Normal people would just say ‘oh well’ at this and let it be- but I thought wtf, I’m a foss developer, I’m just gonna write my own. Well I didn’t write scabble (yet…) but I did manage to hack up a pretty good tic tac toe game using cairo, and telepathy tubes for two player action. Expect more multi-player games from me when I get sick, telepathy tubes are so great, and pretty simple to use. I used the telepathy-python bindings and hacked this together in just a couple of days (really, I do mean hacked. The code is quite ugly but seems to work well). All of the basic functionality was there the first day, but then I wanted to make it a little bit nicer and package it and everything so it took a bit longer. Anyway, now to the cool part, screenshots!

Play the computer, or against friends.

Who would you like to play with?

I win

source can be found at lp:tictactoe, and jaunty/karmic packages are in my ppa https://launchpad.net/~alexlauni/+archive/ppa. I’m still not sure if I’m going to do intrepid, I think the other deps are too old. Feel free to file bugs, make patches, etc. I plan on maintaining it for what it’s worth.

Alex Launi: tangerine 0.3.2.2! The one that probably works!

Sun, 16 Aug 2009 15:57:16 +0000

Photo by Just A Slice (CC-BY-NC-ND)

Inspired by an armless, homeless, West Philadelphia man who rides his mountain bike down south 48th street periodically, I give you Tangerine 0.3.2.2!! 0.3.2.2 is entirely a bug fix release, with 8 kind of serious build bugs fixed. At this point everything should work out ok, all of the optional deps are actually optional, and no symlinking the banshee indexer library should be needed! I’ve learned quite a big about autotools with these past couple releases, so future releases (0.4) should have fewer point releases due to dumb build issues. Check it out on launchpad! https://launchpad.net/tangerine/0.3/0.3.2.2

Now onto 0.4! 0.4 is looking to get the plugins ported to Mono.Addins, a smarter, more robust logging system, become fully translatable, and will get a total change of infrastructure to a service based model similar to Banshee or GNOME Do.

It’s likely that 0.4 will no longer be cross platform as I don’t have windows or a mac, but the point of the service based architecture is to make it so that being cross platform will be more maintainable, and not a series of preprocessor #if directives. If you’d like to help out, PLEASE come hang out in #tangerine on GIMPnet, branch tangerine from launchpad (lp:tangerine), file bugs, register blueprints, etc., etc.

Keep sharing!

Alex Launi: Whoops. Heres Tangerine 0.3.2.1! Now with a screenshot!

Mon, 10 Aug 2009 04:45:41 +0000

About 24 hours ago I released 0.3.1. I quickly realized I had forgotten something very important. I didn’t set up the GUI to build. This is now fixed and I’ve released 0.3.2.1! You can join the party at https://launchpad.net/tangerine/0.3/0.3.2.1. I don’t really recommend using 0.3.1, although there are no code changes, one has a config gui and the other you have to write the config file yourself. If you’re wondering why this is 0.3.2.1 and not just 0.3.2, well I did a 0.3.2, but then realized I had also forgotten to include a sample conf. So 0.3.2.1 has the gui and a sample conf for you. Also now the exec paths are correct on 64 bit thanks to my first patch from Paul Lange. Enjoy!

Alex Launi: I know you feel lied to from those other resurrections, but I swear this one is real. Announcing Tangerine 0.3.1

Sat, 08 Aug 2009 18:06:30 +0000

Sharing is fun. When you’re really little you want everything for yourself, but quickly you learn that playing is more fun when you share what you’ve got. Now we’re all grown up, we know that sharing is caring, but still we keep our media collections under lock and key. Let it out!

Tangerine is a sharing program for your music. Tangerine allows you to share your media with people nearby over the DAAP protocol. Tangerine hooks into your media player, or watches your file system to share your collection, with the tags and info you’ve given it. Your media- shared.

Tangerine isn’t new; Tangerine has been around for a little bit but unfortunately its development has stagnated. I love sharing, and I wanted to be able to share media from Banshee, so I talked to Snorp, and picked up development. Tangerine 0.3.1’s only major “feature” is a ramped up Banshee plugin that will share your Banshee media collection, but if you ask me, the face that I’ve moved development to Launchpad hosted bzr (lp:tangerine), bug tracker, and projectmanagement is a feature in and of itself. You can find all of the fun new project stuff at https://launchpad.net/tangerine.

But now the exciting part! We’ve got a new release!! Tangerine 0.3.1!! https://launchpad.net/tangerine/0.3/0.3.1. The release tarball is linked from there, as is the md5sum. I haven’t built packages yet, but they will be available from the tangerine-developers ppa in a couple of days.

0.3.1 isn’t a big release, but we’ve got big plans. If you’re interested take a look at the planned releases, and help us hack up those features, plus that one you think would make Tangerine a killer. We want to hear from you about bugs, wanted features, how much you love it, anything. We’re on GIMPnet #tangerine, or our mailing list tangerine-users@lists.launchpad.net. Come chat, and share.

yonkeltron: Getting somewhere with Scala

Fri, 31 Jul 2009 21:47:44 +0000

Ok, so when beginning with Scala, it was hard to figure certain things out. For example, what’s the best way to document my code? How should I go about testing my new programs? Which race in StarCraft best represents the ideals of Scala? Well, to prevent other people from getting stuck on these, I am writing this post to help out.

Writing code

How best to write code? Are there IDEs which support or maybe even encourage Scala? Yes! Yes, there are!

IDE plugins (Eclipe, NetBeans, IntelliJ)
Emacs – and a post on how to put many Scala pieces together with Emacs.

Project management

How best to keep track of Scala projects? Well, everyone’s got their favorite way to build stuff (Make and cousins) and those will most likely work fine. However, if you want something a little more specialized there are several tools which can help.

Testing

There is no one way to test Scala, and this is true of most langauges. However, Scala does include a test framework called SUnit. That being said, it’s slated for deprecation as soon as it can be replaced and removed. Also, the current thinking out there seems to be that SUnit sucks. So, you know what, kids? We aren’t going to let that bother us because there are many great frameworks out there which may be used to effectively test Scala. Since Scala compiles to Java bytecode, there shouldn’t be trouble testing Scala from Java or Java from Scala.

ScalaTest
Specs
ScalaCheck (pseudo-clone of Haskell’s QuickCheck)
JUnit – Oldie-but-goodie still works well from Java or from Scala (helpful list of assertions)

Documentation

Who doesn’t love documentation? I know that I sure do! To help satisfy my deep-seated urge to author code-centric exposition, there is scaladoc. Ever hear of javadoc? Yeah, it’s like that. Read about javadoc here and then check out the scaladoc manpage. If, for some reason you don’t like scaladoc’s output, you’re in luck because there is an alternative generator called vscaladoc.

Notes

This list is most likely incomplete and certainly not exhaustive. If someone would like me to add something, please provide the link and info in the comments.

JonathanD: Mailing List Spammers

Wed, 29 Jul 2009 10:59:32 +0000

Yesterday I received some spam. Just another day on the internet, right? Well, this spam was special.

yonkeltron: Talks I am willing to give

Mon, 20 Jul 2009 13:29:22 +0000

The following are possible talks I am willing to give at a future SCOSUG meeting.

Emacs concurrency (or lack thereof)
XSLT for fun and profit
Programming with distributed objects in Ruby
Unit testing JavaScript

That is all.

AndyML: Auto-Provisioning Linksys SPA phones for Asterisk

Thu, 16 Jul 2009 12:15:52 +0000

Information on the web regarding automatic provisioning of these phones is limited at best, and the SPA Configuration Profile Compiler is no longer available for free, so I am sharing my experience provisioning a few of these phones in our office. I have two Linksys IP Phones, model SPA962. we are using AsteriskNOW 1.5. I will [...]

yonkeltron: The trouble with USE

Wed, 15 Jul 2009 18:26:43 +0000

Today, I was caught a little off-guard by a question regarding the USE directive in SQL. While, it stumped me, it did give me pause to think. First and foremost, it occurred to me that part of the reason that I was caught off-guard was because, in the past, I’ve been advised to stay away from USE or similar directives in any language for a variety of reasons. For example, in languages like Ada, a use directive is a perfect way to completely clobber a namespace. Regarding the usage of JavaScript’s equivalent with operator, guru Douglas Crockford has dire warnings about actually writing programs which use it. Note, that the using directive from C#, is equivalent to an import in other languages (Python, Java) or even a require (Lisp, Ruby).

The perils of clobbering namespaces for the sake of convenience in any language are well know. Now, in some languages, a use directive is not always the culprit when namespaces get clobbered. Languages which also have an as clause can be dangerous as well because they obscure the true origin or name of a given object.

Part of the reason is that using USE or one of it’s friends is dangerous due to the fact that programmers risk forgetting that they used it. This kind of error is not restricted to the USE statement, however. For example, in Lisp, global variables are typically enclosed in asterisks by convention. So what, right? Well, early on in Practical Common Lisp, Peter Seibel points out that redirecting something like *standard-output* to point to another source, it would be all too easy to forget to put it back! Just like it’s easy to screw up redirection by forgetting to put it back, USE in most languages makes it quite easy to make trivial mistakes.

Now, I am not suggesting that these examples are directly equivalent. I am, however, making the point that, in this day and age, we are well aware of how easy it is to make careless mistakes and USE directives only make it easier. Say we have an example instance of an SQL server and there are a couple of databases in there. We want to work with two of them (called panda and bamboo) each but we’ve got quite a lot to do with one. For an example such as this, databases like MySQL, provide USE only to be compatible with Sybase and even then only as a utility function.

In any case, we’re going back to the example. We’re at the prompt and we’re going to work with with our first database, panda. So to save ourselves trouble, we go forth with a USE panda; and then go about our business. Then, we’re ready to go ahead and delete a whole bunch of records from bamboo so we run our DELETE * FROM trees; and all should be well, right? WRONG! We forgot to switch to the right database by using USE bamboo; so now we’re in trouble. While this is a simplistic example, the world would be hard-pressed to find a competent programmer who’s never been tripped up by the most simple of errors.

It is for this reason that I am wary of USE and it’s friends. Unless there is a very good reason to USE USE, I suggest that programmers be very careful AS well AS when we USE AS. Together, they can be a tricky combo.

Alex Launi: Pete, Pete, meet math rock

Sun, 12 Jul 2009 17:26:04 +0000

Just found this awesome video on Tera Melos’ myspace – Tera Melos doing a Polaris cover! This is epic to say the least.

and of course for the places where the flash embed doesn’t work, here’s a link http://www.youtube.com/watch?v=oMt8HtDG8No&feature=player_embedded

Alex Launi: memeing it up

Sat, 11 Jul 2009 22:32:39 +0000

I wasn’t at GCDS, so I didn’t see Stallman’s speech, and while I’ve heard differing reports of its severity, I’m not commenting on Stallman’s speech. This is bigger than one irrelevant pseudo-celebrity’s ignorant comments.

I want the [...] open source [...] communities [I participate in] to be a dignified, respectful, inclusive, and welcoming place. … We’ve all been witnesses to off-color jokes, misogynistic back channel chatter, questionable imagery and unnecessary, trolling comments. I pledge to do better to stand up and call this behavior out when I see it in conferences, online and other public settings. I don’t expect it to go away but I’m not going to tacitly condone it any longer.

Alex Launi

Sat, 11 Jul 2009 17:23:28 +0000

Alex Launi: Does your philosophy influence the way you code: A worthless braindump

Fri, 10 Jul 2009 22:19:30 +0000

Tonight while goofing off I had an interesting thought about why some of us prefer one type of programming language, and some of us like another, and why some people are fence walkers and do it all. I propose the following-

People like me, people who like rigid, well defined, scientific philosophy, a la Immanuel Kant, prefer the strict typing, cold hard symmetry and stability of blocks wrapped in braces, don’t like lots of what seems like magic prefer languages like C, C#, and, Java. Consider this block of code, does it make you warm?
public class Bruced { ... ... public override void ToString () { throw new NotImplementedException ("\nyo dawg, we heard you like exceptions\n" + "so we put exceptions in your exceptions\n" + "so you can try while you catch!\n"); } ... }

Mmm.. Just like mom used to make. With the big chunks of chocolate and everything.

Now what about the other type of people? They like questions as answers to questions. They’re busy talking about why there’s nothing to talk about and how we’re not really alive we’re just figments and points in an N-dimensional universe where N is some complex system.
def show_path(anode)

#shows the path back from node ‘anode’ by following the parent pointer

curr=anode

pathmap=@costmap.clone

while curr.parent do

pathmap[curr.y][curr.x]=’*’

curr=curr.parent

end

pathmap[curr.y][curr.x]=’*’

pathstr=”\n”

pathmap.each_index do |row|

pathmap[row].each_index do |col|

pathstr<<”|#{pathmap[row][col]}”

end

pathstr<<”|\n”

end

pathstr

end
WHAT???? I know less now than when I started. Yet some people zen out on this jawn. Just something meaningless to think about.

yonkeltron: Can I get a little MapReduce from my Debian people?

Sun, 05 Jul 2009 20:13:17 +0000

Debian is a world-class Linux distribution. It is used on it’s own for so many applications (desktop, laptop, workstation, handeld, server, etc.) as well as the foundation for so many wonderful projects ((U|K|X)buntu, Maemo, etc.). Personally, I run Debian on my laptop as well as my servers. In fact, when I went to see about setting up a little ad-hoc cluster, I was rather disappointed. Though there are a few clustering tools available, as well as several distributed filesystems (GFS, GlusterFS, OCFS2, and Lustre), shockingly, I could not find any implementation of MapReduce available in the Debian repositories.

For those who might not know, MapReduce is a novel data-processing system developed by Google for internal usage and described in their publication entitled MapReduce: Simplified Data Processing on Large Clusters. For the enlightened out there, it should be clear that the name and mechanism are derived from Lisp’s map and reduce functions. In any case, though Google’s implementation is proprietary, there have been several implementations based on their paper both written in and geared toward a variety of programming languages. Unfortunately, none of these are available in the Debian repositories. In all fairness, Debian does include CouchDB which uses map and reduce functions for generating views. However, it’s not a solution aimed at sorting and processing huge amounts of data, though it is an interesting and capable piece of software.

So, to try and get things moving, I have filed three Debian RFPs (Request For Package) for a few seperate MapReduce implementations.

Hadoop – Probably the most well-known of the Free/Open Source implementations. Includes a distributed filesystem (HDFS), scaleable distributed database (HBase) and tools to get you going from start to finish. Hadoop is written in Java though it can interoperate with other languages (Scala, too). It’s a top-level project of the Apache Software Foundation and licensed under the Apache License 2.0 – http://hadoop.apache.org
Skynet – A MapReduce implementation written in Ruby. It’s designed to be fault-tolerant and distrubuted, just like the big boys. Originally written for use at Geni.com and licensed under the MIT License – http://skynet.rubyforge.org/
Disco – Though the implementation is itself written in Erlang, thus providing excellent distributed fault-tolerance, Disco jobs can be written in Python. It was developed as an in-house tool for rapid data analysis at Nokia and they seem to be quite keen on it. Disco is licensed under a modified BSD License. Page at http://discoproject.org/ and code at http://github.com/tuulos/disco/tree/master

Ok, there might be a few objections to my choices. Why did I leave out neat projects like GridGain, FileMap and BashReduce? Well, for starters, GridGain is another Java implementation that doesn’t seem (at least to me) to have the same momentum Hadoop does. FileMap and BashReduce, while novel, useful and fascinating, are not designed for use in networked environments and are therefore unsuitable for cluster situations. So then whey not MapSharp? Well, primarily because of all the Debian Mono debates going on right now (Gnome’s fail!) . I’ve done work in C# and it’s got some neat features but cool stuff doesn’t and will not ensure that users are not liable from patent litigation.

Also, it seems like those RFPs have some mistakes, so if anyone figures out how to edit them, let me know so I can clean them up.

yonkeltron: Languages, platforms, paradigms and speed

Fri, 03 Jul 2009 02:13:15 +0000

Ever since the latest round of Ruby benchmarks came out and everyone got all excited, I got to thinking about the overall discussion about languages and the interpreted vs. compiled debate. To be fair, there will always be those who take a specific side for some small-but-important-to-them reason yet this has not stopped so many projects from bridging the gap, albeit with varying degrees of success. In many instances, it comes down to the different approaches taken by various language themselves and the payoffs they offer.

In my investigation, I came across some very enlightening sources of information on the overall discussion of language speed, code optimization and the tension between different paradigms. Please peruse the following:

Compiling and Optimizing Scripting Languages – Google Tech Talk by one of the maintainers of phc, the PHP compiler.
Compiling Dynamic Languages – Another Google Tech Talk focusing on optimization of Python.
What To Know Before Debating Type Systems – From the neo-classics department, this essay by Chris Smith helped me gain a great deal of perspective with respect to type systems
LLVM stuff – Google Tech Talk about LLVM 2.0, tons of LLVM-related papers and presentations
A formally verified compiler back-end – Paper by Xavier Leroy that was too long for me to finish

Something missing from this list?

JonathanD: Southeast Linuxfest: We're back.

Fri, 26 Jun 2009 14:18:55 +0000

Got back from SELF about 2 weeks ago. It was by far the best FOSS event I've been to, not that I've been to that many.
It bugs me that there aren't very many good ones. Ohio linuxfest is well spoken of, but it's pretty far, not that SELF was any better.

Alex Launi: Jolicloud + Do bring the awesome

Wed, 24 Jun 2009 16:36:31 +0000

I’ve yet to really blog about what I’ve been doing this summer, not because it’s been top secret, mostly because I’m lazy and have been incredibly busy. I started off at UDS in Barcelona, thanks to Canonical. UDS was a blast, we got a lot done, and I can’t wait to see how Karmic turns out.

So what have I been up to since UDS? Well, I got stuck in Barcelona for a few days more than I had expected, which although expensive, was a lot of fun. I met a bunch of cool dudes from Canada (If you read this get in touch! Haven’t been able to find you guys on facebook!), and some other sweet people. Good stuff. Since then I’ve been in Paris, where I’m working for Jolicloud, where I’m putting my self-described usability expertise to good use. I’ve learned a bit of ruby, done some C, some Python, but mostly I’ve been working on making Do kick absolute ass on JoliCloud. Want to see what I’ve been up to? Check out this short screencast I made demonstrating the awesome. This stuff is a work in progress, and although I’m moving quickly some of this is brand new so expect to see some changes before the final release.

Or, if you’re getting this via a feed reader, here’s the link to YouTube.

Planet PLUG

yonkeltron: Rails + Merb = Rails 3

Alex Launi: March record scores!

Alex Launi: gudev-sharp is born!

mikegrb: Task Management with Hiveminder and Perl

My Weekly Workflow

Conclusion

yonkeltron: Enthusiasts talk shop casually

Alex Launi: Docky has been removed from Do!

Alex Launi: Anything but the buttons, or how I learned to stop clicking and love Do

Paul Snyder: An Emacs Bisection

Paul Snyder: Extract n largest values from a Python dict

yonkeltron: Neato article about NoSQL on Ars Technica

JonathanD: Lower Merion laptop spying

JonathanD: fosscon continues, building a workshop, and more.

Alex Launi: Making my bed

Paul Snyder: Eliminating a nasty, loud ‘crack’ before playback in Ubuntu Karmic

Alex Launi: Jan-Feb 2010 Record scores

JonathanD: We're having a fosscon!

yonkeltron: The zen of backup theory

Paul Snyder: Moonlight just works

AndyML: 24 hours with the Google Nexus One

mjd: A monad for probability and provenance

AndyML: How to: Mount an ISO image under Linux

AndyML: How can I disable the User Account Control (UAC) feature on my Windows Vista computer?

AndyML: Hack your old wireless router into switch-only mode.

mjd: A short bibliography of probability monads

yonkeltron: Cool Free/Open Source Software from Google

mjd: Gray code at the pediatrician's office

yonkeltron: ES5 is finally here, JavaScript geeks rejoice!

mjd: Monads are like burritos

mjd: World's shortest explanation of Gödel's theorem

mjd: Another short explanation of Gödel's theorem

mjd: On failing open

mjd: Dijkstra was not insane

Some additional notes

Further reading

yonkeltron: Messing with OpenStreetMap

yonkeltron: Data visualization on a web page

JonathanD: CPOSC

yonkeltron: Good talk about DSL construction in JS

yonkeltron: Thoughts on different types of data

Alex Launi: Tictactoe 0.3.2! How did I get to 0.3?

AndyML: how to REALLY speed up FreePBX

AndyML: HowTo: Avoid ‘losing’ shared folders on iSCSI targets on reboot.

AndyML: Making Linux hosts play nice with Windows DHCP/DNS

mjd: You think you're All That, but you're not!

yonkeltron: Podcasts I now listen to

Alex Launi: Dont tie your tubes, telepathy can use them for mega fun

Alex Launi: tangerine 0.3.2.2! The one that probably works!

Alex Launi: Whoops. Heres Tangerine 0.3.2.1! Now with a screenshot!

Alex Launi: I know you feel lied to from those other resurrections, but I swear this one is real. Announcing Tangerine 0.3.1

yonkeltron: Getting somewhere with Scala

JonathanD: Mailing List Spammers

yonkeltron: Talks I am willing to give

AndyML: Auto-Provisioning Linksys SPA phones for Asterisk

yonkeltron: The trouble with USE

Alex Launi: Pete, Pete, meet math rock

Alex Launi: memeing it up

Alex Launi

Alex Launi: Does your philosophy influence the way you code: A worthless braindump

yonkeltron: Can I get a little MapReduce from my Debian people?

yonkeltron: Languages, platforms, paradigms and speed

JonathanD: Southeast Linuxfest: We're back.

Alex Launi: Jolicloud + Do bring the awesome