Mes Souhaites

I have abandoned livejournal to run things myself at hoenir.himinbi.org.

I regret the loss of the privacy protections, but wanted to explore the medium a bit more than livejournal would let me.

If you're interested in still keeping track of me (though I'm currently being a bit more professional in my writing), I'd recommend using a RSS feed reader. I'm using Google Reader to follow about two dozen blogs (including all my livejournal friends) and it does a good job of letting me manage everything.

I've been contemplating and discussing the idea of a pervasive recommender system. I'm find most people find it creepy moreso than intriguing. I don't know if it would always have been like this, but particularly in this modern age of corporate malfeasance and secret government torture facilities people just don't trust organizations to act responsibly.

I'll be honest that I don't necessarily like it all that much myself. It kinda makes me skirmish to guess how much Google knows about me.

Privacy has been called the ability to pretend to be like everyone else. As a researcher I want as much raw data as possible. What steps can be taken to make people feel comfortable?

Two main things come to mind:

1. Allow a complete separation of a person's recreational and professional personas. If I'm a respected writer by day and dress up in a bunny suit for sexual pleasure at night, privacy is the capacity to keep those two lives separate.
2. Give the user complete control over their data. At any time if I can see the specific information I have entered into the system from any source and I should be able to modify or at least delete it. If there are derived attributes that is the organizations discretion to disclose. Some dating sites create an emotional stability score. They don't tell members because having a computer tell you you're unstable is slightly insulting to many.

As I was preparing the last post I looked up a link for my Chase Freedom credit card.

The basic idea is that you get the usual 1% cash back on most stuff, and for you top three spending categories you get a bonus 3%. In their description they say: "3% at the top 3 types of places you spend the most each month. There are 15 categories, more than any other card!"

I wonder what percentage of people will realize that the larger the number of categories the smaller the number of things in each division will be. That exclamation point threw me for a minute. It seemed like they were excited about it so surely I should be as well. I think they're just excited because they now have to give me less money. ☺

I have a Chase Freedom credit card. I use it to buy most of my stuff and the balance is automatically paid in whole from my HSBC Direct savings account. I get a bit of cashback and it keeps a larger balance in my savings account. Both of these are negligible bits of money, but who am I do turn down free money?

The one thing that has been irritating me is the huge amount of crap I get in the mail from Chase. They like those promotions which say, "Here's a check for $20 just for being you (btw, cashing this check enrolls you in our $15/month identity protection program)."

I sent them a message today saying, "Please leave me alone Chase. I'm tired of having to destroy all the things you send me that are much more attractive to thieves than they are to me."

They didn't entirely agree to it, but they said they'd leave me alone for five years or until I changed my address. Not perfect, but I reckon I'll take what I can get.

So, on blogging, I was reading an post by Kurt Jacobson on avoiding mob rule in determining music quality. I have thoughts on the issue and want to have a conversation. I don't actually know the guy, so through the medium of blogging we just both shout our ideas out into the ether.

The basic idea he's hitting on is social proof. It's essentially the fact (and I say fact because there's enough research to make it pretty much indisputable) that human perception and preference is highly malleable, and one of the biggest influences is what they see other people doing.

In music recommendation this played out in a recent study on recommendations in an artificial culture. One group was given a bunch of songs and asked to rank them. Another group was given the same task, but they were given feedback on how much other people liked the same songs.

The results from the first group were fairly consistent. This suggests that there are some inherent qualities of songs that are consistent across listeners. (The test was conducted on a "teen-interest" site, so there was probably some perceptual preference similarity among samples.) With the introduction of social influence, scores became much more erratic and predictability dropped.

Jacobson's argument is essentially Britney Spears is crap and if we let the mob decide what good music is, we'll have a sort of dysgenic collapse as the masses all follow each other into a collective mediocracy.

There's certainly something to that argument. The consistency in preferences without social influence says that there is some sort of inherent quality to music. (Whether this quality would hold up under a more diverse sample population is debatable.) If I am wanting to recommend a song to someone who has none of the social feedback, I am going to want to use the consistent sample set because I can predict with less error that they're going to like the song.

The idea though that technology should somehow try to correct for humanity's herding instinct is philosophically dangerous and ultimately counter productive. Most people don't want you to correct for their weaknesses. They want you to give them what they want. In fact, the bulk of people would find the technologist's assertion that they need their preferences accommodated for slightly insulting.

One of the things I would have liked to see in the study was means for the data. They put data in terms of market share which is interesting, but I would like to know if the average rating was higher. There was the recent study on how a $2.50 placebo is 25% more effective than a $.10 one. I wouldn't be surprised if the user enjoyment of music with high community approval was higher than when the song was heard in isolation. It isn't just that people take cues from their environment, but that their actual experience is different.

So instead of trying to somehow fight against social proof, instead use it and try to introduce systemic constraints to encourage novelty.

Imagine a site where people nominate songs for song of the day. A song of the day can never have been a song of the day before. (Or to have more variety, disallow the top 10 each day from future consideration.) The criteria for deciding the song for the day is based on votes from all the users. There's some sort of mechanism where the more often a user is correct the more their opinion counts. I'd have to think on the details, but you essentially develop experts within a particular genre whose natural preference profiles are very similar to a stereotype. This would bias the results toward reproducibility.

The details would need some thinking, but the basic idea is you want to use social proof to increase a user's enjoyment while somehow encouraging the introduction of novel ideas. There's already a hugely implemented system dealing with this exact problem: the stock market. You want to find a resource before anyone else knows about it (find novel music) and after you've invested in it, you want to see it perform as well as possible (have as many people rank it as possible). I think a site based around that concept would be interesting.

I don't disagree that there is such a thing as great art. I'm just not convinced that computers are capable of recognizing that on their own. The best that we can do is set things up so that hopefully the human capability to recognize beauty is captured as much as possible and leveraged.

So, I'm lining up summer plans and looking to do something music informatics.

One of the front runners for places I would like to end up is at Sun interning with Project Aura (which is a meld of Sun's Search Inside the Music and Advanced Search Technology).

I had a phone interview earlier this week and one of the questions was the specific areas I'm interested in investigating. I attempted a literatue survey and discovered the field of MIR is large and active. As I was contemplating the question of the most interesting parts last night, I remembered a list of MIR research blogs.

I'm realizing the extent to which the internet is growing to play a role in certain areas of research. My adviser was saying it can take up to a year to get a paper from submission to publication. Blogs in particular let a community of researchers keep up with each other. From reading the MIR blogs, I can't imagine a researcher really being current in the field without paying at least a little bit of attention to them.

I've been wanting to set up a feed aggregator. I figured why not use this as an opportunity to deepen my relationship with my gracious electronic overlord. So far I've been pretty happy with Google Reader. I understand blogging a bit better both as a community building tool, but also as a medium for communication.

I have a bit on blogs, MIR and technologists, but I'm going to split it out into another entry…

This is just a brief note to any command line users such as myself to remember the handiness of the little utility dc.

I was wandering the web and came across an estimate of basal metabolism. Now I know such thinks are affected by fitness level and whatnot, but I wanted to know quickly what the result was and I wanted to see all the numbers at once so I could know if I was copying them correctly. On the command line, this is just:

echo 195 2.2 / 11.6 \* 879 + p | dc

Where 195 is my weight, 2.2 is the pounds in a kilogram and 11.6 and 879 are the line parameters.

As I look at this I start to recognize that most people would find calculator more intuitive. ☺ This is for the geekier among you.

Recent years have seen an increasing amount of interest in music recommendations. Music is being created and published at a rate that far outstrips the ability of any listener to keep up with. How then, if I'm an artist, can I get my awesome new song to the ears of listeners? Or, if I'm a music connoisseur, how can I sort through the thousands of songs to find the few I'll love?

The oldest recommendation system is still going strong: word of mouth. "Dude, you've got to hear this…" will always be a vector for music to spread. Another old-school system is one where I pick a bunch of songs I like and send them out over the airways to anyone who would like to hear. The internet is changing the ballgame, however. The advent of computers, in addition to fundamentally altering how people create and distribute music, also promises to change how they discover it.

Some of the earliest systems were what are known as "collaborative filters." A computer system, like Amazon, knows thousands of albums that various people have purchased. Assuming that people liked the albums they bought, if Bob and I both bought ten of the same albums, when Bob buys an 11^th, there's a better than average chance that I'll like that album as well.

Systems like Pandora work in a similar way, but not only have they broken things down from albums to individual songs, they also look at specific characteristics of songs. So, now the system can say, not only, "people who liked the songs you like also liked…," but also, "you have liked 9 of 10 R&B songs I played for you with melodic tones and three male singers, here's another song that sounds like those."

Pandora's knowledge of music comes from the Music Genome Project. Human listeners have spent countless hours reviewing "tens of thousands of artists" and cataloging God-only-knows how many characteristics. That's cool, but not really an undertaking the average person has time for, so there's quite a bit of research into how to get computer programs to listen to the songs for us.

Systems employ a variety of different types of signal analyses, most taking into account human perceptive characteristics (we are more sensitive to high frequency noises than low ones). They not only attempt to determine traditional characteristics such as beats per minute, key, and timing; they also represent "timbral characteristics," like the signal strength 3-4kHz, that human listener would never use to describe music.

There's quite a bit of research in this area including the International Conferences on Music Information Retrieval and Related Activities, the Laboratory for the Recognition and Organization of Speech and Audio at Columbia, and the Music Technology Group at the Universitat Pompeu Fabra. Research is also driven by a growing rapidly group of private commercial interests too numerous to list.

The more interesting developments for me, a solo programmer and graduate student in an unrelated field, are open-source projects. These are areas where I can take part and help create something interesting. One of the more promising is Paul Lamere's Search Inside the Music project with Sun Microsystems.

Which brings us to a crucial issue for every computer program ever written: data. Systems like Last.fm will integrate with your music player so that every song you play goes toward helping the system build a profile of preferences. Accessing actual songs to play with spectral analysis on a large scale is a bit more complex. The options are fourfold:

1. Make a deal with the record companies gaining some sort of license — Unlikely
2. Buy tens of thousands of songs online — I'm poor
3. Use creative commons sources such as Magnatune — Limits selection
4. Steal as much music as the internet will allow — Illegal

Each of these has advantages, but ultimately with all of them I run out of space to store all these songs. Also, part of the point is community building, and the point at which I try to open the system to other people (other than with the creative commons sources), I'm in legal trouble. What would really be nice is if I had an online system that I could query about the characteristics of songs (or a database of info I could download). The tricky part is building that database.

In the last couple weeks, a couple systems have come online (EchoNest & Vienna MIR) that let you send in a song and get back an XML description of the audio characteristics. That's a fine idea, but it limits the songs I can do computations on to songs I have a copy of. It also means I need to transfer whole files over which means if I want to experiment on a 5,000 song set, I need to start about a week early just so I can send all my songs over.

Right now I'm listening to The Nature by Talib Kweli. If someone else has already analyzed this song, I don't need to send the song over. I just need to tell the system I want The Nature and it can send me the data I want. This saves both the time to transfer the song and the time to analyze it.

Now the problem becomes how to tell the system what song I want. For many rap and folk bands there are often myriad versions of a particular song. Lots of new XML based technologies use URIs, and that makes sense, but how to make a unique one for each song? Enter MusicBrainz. They are a huge (more than 6,300,000 tracks and growing) community maintained database of songs, and they've assigned a unique identifier to each one. If I know the MusicBrainz id for a particular song then I can refer unambiguously about it to anyone else.

MusicBrainz has some limitations that would eventually be a problem. Mostly that they are "album-based" meaning they don't deal with DJ mixes or bootlegs. Also so far as a completely open process, they use MusicDNS which is partially encumbered. That would matter eventually, but those concerns are outweighed by the huge amount of data and community support you'd get.

There may well be other good solutions to the data problem. I like the MusicBrainz idea because it gives a project like Search Inside the Music a pre-built community and support structure. It would also draw users to help build MusicBrainz as a resource for both tagging and music recommendation applications.

I went with the_archange1 to see Uncounted a couple nights ago. The thesis of the movie is that modern electronic voting machines are not only open to hacking, but there is extremely strong evidence that voter fraud has been committed in several elections including the 2004 Bush/Kerry presidential election.

Honestly, when are conspiracy theories ever true? Their case looked pretty solid though.

One of the more interesting statistics were anomalies in undervoting. An undervote is when a voter comes to the poll, votes in some races, but expresses no preference in others. This makes sense at times. Yesterday I could easily not have had an opinion on who the Democrat candidate for city council should be.

In presidential elections there tends to be an undervote of 1-3%. In New Mexico in 2004 in certain Native American and Hispanic precincts there were undervotes of up to 80%. 4 out of 5 people went to the poll and just didn't have an opinion on who the President should be. It's particularly interesting that it happened in Democratic strongholds.

There was also an analysis of a comparison between the exit polls and the actual results. I'm actually in a stats class right now and I have a newfound respect for the degree to which studies and statistical analysis have a serious mathematical basis. The issue wasn't simply that the polling data disagreed with the result. There could have been some unseen factor to push error beyond any previous amount of error they've seen. It was that over and over the errors favored Bush.

There's quite a bit more about things like the suspicious proportioning of voting machines that had primarily low-income minorities standing in line to vote in Ohio for up to 12 hours. I'd highly recommend watching the movie if you have a chance.

If you happen to live in Tennessee, you can support SB1363/HB1256 which would remove Tennessee from the 12 states that allow voting machines with no paper trail whatsoever. (With no paper trail the vote count is whatever the machine tells you it is and there is no method for verification. If nothing else, anyone who has dealt with a computer should know better than to do this. Absolutely no programmer writes bug-free code all the time.)

Even if you don't think there's been a conspiracy, there's no reason not to prevent one in the future.

This is complete geekitude, so feel free to disregard…

I want to take a DVD, stick it in the drive and at some point have the smallest file possible that contains as much of the original structure as possible.

There are a few problems with most of how this is done most of the time. When you download a video off the internet there are several parts that go into that file. There's the sound and the video streams and then a container that joins them all together. The container syncs the streams and says, this second of audio goes with this second of video.

The most common container format in use currently is AVI from Microsoft. AVI is a fairly old format and it has several limitations. One of the most important for my purposes is it pairs one video with one audio track. That's fine when you're encoding your home movies, but it can't handle dubbing and director's commentaries for DVDs.

There are a couple new formats to consider, but the one I like the most is Matroska. It's flexible and supports not only the multiple audio streams but also multiple subtitles.

Inside the container there are even more standards at work. The raw data for audio waveforms and video images is huge. Different compressions schemes, called codecs, are used to compress the data and make it take up less space. They operate in a variety of ways, but ultimately the goal is a tradeoff between the degradation of the image or sound quality and the size of the output file.

Audio compression has seen a bit of improvement over the old workhorse MP3, but the real improvements in technology in recent years has been in the realm of video compression. H.264 is able to produce a file that is nearly visually identical to the MPEG-2 used on DVDs in fraction of the space.

There are some programs, notably HandBrake, that do a pretty good job of doing what I want. Before HandBrake was released however, I started on a ripping program of my own. I've been playing with it for a bit and this morning I had a bit of a breakthrough.

When I list the contents of the DVD, it tells me in general that the video is in NTSC. That makes sense, since NTSC is what we use here in America. The encoder was complaining though about dropping frames which shouldn't happen if I was encoding at the right rate.

It turns out that though NTSC is 60000/1001 frames per second and interlaced every other frame NTSC is 30000/1001, movies are filmed at 24000/1001 and that is how they're stored on progressive DVDs. In running my encoding previously at 29.97 and ending up with a file around 1500mb. I switched to 23.976 and the file size dropped by 500mb.

On the subject of grad school, I got the reviews from my teaching assistantship. Once a week I would give a recitation on some subject concerning ethics and computer science. All sorts of stuff from net neutrality to the history of hacking to the development of intellectual property law in America. I really enjoyed it. Unfortunately I also had to grade an essay per week for each of the 35 students and that was less entertaining.

I felt pretty good about the recitations. I'm about as geeky as anyone I know and this was essentially geek story time for an hour. My audience was 30 Malaysian students which was a bit challenging, but over the course of the semester I saw a real improvement in their critical thinking skills and writing proficiency. In the end that's what I really enjoy about teaching.

I did wonder though about what they thought. After I'd stepped up the grading criteria on one of their essays, I asked them if they'd rather if I just saved us all some time, skipped reading the papers and just gave them all B+. All but two said "yes."

I've been wanting to play with Google's Charts API.

How helpful and responsive was the TA?

How was the TA's fairness in grading?

Did the TA communicate effectively?

What is your overall rating of the TA?

After bleeding all over their papers and making them work fairly hard, I'm pretty cool with having 80%+ giving me good or excellent. I'd really like to get to teach a class like this at some point in the future. I would really like to see what could be done with it if it were treated as a persuasive writing and critical thinking class as much as a survey of ethics issues. Many of the students weren't even really in a position to fully appreciate the issues they were introduced to and I think it would have been interesting to try and address that problem.

My first semester as a graduate student has come and gone. It has certainly been a learning process.

So far as the amount of actual information I learned, it was acceptable. I had Human-Computer Interaction which primarily dealt with user interface tests for software development. I'd seen most of that in work before, but there was some bits about doing statistical analyses that were new and interesting.

Statistical Inference was my favorite class. It was also mostly a repeat of information I'd had before, but we had nice little tables instead of all that messy integrating. The different tests were new though and the review was useful. I also managed to get more of my notes types up this time which I figure will be useful in the future.

Artificial Intelligence was drudgery. They combined the grad and undergrad sections since one of the professors was off on sabbatical. The main difference ended up being huge amounts of homework. I'd certainly not claim that I didn't learn anything, but the process was at times a bit frustrating.

All in all though I'm still happy with my decision to start this trip. Grad school in some ways aged me. Having two or three homework assignments that required 30-40 hours to complete over the course of two weeks actively sucked some of the fun out of me. It meant I spent quite a few nights at home while the_archange1 was out and about. I'd like to be able to say that when I had fun it was all the more intense, but honestly it wasn't. The stress and the fatigue made me more boring, but I learned how to get a shitton of work done and that's a skill I've been wanting for a while.

My dinners for the last week:

• Teriyaki pan brazed flank steak with broccoli
• Cayenne chicken medallions with squash
• Grilled flank steak with rice and scallions
• Grilled chicken breast with spinach salad
• Seared salmon stuffed with dill
• Grilled rib-eye with potatoes
• Campfire roasted corn and hot dogs with smores

New roommate loves to cook and is terribly good at it. I just show up each night and there's something tasty waiting for me. It's great.

He wants to join a farm cooperative which will theoretically get us random local food every week, so things should just keep getting better.

The Windows Verification program now has a firefox version. Maybe IE will eventually kick the bucket:

I noticed a neat essay in titivillus's journal: Is There Anything Good About Men? It makes some interesting points about potential cultural and evolutionary reasons for the existence of a patriarchy. It also makes the good point that while many of our leaders are male, a sizable chunk of our criminals are as well.

I think it's a interesting article to read and contemplate. I don't really like the dichotomizing that seems to underlie the whole thing. Gender has been a big issue for me for a while. It started because I was detached and unemotional to the point of feeling dead in my life. As I started trying to understand ways to be more connected in my life and relationships I kept bumping into the stereotype that the things I wanted weren't part of being a real man. It really took me down for a while. Remember that it wasn't that I was having these feelings and was looking to deal with them; I felt completely broken away from my life and I felt at times that people were telling me that there was nothing I could do about it because I have a Y chromosome.

My response to gender at this point is there may be certain cultural or genetic predispositions, but their strength and prevalence is so variable that it's completely useless as a measure of anything about an individual — about like race. So I could be reading an article about how slavery and selective breeding affected the genetic characteristics of Africans in America. It would be interesting and the statistics would be interesting to consider, but I would not let the information affect how I deal with a random black person I meet on the street.

I was thinking about that example. I, and I think most people, have a little voice in my head that actively chastises me if there are any racial stereotypes in my head when I'm talking to someone new. (Though I spent about two hours last night learning to be aware of race — I was watching Chappelle's Show.) I've got a definite socialization against being racist. So far as gender though and making assumptions about a person based on their gender, I dislike it, but it's mostly from my own thinking. Socially I've got some little voices about the value of women's rights, but things about a person's emotionality, aggressiveness, or interests are all fair game.

The bit that I distinctly disliked was the last paragraph. I read it as essentially, "There are many critics of how our society operates. There are a variety of reasons that things have progressed this way and the fitness of this type of society is why it has won out. We should think twice before screwing up something that works."

The "this is the way we've been doing things and it works" is a horrible argument and one neednt look any farther than the pyramids to see accomplishing amazing things is not the only criteria for judging a system. I really do believe that the human animal has a capacity for amazing complexity that most people never even get near because they're too busy chasing the brass ring or pleasing their parents or their church or just playing out some random drama. I don't expect to live in a world that encourages anything else within my lifetime, but I think so far as a goal, it's a good one.

I've been getting my schedule worked out for the next semester. I figure it's best to treat school like a job and show up and work fairly regular hours.

I've been riding in with the_archange1 and that's nice since it puts me here around 8:00. I was listening to an insightful podcast get_brett that recommended at least having productive things to procrastinate on if you're going to be doing it. I figure I'll at least not sleep or watch TV if I'm in my office.

My schedule is filling up rather quickly:

Event	Days	Time	Location
Office Hours	MW	9:30-10:30	Jacobs 357/360
PSY-3100P — Statistical Inference	MWF	11:10-12:25	Hobbs 107
CS-352 — Human-Computer Interaction	TR	9:35-10:50	Stevenson Center 5312
Office Hours	TR	11:00-12:00	Jacobs 357/360
CS-360 — Artificial Intelligence	TR	13:10-14:25	Featheringill 211
Lab Meeting	T	14:30-15:30	Jacobs 360
Withit Lecture	R	15:10-16:10	Featheringill 130
CS-151 Tutorial	T	19:00-20:00	Jacobs 306

I'm still dealing with getting settled here. I'm having trouble seeing how I go from where I am now to a settled adult life and that is a little depressing. I figure if I just keep plugging away either I'll get though it or I'll learn definitely what needs to change. In either case the thing to do at this point is to keep working.

I'm getting ready to start grad school in the fall. I've been e-mailing back and forth with my adviser and discussing potential classes. I'm pretty sure I know what I want to take:

CS-352-1	Human-Computer Interaction	TR	09:35 - 10:50
An overview of human-computer interaction and problems of current interest. Human factors, GOMS, user interface design and evaluation, interaction modalities, distributed cognition, ubiquitous computing. A project involving design and evaluation will be performed.
CS-360-1	Advanced Artificial Intelligence	TR	13:10 - 14:25
Discussion of state of the art and current research issues in heuristic search, knowledge representation, deduction, and reasoning. Related application areas include: planning systems, qualitative reasoning, cognitive models of human memory, user modeling in ICAI, reasoning with uncertainty, knowledge-based system design, and language comprehension.
PSY-3100P-1	Statistical Inference	MWF	11:10 - 12:25
Introductory course designed to familiarize doctoral students with the principles and procedures of statistical inference and to prepare them for more advanced work in research design and analysis.

I had hoped to have only Tuesday/Thursday classes so I could sneak off to Baltimore for long weekends, but that doesn't really look possible. I could take a different CS class to make it work, but the psych statistics class is part of a plan to integrate some more rigorous testing methods into HCI evaluation and I like the idea.

I don't know if it is the exhaustion from working all day or if I'm just in a sappy stretch of shows, but the last three episodes of This American Life that I've been listening to all day have brought me to tears.

It's a really good show. The more recent episodes with the stories from Iraq are particularly interesting.

Between Wayne, my mom and myself took about 2000 pictures over our two weeks in Peru. They were all named random stuff like IMG_1985.jpg. I figured to get them sorted, I'd name them according to their dates. Once I got it done I discovered that Wayne's camera is off by 12 hours. After I did a little research I found that there's a handy Linux utility that's exactly what I need:

jhead -ta-12:00 -model SD500 *jpg

To do the renaming then is:

jhead -n%Y:%m:%d_%H:%M:%S *jpg

Gallery has died again. It's getting kinda irritating since it tends to do so about every month or so. I don't think it is really the gallery program's fault entirely. I suspect that Dreamhost is cutting the processes off early and it isn't handling it well.

It's a pain not only because of the outage but also because I have to reload everything. There's too many pictures though and it chokes. Even after loading them incrementally, I still lose the captions. (Version changes kept me from restoring the database.)

Flickr is cool. I like the geotagging and community bits. I'm tempted to drop the $25 to create an account for a year. (Since it will take a while to upload 10gb of photos 100mb at a time. Though smugmug looks pretty good, but it's $40. There's also Picasa which would be $25 for 6.25gb.

Honestly I'm frustrated since I have hosting. It's kinda sad that there's not a FLOSS offering that works well. I love the idea of trying to write one myself, but who's got the time? Maybe someday.