A Snake and a Plane

Google Appengine + Django

AppenginePython Logo

Getting started on a new project can be tough, especially when you have little to no idea about 80% of the technologies involved. As I enjoy making life difficult for myself, I’ve decided to tackle a new web venture from the top down – in other words, tackling some of the big software questions before writing a single line of code – and I think I’m almost there. Hopefully this can act as a template for future web projects.

The Big Decisions

So here’s a brief rundown of the buzz words I was looking for in the platform:

  • Cheap! (not really a buzz word) – I want this site to be pretty much free to get up and going
  • Scalable – The platform should scale gracefully (both in cost and capacity) with the  amount of traffic
  • Portable – The ability to pack up our toys and go elsewhere
  • Open – In a number of ways: To be able to clearly communicate about the technologies involved without giving away any trade secrets, to leverage as much open source as possible to begin with and hopefully be able to give back to the open source community as much as possible
  • Simple! – Stick to standard technologies so that getting up and running with development is (should be) a breeze
At the end of the day I was won over by Appengine (on pricing, scalability and portability), Django (on openness) and Python (on simplicity). To complement this trio of nerd hipsterness I’ve gone with a Github private repo (some of which will become public) for development, Virtualenv (to keep Python dependencies in check) and most importantly Django-nonrel and the related apps over at allbuttonspressed (to give a portable wrapper over Google Appengine’s Database backend).
For the front end, I’m thinking jQuery and Coffeescript (a nicer and hopefully more familiar syntax for writing Javascript).
If it all goes pear-shaped, at least this post has been deliciously trendy.

Goodbye eMusic

Dear eMusic,

It’s not me, it’s you.

The last 5 years have been fun. You’ve exposed me to bands such as The National, Phosphorescent, Beach House and many more. In my mind, you’ve always been the little guy, the truly independent alternative to monsters such as iTunes. The place where the cool kids can hang out.

I tolerated your little quirks: the download manager that never seemed to work, the distinct lack of bands that anyone had ever heard of, the minefield of “best ofs” that were really just crappy live recordings or (even worse) re-recordings. It seems now that you’ve lost your way and a loyal customer in the process.

Our relationship started going sour after you began courting the Rolling Stones. Then it was Sony, then Warner, then Universal. In all of those deals it seems that your customers and the smaller labels were squeezed out to make room for the big boys. Your “variable pricing” scheme caused prices to rise across the board and in the process the real value of my subscription kept going down down down. Some of your better smaller labels (Domino, Merge and Beggers Group) started to pack up and leave. We’re talking here about bands such as The Arctic Monkeys, Arcade Fire and, sadly and ironically, The National! How much more indie do you want??

In the mean time, as an Australian, I still couldn’t download over half the albums in your top 100 of 2010 – not entirely blaming you here, the record companies are a bunch of protectionist jerks in this department.

The final straw was your massive backwards step in preventing me from re-downloading tracks that I’ve already purchased. I mean, come on! Even iTunes lets me do this.  There is no logical reason to do this. Piracy? Give me a fucking break. And your download manager is still a pile of crap.

So it’s over. I’m going to miss the good bands (most of which have moved to iTunes and Amazon), the great recommendations, the community and those little quirks that reminded me that you were a little site fighting the good fight in the chaos that is online music.

Sincerely,

Me

Music All My Days

I just completed a new song that I’m really excited about called Music All My Days. This song was inspired by my friend Will Udall heading off to London to pursue a career in music. It’s a reflection on why we learn, write and play music and I guess it also doubles as a reassurance to Will about the reasons why he’s leaving. I wish him all the best of luck over there – he is one talented mofo, you should definitely check him out.

P.S the John Lennon shirt was kinda deliberate. Been thinking a lot about him recently.

Music All My Days

There’s only 12 notes in every song that I know,
And every song that I’ve learned has helped me to grow,
Each combination has touched me in a different way,
That’s why I’ll learn music all my days.
 
When you start to write you’re building on a history,
And this grows every day like a beautiful mystery,
When you write a song it doesn’t take another away,
That’s why I’ll write music all my days.
 
Everyone can make it, all you need is life,
That’s why we’ve been doing it since the beginning of time,
Yeah you don’t need money, or a license to play,
That’s why I’ll play music all my days.

Remember when climate change was called global warming?

I’ve had just about enough of this climate change debate. What really caused my attitude to change from hmmm let’s talk about it to let’s just do something about it already was a recent episode of Insight on SBS TV (great show by the way!).

I get that sinking feeling that we’ll still be arguing about this until the oceans have risen up to our necks

Climate change exists. Humans are causing it. Humans (we) can help to slow it down. Fact. Fact. Fact.

I have been convinced of this without a shadow of a doubt ever since I read the release from the Australia Academy of Science – The Science of Climate Change.

Now, we can’t all agree on every single detail. Some of the literature supporting the claims in the report may not be true. Believe it or not, scientists are humans and humans make mistakes. Despite the thorough review process that academic papers must go through, sometimes factually incorrect material slips in. But at the end of the day, the majority of the scientific literature and the opinion of climate scientists point to the same conclusion.

Jumping on a handful of points that are wrong in the massive body of literature out there and using that as an excuse to dismiss the whole theory is ridiculous. It reflects either ignorance, in most cases, or a more sinister motive – which is scary and anyone that is taking part in that practice should be forced to live on a small pacific island near the beach so that they eventually drown in their own jerkiness. The point is, most of the arguments “proving” that climate change doesn’t exist is either legitimate data presented out of context or urban myths gone viral.

If you continue to bully the Earth, eventually it will fight back

So what do we do about it? If there was an easy answer, we would have done it by now and be perched under a rainbow holding hands. There is no optimal solution to this problem and people will suffer while we try to solve it. The Earth itself has been suffering since humans figured out ingenius solutions to all that food, water and shelter business and then got a bit bored and invented things like factories that crap out smoke, mineral processing that leak arsenic and stock markets that crash. As a result, we’ve gotten ourselves into a state where people like me (born and bred in Australia) can sit on our Macbooks blogging about climate change while people in parts of the world that I’ve have never heard of are still struggling to get a hold of that food, water and shelter stuff. And I admit that I still get shirty when a I pay $4.00 for a coffee and it’s too milky (by the way, how hard is it to get a fucking coffee right when that’s your only job).

So we, collectively, have to brace ourselves because the Earth has an uncanny ability to self-correct. That’s right, the Earth can screw us right back. Hard. And not just us. Our children and their children, etc. Until there will be no more children. No one (specifically) is to blame. Not even Tony Abbott. If you really want a head on a stake, go dig up a bunch of people that lived in the era of the steam engine. Then dig up everyone that came after them. You’ll need a whole bunch of stakes.

No more talky talky

Ok so it exists. It’s our (collectively speaking) fault. What the crap do we do now? Most would agree that we have to switch to cleaner, greener forms of energy production.

How to deal with climate change is also something that very smart people have been studying for a long time. The best solution that we have at this exact moment is to put a price on carbon emissions. In the future, if we think of any better ideas we can implement them too. But the time to act is now!

Here’s the basic idea (Carbon pricing 101): We make something that used to be free, emitting carbon into the atmosphere, cost something. That makes prices for things down the chain, e.g. electricity prices, fruit that is shipped around the world because we want a mango during winter, cost comparatively more. Eventually, people change their behaviour towards cleaner, greener alternatives. The money that is raised by pricing carbon goes towards compensating those people that may be totally screwed after the implementation of such a scheme and the remainder goes towards investing in some cool stuff like green technology. This accelerates the growth in said technology and makes it comparatively cheaper in the long run. A rainbow begins to form and I hold out my hand to yours and you grab it and we sit under that rainbow. You can read more about the two approaches to pricing carbon – a fixed carbon tax or an emissions trading scheme (ETS) – elsewhere. All you need to know is that this solution has been predicted to be effective and economically viable. A friggin economist said so!

Now, that’s the theory. In practice, things get hairy. People will lose out. We probably will get the price way wrong and have to adjust it. Scam artists will crop up and take advantage of stupid people. The market for carbon permits, under an ETS, could collapse. Big polluters will spend their last remaining cents on lobbying for lower priced (or free) emissions to “help with the transition”.

Now to pause for a snack of Political Hot Potato: The argument that people who are already on struggle street will be worse off because of an ETS is bogus!  If you’re on struggle street to begin with, an ETS is the least of your worries. Complain loudly if you are actually going to be worse off after the ETS – i.e. the compensation package doesn’t support you enough. In the mean time concentrate your efforts on getting off struggle street, not on trying to predict the future.

All of this is inevitably going to happen. We’re dislodging a behaviour that’s so entrenched in our society that the change is going to feel like having brain surgery through your arse hole. But we have to change, the change has to be drastic and the change has to happen now.

A few token gestures to distract, confuse and ultimately shut you up

The alternatives I’ve heard sound like a load of business-as-usual bollocks. They all fall over under closer scrutiny, they’ve all been thought of before and none of them work. A couple of examples:

  • Subsidise clean and green technologies – Sounds great on paper right? But we already subsidise these technologies. It’s a good thing. If we want to subsidise them some more where do you think the money is going to come from? BP? No. YOU! The taxpayer. Or even worse, the schools, or the hospitals, or the pensioners, or wherever we can scrape it together from (I hear defence has a bit of cash to splash around). Try and take money from any of these groups and see how popular your little “greenie science fair projects” will become. Sure, if we only subsidise green technologies it will accelerate their development but, on the other hand, there’s no incentive for big polluters to pollute less while the technology is catching up. So we’re only tackling half the problem.
  • More regulation – Chuck another fistful of laws onto the pile and get the Carbon Cops to lay the smack down. This is an even stupider idea. We’ll go round in circles having the same arguments we’re having now about what should be in these laws. All the while it’s the taxpayers that will be funding this political ballet. We already have regulations. They aren’t working. Adding more regulation is putting a bandaid on cancer.

Green is the new black

No one can argue that being lean, clean and green is a luxury at the moment. When the money isn’t flowing, it’s Mother Nature that picks up the tab. As a result, the countries that don’t have two cents to rub together will continue to pollute their way into the 21st century – they’re just a little late to the whole Industrial Revolution party, that’s all. You can’t blame them.

Please start playing “I Still Call Australia Home” in the background.

But we live in the lucky country. We can afford to make major changes towards turning the tides, literally, on climate change. In the process we may even find something so super cool that we share it with our poorer, carbon farting neighbours. Although we actually emit a small fraction compared to the rest of the world, us, as Australians, always punch well above our weight division when it comes to forward thinking. If you’re still not convinced that we can do it, I’m sure that I can arrange a swap with someone on Christmas Island (I hear that they have a really optimistic view about Australia). Joking about the last bit, but not joking about everything else. Let’s just do it already!

Ok now turn off that shit. Qantas totally ruined that song for me.

The LightSwitch Is On

…but is anyone home?

Ok horrible pun, but here’s the question – is Microsoft LightSwitch worth the price of admission?

A little autobiographical interlude

So I attended my first meeting of the Perth .NET Community of Practice yesterday. It was held in the seminar room at Ennex 100, right in the heart of the CBD. I had no idea that the Perth Microsoft Office was right next door – they must have the most kick ass lunches!

The talk was by Andrew Coates about LightSwitch. The talk itself was great and I think I’ll be attending more of these kind of things. Was a little bemused by the fact that Visual Studio 2010 managed to lock up during the demo several times, especially considering that Andrew’s laptop was rocking 8 cores!

Back to the point – LightSwitch

LightSwitch is a totally RAD, sorry I mean Rapid Application Development, tool which gives non-software people, or as Andrew referred to them: “The Omega Geeks”, the ability to quickly build a forms over data application. My first gripe is with the non-software people part, but I’ll get back to that.

So when LightSwitch fires up, you’re asked to specify where the data is coming from. You can pull in data from a variety of different sources, as long as they are Microsoft SQL Server or Microsoft Sharepoint. Luckily this aspect, along with most other aspects of the product, is extensible.

Next, you’re essentially asked to design the database schema. Woah, hold up! That doesn’t really sound like a non-software person kind of job. Even the plain English definitions of One-to-One, Many-to-One, etc. will probably be lost on the target audience. Validation functions? Computed fields? Forget about it. Second gripe.

Next, you define your screens – essentially the ‘view’ part if we’re talking MVC. This part looks rather neat. There’s a bundle of readily available common screen formats available: row edits, list and details, new entry, grid. Select your screen, point it at a table in your database and BOOM, out pops a… wait, what? A tree view of the screen? This is my third gripe. So now you’re non-software people have to arrange everything in this tree, because apparently non-software people don’t have a clue about HCI either. To see the result you have to build the solution and launch the application. Confused? Imagine how the user is going to feel.

Ok enough hating, now the good bits. Everything is extensible. You name it, it’s extensible. And not in some junky macro language extensible I mean plug in there and start writing C# code extensible. So here’s the idea: Non-software person reaches the limits of their ability. They can then call in Captain Software to the rescue, who dives in and starts developing more powerful features with all the power and flexibility and portability of .NET. Oh wait sorry, forget the portability part.

So I admit, this is much better than the existing alternatives – Microsoft Access databases or (worse) an Excel Spreadsheet! Another biographical aside. I once worked for a company that did all of their company finances in Excel. With macros, cell formulas and the rest of it. No wonder the books didn’t balance. Eeeeek. But is this really a solution for the non-software guy, or is this simply another enabling tool that allows people out of their depth to dig their way into a giant, expensive, ominous mess?

The takeaway: Don’t let the “Omega Geeks” near our software!

I don’t think LightSwitch is the answer to the knowledge gap between us “Alpha Geeks” and the “Omega Geeks”. I can see the point that there is a great divide between these two camps and both of us are just as bad as each other when it comes to communicating effectively.

An example dialogue (paraphrasing from Andrew’s presentation):

“Omega Geek”: I need an application for my book store

“Alpha Geek”: Ok so give me a list of requirements and I’ll get back to you in a few weeks to talk architecture

“Omega Geek”: Archi..wha? I already have a house, I don’t need another one.

MONTHS PASS

“Alpha Geek”: (panting) Ok here it is. It works using a plugin architecture, just enable the plugins you need at runtime. Oh and configuration is done using a backend database. To change the configuration you just need to do a few SQL queries. Simple right? Oh and I had a problem getting it to run on Windows XP, the target platform you originally asked for, so you’re going to have to upgrade everyone to Windows 7. Oh and I didn’t get time to write the user manual. Cool?

“Omega Geek”: Dude. I run a book store. Well I used to. Now I’m broke. Thanks a lot.

Now a counter-example:

“Omega Geek”: I need an application for my book store

“Alpha Geek”: Ok what do you want it to do?

“Omega Geek”: I don’t know. Database or something. Reporting. Automate. Internet. And stuff.

“Alpha Geek”: (sigh) Have you heard of this new awesome LightSwitch thing. It’s awesome. Bye.

I’m not trying to be jerk to anyone here, it’s just the nature of the beast that we attempt to tame every day. So how do you cross this great divide? In my opinion, you build a bridge. You insist on close and consistent communication between both parties. You iterate early and often. You remain flexible to change at all times. Uh wait, sounds like I’m stumbling towards something like Agile. What I’m adamant that you don’t do is give the “Omega Geeks” a hang glider and say JUUUUUUMP!

An alternate use case: Tool for developers and aspiring developers

So just quickly, I’m already past the point of rambling here, what can we do with a broken LightSwitch? How bout we turn the tables. I see LightSwitch as a fantastic tool for “Alpha Geeks” to use as part of their development process. Want a working GUI prototype in seconds? Boom. Want to let your customer populate the back end before the front end has caught up? Boom. Want to get fancier? Reach for your C# and boom! (ok, you get the point. Boom)

What about the kiddies? (I say knowing full well that most “kids” are already far more advanced in their software career than I am, after selling their fart app on the App Store). I see this as a great way for aspiring developers to get started with .NET. Think about it, a rich, graphical, tweak-able application seems a lot more exciting to me than “Hello World”. As their code ninja skills increase, they can delve further and further into the code until they look around one day and say “Holy crap, I’m an Alpha Geek!”.

Ruby on Rails on Windows Quickstart

Just started messing around with Ruby on Rails development. Here’s my quickstart to getting it up and running on Windows XP:

Download the one-click installer from http://rubyforge.org/frs/?group_id=167 (I installed version 1.8.6-27 RC 2)

Use all the defaults for the install (make sure you opt to install the gems package manager)

Download the latest version of Sqlite3 (a small, simple database) without the TCL bindings (i.e. sqlitedll-*.zip). I used version 3.6.13.

Unpack the zip file and copy the contents to your c:\ruby\bin directory (replace c:\ruby with your own install path)

Now open the RubyGems Package Manager command prompt (which should be in the new start menu shortcut under Ruby -> RubyGems)

type:
> gem install rails
> gem install sqlite3-ruby -v 1.2.3

(this last step is necessary as the latest version of sqlite3-ruby doesn’t seem to work for me – see here for more details)

now you can test using: (from the command prompt)
> rails test
> cd test
> ruby script\server
(this should start the local webserver)

navigate to http://localhost:3000/ and you should be greeted by the Ruby on Rails: Welcome aboard page

Yesterday

Recorded a new tune the other day.
Yesterday

Did the guitars using my crappy Behringer DI and vocals using an SM58 beta. Other guitars are recorded by pointing the SM58b towards the amp and hitting record!

Mastered the tune using Jamin.

Let us know what you think!

Mountjoy Studios – Now With Piano

It looks like the studio is all coming together now that I’ve added an upright piano into the mix. I’ll post the full details of the studio soon but, for now, most of my time will be spent learning how to really use all of this gear to its full potential. Running Ubuntu Studio 8.10 with Ardour as the editor and Jack for the audio plumbing. I’ve been playing around with this stuff for about a year or so now but decided it’s time to actually learn it properly. The good news is that the community around this sort of setup is really friendly and helpful.

Some useful resources for getting started:

The New Addition

The New Addition

Laser USB Webcam gets Linux Drivers! (kind of)

Knowing that driver support for USB webcams on Linux is usually quite hit-and-miss, I didn’t have very high hopes after buying two Laser USB Webcams from zazz. The main problem is that device manufacturers usually only release drivers for the majority of their market (i.e Windows users), leaving Mac and all the various *nix’s in the dark. This leads to people like this guy having to pick up the slack, usually with no help from the original manufacturer.

However, one point which does help driver development is that a lot of these cheap re-branded Chinese products use one of only a small number of chipsets (the main brains of the device) – but good luck trying to find the chipset brand in the product documentation. One useful tool for helping to find the product chipset yourself is a free tool called USB sniffer for Windows (I found version 1.5 seemed to work the best for me on a Windows XP SP2 machine). The problem with this method is that you need a Windows box to run this tool. Anyway, using this I found the chipset number was SN9C201 , which is made by a company called Sonix and is employed in numerous USB webcams.

After a bit of googling, I managed to find this which does have a working binary driver for this chipset. The only drawback is that this driver only works under Ubuntu 7.04. So if you’re not a fan of this distro, you’re going to be struggling to get the driver to work (it didn’t work under my Ubuntu Studio 7.04 as this distro uses the low-latency kernel). I’m going to try it on a stock installation of Ubuntu 7.04 later today.

update: ok so it does work under Ubuntu 7.04, just not very well – output is very blocky although framerates are acceptable.

In the mean time, you might want to watch this space for the development of an open source driver.

update: Thanks to a commentor pointing me to this google group, i’ve managed to get an open source solution working. Here’s a rundown on how I got the Laser USB Webcam working under Ubuntu Studio 8.10 (Intrepid Ibex) – I’ve simply augmented the instructions located here:

$ sudo apt-get install git-core
$ sudo apt-get install libv4l-0 libv4l-dev kernel-package build-essential
$ git clone http://repo.or.cz/r/microdia.git
$ cd microdia
$ make

(Don’t worry about the Error 127)

$ sudo modprobe videodev
$ modprobe compat-ioctl32
$ insmod sn9c20x.ko

Test with:

$ sudo apt-get install mplayer
$ LD_PRELOAD=/usr/lib/libv4l/v4l2convert.so mplayer tv:// \
-tv driver=v4l2:width=640:height=480:fps=25:device=/dev/video0 -vo x11

Install with:

$ strip -g sn9c20x.ko
$ mkdir -p /lib/modules/`uname -r`/kernel/drivers/media/video/usbvideo/
$ sudo cp sn9c20x.ko /lib/modules/`uname -r`/kernel/drivers/media/video/usbvideo/
$ sudo depmod -a

Midomi – Music Search Engine

Only recently, I had a friend ask me how hard it would be to create a music search engine that could act like your local (underpaid) CD store worker – where you could come in and say “I heard this song on the radio that kind of goes ‘do do da da doo do baby, yeh’, do you have this one?”. Like anyone, I thought this would be pretty-much impossible as it would need a massive database of melodies (read – every song in the world ever written) and a very powerful analysis engine which could quickly pattern-match whatever you sing into your microphone. That’s why, when I found this website, I was left eating my own hat with humility, as they seem to have done it (although not amazingly well).

Midomi seems to approach the problem by leveraging the very Web-2.0 concepts of user-generated content and social networking. The front page contains a little Flash control which allows you to sing a melody into your microphone and click the “done” button. The search takes around 20 seconds but, sadly, only seems to return a relevant result about 1/4 of the time (for example, “Heaven” by Bryan Adams seems to work but I was very offended when my favorite karaoke song “Total Eclipse of the Heart” by Bonnie Tyler didn’t return any results). It seems that it is the users that have solved the problems of that massive music database, by submitting their own renditions of themselves singing popular tunes. The search engine then attempts to match your input to their massive collection of uploaded content.

It would seem intuitive that as more content is uploaded, the search function will probably get a lot better. On the other hand, the site also, at times, seems to be nothing more than a chance for closet musicians to get their talents out there (this guy has done over 80 renditions of various songs) – which isn’t at all a bad thing, except that there exist zillions of other sites already out there that seem to do the same thing. The success or failure of this site will be whether Midomi can effectively utilise all of this content to help visitors find music online and, eventually, sell them the end product at an affordable price.

Links:

Midomi.com