Hackwee Day IV - The final showdown

Yesterday was a bit of an anti-climax codewise. Pretty much everything I had planned on doing I had completed by midday. I then had a nice long chat with lamalex about what I had to do to get ipod support into his udev branch and hashed out a rough idea about how to upstream these changes. I believe he's going to do a bit of work over the next week or so and once that's complete I'll be able to upstream the remainder of my patches.

So at the end of the 4 days the state of things is:

Device support:
1) Hotplugging works
2) Syncing music to and from the device works
3) Removing music from the device works
4) Basic properties can be read about the device and displayed in banshee
5) Playlists aren't supported yet but should be fairly simple to do

Code written:
1) I've upstreamed my patches for libgpod-sharp
2) I spoke with lamalex about upstreaming the rest of my work there.
3) There are now packages available for openSUSE 11.2 courtesy of FunkyM which can be used to provide most of the required packages.
4) There are still some bleeding edge libraries required which aren't packaged anywhere. These will have to be packaged.
5) I think the best way to support the libgpod based iDevice would be to create a new addin so that you can run both the old addin and the new addin in parallel as some of the required libraries for libgpod based iDevice support are very new and not widely available as of yet. This will take a little bit of time.

I'm off on holidays next week so when I get back I'll start pushing my remaining work upstream. Once that's done it's just the boring process of streamlining the build to remove the dozen manual steps which are currently required :) All in all, it's been a successful hackweek. But as with all good hacks, the 5:1 rule applies. For every day spent adding functionality, I'll need 5 to make it stable and usable for everyone.

I have to thank teuf again for all the help over the last few days (and for libgpod itself!), Nathaniel McCallum for the awesome start to the .NET bindings for libgpod, Alex Launi for the banshee gio/udev work which saved me a lot of time and also for sparing the time yesterday to talk about how I can get my work upstreamed.

If I've forgotten anyone or anything, sorry! But it's been a hectic week and it was hard enough keeping track of all the packages and patches I had floating around, never mind everyone I was talking to :)

I'll post again when I get everything upstreamed and I'll have a nice set of instructions for anyone wishing to test out their iDevice with banshee.

Hackweek Day III - The coffee shortage

Today was a slow but interesting day (again). I took all my banshee work and rebased it on top of lamalexs work to bring udev/gio support to banshee. This saved me having to write my own backend, but it meant that once again I had to start upgrading core parts of my OS. Luckily nothing blew up this time!

So after a few hours hacking and pulling my hair out because seemingly simple commands were failing (they still are failing, i just worked around it) I now have proper detection implemented. There are no more hardcoded horrible hacks :) Anyone with the right pre-requisites and the udev branch of banshee with my patch can have basic iDevice support (tracks only).

This is awesome!

This will not be pushed to the main banshee repository as it depends on udev/gio to be useful. I will instead push my patches upstream to lamalex and his udev branch. All that has to be done then is to push the udev work to mainline banshee and everyone can enjoy full iDevice support.

Tomorrow I hope to get in touch with lamalex and iron out a few bugs I've experienced and then upstream my work to him. My bugfixes to the libgpod-sharp bindings have already been upstreamed and should be merged with the main codebase very soon.

Hackweek Day II - Attack of the code

Today has been reasonably productive. I found some bugs in the libgpod bindings which were fairly major and implemented some more features in banshee.

First the fun bug. The libgpod bindings did the normal thing of defining a managed struct which mirrored the native struct that libgpod uses. This means we can do a trivial byte copy of the unmanaged memory into one of our managed structs and trivially access the information in a safe way. This is all well and good right up until you realise that when you update a property on that struct, you are not actually changing the value of the unmanaged memory. Essentially you have two copies of the same data which are disconnected - one in managed memory and one in native memory. Updating one copy is not reflected in the other and there's no easy way to keep them synced without the risk of losing data.

After a bit of brainstorming with Jeremie, we came up with a rather neat and nifty solution. Take this struct as an example:

struct NativeDate {
    public int Day;
    public int Month;
    public int Year;
}

If native code allocates one of those and passes it to .NET as an IntPtr, there is a trivial way to map this to managed code without requiring a copy of the data.

unsafe class Date {
    IntPtr Native {
        get; set;
    }

    public int Day {
        get { return ((NativeDate *)Native)->Day; }
        set { ((NativeDate *) Native)->Day = value; }
    }
    public int Month {
        get { return ((NativeDate *) Native)->Month; }
        set { ((NativeDate *) Native)->Month = value; }
    }

    public int Year {
        get { return ((NativeDate *) Native)->Year; }
        set { ((NativeDate *) Native)->Year = value; }
    }

    public Date (IntPtr native)
    {
        Native = native;
    }
}

All we do is use a bit of unsafe code to take the pointer that we got from native code and cast it to a NativeStruct* and read or write the data as appropriate. Now both managed land and unmanaged land are working off the same hunk of memory so they can never get out of sync.

Now for the features! Banshee can now sync music to my iPhone and remove it aswell! This means all the basic stuff that's required for basic syncing is supported now. It took a lot longer than it should've to get this working because my iPhone wasn't actually set up right. There was a command (i can't remember the name of it now) which should've been run automatically to populate iTunes_Control/Device with some required information. This was never run for some bizarre reason (yay for bleeding edge) which meant nothing I did could ever actually update the database.

However once I got that setup, things progressed pretty rapidly. There's one major feature left which is automatic detection when the device is mounted. If i can get this done tomorrow i'll try to upstream everything and get people testing this by the weekend (or friday!).

Hackweek V - Day 1

So at the end of Day 1, things have progressed well enough. I managed to get all my requirements installed (just about!) though to do it I needed to install older versions of some of the ipod stack. Hopefully this won't bite me in the ass as the week progresses ;)

So a brief synopsis of the state of the world as I found it after one day:

1) Things are still very bleeding edge. On opensuse you still need just the right packages and svn checkouts to even begin to get things working. Nothing you need is in the standard repositories, I had to use the awesome opensuse build service to provide some packages (gvfs >= 1.5.1, libplist, umuxd) and had to compile just the right version of libimobiledevice manually. Not a pleasant experience.

2) Once all that horribleness is done, actually detecting the ipod in banshee is difficult. HAL is useless. You need either gio# or udev to be able to detect the device when it's plugged in. This puts further requirements on your distro to ship the latest goodies. I worked around this by hardcoding my iphones mount point for now ;)

3) Banshee is good to work with. Integrating DAPs with banshee is easier than it used to be. Hopefully things will simplify the more I delve into things and learn what's available. Some things in the existing code I hope are only there for historical reasons and can be removed with this rewrite ;)

4) So far I have two big checkpoints: Banshee can load the tracks on my iphone and it can also import them to its collection. I'm in the process of allowing you to copy tracks to the device, but I hit a few issues with libgpod which have yet to be investigated.

So things are looking good for the first day. Hopefully day 2 will be full of interesting developments. Maybe I'll get playlist syncronisation working, maybe I'll get track uploading working. Who knows!

Hackweek V

Next week is Hackweek V in Novell, the week in which we can work on whatever we want. This year, my hackweek proposal is to give banshee a new ipod addin based on libgpod. This should fix some of banshees bugs.

While working on this I'll try live tweet as I hack so if you're interested, do tune in from tuesday morning onwards (GMT). Mondays a bank holiday so I'm going to have to complete my hack in 4 days.

If you can, do give the proposal a +1. I've no idea what affect it'll have but do it anyway! ;)

EDIT: Link the correct hackweek proposal. Whoopsie :)