Brilliant. Despite telling customers that there would be no change in the anti-copy protection on their latests DVD's, Sony has backed down. They're now issuing exchanges for every faulty disc.

The only thing i find odd is that they claim to have received complaints for less than one thousandth of one percent of affected discs shipped. Now, the pedant in me says instantly shouts "false statistics". Whats the point in giving stats based on units shipped. Surely the only stats that would matter would be on units sold. Obviously if you have 1000 discs in-shop, you're not going to receive complaints until someone buys them.

Despite AACS being broken for both BluRay and HD-DVD once already, and a huge list of keys for DVD's being created for both HD-DVD and BluRay with some parts of AACS permanently bypassed, these mega-corps still haven't learnt that DRM does not work.

Once again, in a desperate attempt to make their discs copy-proof, Sony has managed to screw up severely. A year ago, Sony decided that they'd install a secret rootkit which could not be detected by regular means on computers which attempted to play their Audio CD's. As bad as that was, at least the CD's were playable!

Now they've come up with a new ploy: Make their DVD's unplayable on their own DVD drives. If you can't play it, you can't copy it. To be honest, this must be a PR nightmare for them. They are knowingly selling bastardised imitation DVD's that do not play on dvd drives. Hell, they are still selling their own DVD drives without a warning that it will *not* play new sony DVDs.

I suppose the worst thing out of this is that other dvd drive manufacturers may be forced to spend time and effort updating their drive's firmware in order to allow their customers to use their legitimately bought entertainment discs.

I had a rather interesting experience on the bittorrent mailing list recently. There was a guy who had a server which was severely IO limited and he was trying to figure out how best to improve performance. The problem was that due to the randomness of how a piece is chosen to be downloaded in bittorrent, his server was having to do a lot of random disk seeking thus limiting his maximum upload rate to about 11-15MB/sec even though he had the bandwidth to handle much more than that.

There are hacks to avoid this kind of IO limiting, such as advertising that you have no pieces available When you connect to a peer and then telling them about selective pieces later. The idea is that the piece you advertise having, you'd buffer in memory and so when the peer requests the piece, you don't have to seek to disk to get it for them. With a sufficient memory cache you significantly reduce the amount of random disk access needed and increase throughput hugely

However, this has several disadvantages. The biggest disadvantage of this method is that it can (and does) reduce overall throughput. Take a situation where a torrent has 100 pieces. You put the first 10 pieces into memory and then send messages advertising those 10 pieces to everyone. If everyone already has those pieces, then no uploading will occur until you swap out those pieces and upload the next batch.

Secondly, you *must* disconnect every peer you've connected to every time you decide to swap out those 10 pieces. If you don't, the other peers will remember about the previous pieces you've advertised and may request those even though you no longer have them in memory. This means you once again have random seeking. However, the amount of random seeking would still be significantly reduced as each peer would only know about 20 pieces out of the actual 100 if they weren't disconnected.

Thirdly, this method of seeding requires special logic. Therefore if you want this kind of special handling you must either hack it into an existing open source client yourself, pay someone to do it, or find a client that supports this and use it (regardless of whatever other problems/deficits that client may have).

Now for the fun part: This issue was addressed in the Fast Extensions for the bittorrent protocol with the "SuggestPiece" message.

Suggest Piece is an advisory message meaning "you might like to download this piece." The intended usage is for 'super-seeding' without throughput reduction, to avoid redundant downloads, and so that a seed which is disk I/O bound can upload continguous or identical pieces to avoid excessive disk seeks.

As you can see, the Suggest Piece was designed to fix the exact problems i've described above, unfortunately due to either bad wording, or just bad interpretation, this message is going to fail miserably in that.

The SuggestPiece message is an "Advisory" message. My own opinion is that Advisory was added there to signify that you do *not* have to act on that message. The reason for this is that you could already be downloading piece 3 off peer X when peer Y sends a "SuggestMessage" suggesting you download Piece 3 off him instead. In this case, you would not follow the suggestion, you would just ignore it. That makes sense. It's logical. Downloading the same piece twice would just be stupid ;)

Unfortunately, several bittorrent developers who've implemented the Fast Extensions have mis-interpreted this as "You can completely ignore this message if you want to". One developer cited his reason for completely ignoring this message as "i have no use for this, so i ignore it". This is a stupid reason for not implementing proper handling for the SuggestPiece message. Even "I'm too lazy" would be a better excuse as you'd at least be acknowledging that you have an incomplete implementation.

A "proper" implementation of the SuggestPiece message would be to request that piece off the peer that sent you the suggestion at the earliest convenient time, i.e. the very next piece request off them should be for the suggested piece.

Lets assume that every torrent client implemented the suggest piece message handling as i described above. Now, i'll replay the situation above using SuggestPiece messages as opposed to selective piece advertising.

Firstly, you advertise having all the pieces as being available when you connect to each peer (this requires no special extra logic). Every time you receive a request from a peer to send them a piece, you load the entire piece into memory. As soon as that piece is loaded into memory, you send a SuggestPiece message (this requires no special extra logic) to every other peer. The other peers will then act on that message (this requires no special extra logic) and their next request from you will be that piece. That way every time you load a piece into memory once, you can then send it from memory to every other peer that wants it.

The benefits of this method are that every time you load a piece into memory, you can make other peers request it off you (if they need that piece). You will *never* have a situation where you will not be uploading. You will *always* have better performance than the initial scenario where you were IO limited. Assuming that you're still doing the same number of random disk reads as before, you will be able to increase your upload rate significantly because each piece that enters memory will be sent to more than 1 peer.

I'd guess that you could increase performance by, at the very least, 5 fold in a torrent with 100 other peers using the SuggestPiece method. The other major advantage is that you require no special logic in either the seeding client or downloading client! All you need is a proper implementation of the SuggestPiece message. It's not that hard!

EDIT: Despite promising myself i wouldn't, i think i will name and shame the clients that completely ignore the SuggestPiece message despite claiming support for the Fast Extensions:

KTorrent
Bitflu

The clients that i know of that fully support the extensions are:
MonoTorrent
MainLine

I was just talking to J.D. Conley (part of the Coversant team. He also has a blog on the net) about some if the issues i've been having in monotorrent with threading and async sockets. He had a few good ideas about what i could try, so hopefully over the next week or three i'll be able to test out a few of the ideas to help improve performance. Anyway, as part of the discussion i realised that i could probably replace a lot of my lock(object) statements with a much nicer ReaderWriterLock().

The way the code works in monotorrent is that i read from my lists of peers much more than i modify them, therefore it makes sense to have a locking structure that allows multiple readers and a single writer! This gave me the idea for this blog post.

You see, the problem with the ReaderWriter lock is that you have to be careful in how you use it. It's not as simple as the lock(object) statement. If you forget to release a reader lock, or worse, a writer lock, you will experience deadlocking and it will be very hard to track down the exact place where you haven't released the lock!

Also, suppose you want to a writer lock, but you forget to check if your thread already has a reader lock, you will end up throwing an exception as you will not be able to get the writer lock.

However, it's very easy to abstract away all those horrible problems with two simple structs:

public struct ReaderLock : IDisposable
{
    public ReaderWriterLock Locker;

    public ReaderLock(ReaderWriterLock locker)
    {
        Locker = locker;
        locker.AcquireReaderLock(1000);
    }

    public void Dispose()
    {
        Locker.ReleaseReaderLock();
    }
}

public struct WriterLock : IDisposable
{
    private bool upgraded;
    private LockCookie cookie;
    public ReaderWriterLock Locker;

    public WriterLock(ReaderWriterLock locker)
    {
        Locker = locker;
        upgraded = locker.IsReaderLockHeld;
        cookie = default(LockCookie);

        if (upgraded)
            cookie = locker.UpgradeToWriterLock(1000);
        else
            locker.AcquireWriterLock(1000);
    }

    public void Dispose()
    {
        if (upgraded)
            Locker.DowngradeFromWriterLock(ref cookie);
        else
            Locker.ReleaseWriterLock();
    }
}

Suppose you have the following code snippet:
ReaderWriterLock myLocker = new ReaderWriterLock();

Instead of having to manually use: myLocker.AcquireReaderLock() and remember to call myLocker.AcquireWriterLock() in a finally statement, and instead of messing around trying to decide if you should call myLocker.AcquireWriterLock or myLocker.UpgradeReaderLock() you can just use the corresponding ReaderLock or WriterLock as needed inside a using statement!

For example, take this code snippet:

try
{
   DoSomeStuffWhichMayOrMayNotThrowAnException();
   myLocker.AcquireReaderLock();
   DoLotsOfStuff();
   myLocker.ReleaseReaderLock();
}
catch(WhateverException)
{
   if(myLocker.IsReaderLockHeld)
        myLocker.ReleaseReaderLock();

   DoRestOfErrorHandling();
}

You can use the much nicer:

try
{
   DoSomeStuffWhichMayOrMayNotThrowAnException();
   using(new ReaderLock(myLocker)
       DoLotsOfStuff();
}
catch(WhateverException)
{
   DoRestOfErrorHandling();
}

Supposing you actually needed a writer lock, just change the using statement to:
using(new WriterLock(myLocker)

The constructor for writer lock will then check to see if it should upgrade an existing reader or if it should get a standard WriterLock. If it upgraded, it will remember to downgrade it correctly later.

Using this method not only makes you need less lines of code, makes your code more readable, but it also completely removes the possibility of creating a deadlock by accidentally forgetting to release a lock!