Possible optimization: It struck me that if the repo is empty (ie. you are cloning), you could set this to a small multiple of BUFFER_SIZE - you know that you'll never have to go back and do a deferred check for a known blob, because you don't have any. Consequently you get the benefit of not having to allocate an array for any blob you encounter :)