File Conveyor

School year: third bachelor year, second semester.

See http://fileconveyor.org http://wimleers.com/fileconveyor! Also see a complete overview of technologies used.

The International Game Developers Organization’s Global Game Jam deals with an enormous influx of new files (the games developed during the GGJ) in a very short timespan (48 hours). These files need to be downloaded very fast, by many people. A CDN is an obvious fit here. That’s not a problem however; during budget constraints, they only have limited space on their (origin) web servers.

http://whitehouse.gov’s problem was that files were being synced too slowly from their web servers to their CDN, because rsync needs to scan the entire directory tree to detect changes. File Conveyor¹ is faster, because it relies on inotify to detect changes.