rsync'ing images

There's two types of daily image users:

  • Those who download now and again (me)
  • Those who download every day (testers)

What's the best way of compressing and transferring the image to meet both of their needs?

I had a play with two headless images that were a month apart, compressed them, and then tried transferring them locally. The file sizes in MiB were:

linaro-m-headless-tar-20101108-2

Compared to gz

linaro-natty-headless-tar-20101202-1

Compared to gz

tar

171.3

253%

168.2

261%

gzip

67.8

100%

64.4

100%

gzip rsyncable

67.9

100%

64.6

100%

xz

46.2

68%

43.7

68%

Note:

  • The xz file is 68 % of the size of the gzip version
  • The gzip --rsyncable version is very slightly bigger than the normal gzip version

I then transferred them locally using rsync, updating from the 20101108 to 20101202 image:

rsync

rsync -z

tar

120.6

236%

44.4

87%

gzip

50.9

100%

50.9

100%

gzip rsyncable

51.0

100%

51.0

100%

xz

43.5

85%

43.6

85%

The numbers are the MiB sent as reported by rsync. The percentages are the relative bytes transferred compared to a plain rsync of the gzip file.

Note:

  • xz wins, followed by the uncompressed tarball
  • gzip rsyncable was worse than plain gzip
  • xz transferred the whole file, but the file started out smaller
  • rsync -z makes no difference on compressed files
  • rsync -z of the original tarball is almost as good as the xz transfer

MichaelHope/Sandbox/RsyncingImages (last modified 2011-01-06 21:28:14)