7.3. Some facts about file systems and fragmentation
Disk space is administered by the operating system in units of blocks and
fragments of blocks. In ext2, fragments and blocks have to be of the same
size, so we can limit our discussion to blocks.
Files come in any size. They don't end on block boundaries. So with every
file a part of the last block of every file is wasted. Assuming that file
sizes are random, there is approximately a half block of waste for each file
on your disk. Tanenbaum calls this "internal fragmentation" in his book
"Operating Systems".
You can guess the number of files on your disk by the number of allocated
inodes on a disk. On my disk
# df -i
Filesystem Inodes IUsed IFree %IUsed Mounted on
/dev/hda3 64256 12234 52022 19% /
/dev/hda5 96000 43058 52942 45% /var
there are about 12000 files on / and about 44000 files on /var. At a block
size of 1 KB, about 6+22 = 28 MB of disk space are lost in the tail blocks of
files. Had I chosen a block size of 4 KB, I had lost 4 times this space.
Data transfer is faster for large contiguous chunks of data, though. That's
why ext2 tries to preallocate space in units of 8 contigous blocks for
growing files. Unused preallocation is released when the file is closed, so
no space is wasted.
Noncontiguous placement of blocks in a file is bad for performance, since
files are often accessed in a sequential manner. It forces the operating
system to split a disk access and the disk to move the head. This is called
"external fragmentation" or simply "fragmentation" and is a common problem
with MS-DOS file systems. In conjunction with the abysmal buffer cache used
by MS-DOS, the effects of file fragmentation on performance are very
noticeable. DOS users are accustomed to defragging their disks every few
weeks and some have even developed some ritualistic beliefs regarding
defragmentation.
None of these habits should be carried over to Linux and ext2. Linux native
file systems do not need defragmentation under normal use and this includes
any condition with at least 5% of free space on a disk. There is a
defragmentation tool for ext2 called defrag, but users are cautioned against
casual use. A power outage during such an operation can trash your file
system. Since you need to back up your data anyway, simply writing back from
your copy will do the job.
The MS-DOS file system is also known to lose large amounts of disk space due
to internal fragmentation. For partitions larger than 256 MB, DOS block sizes
grow so large that they are no longer useful (This has been corrected to some
extent with FAT32). Ext2 does not force you to choose large blocks for large
file systems, except for very large file systems in the 0.5 TB range (that's
terabytes with 1 TB equaling 1024 GB) and above, where small block sizes
become inefficient. So unlike DOS there is no need to split up large disks
into multiple partitions to keep block size down.
Use a 1Kb block size if you have many small files. For large partitions, 4Kb
blocks are fine.