posted by Christian Paratschek on Thu 10th Feb 2005 20:14 UTC

"VFat, Page 3/3"
2.3. File size limits

Linux only supports files up to 2GB size on vfat. You can at least backup bigger files by splitting them up into smaller parts. In the following example, I am backing up a 5GB-cryptoloop file to a vfat partition by splitting it into 2000MB pieces:

	> cd /myvfat
	> split -d -b 2000m /myext3/cryptfile cryptfile_backup
	> ls
	cryptfile_backup00 cryptfile_backup01 cryptfile_backup02 

Files bigger than 2GB must be excluded from rsync of course.

Reassembling the original file can be done via:

	# Linux
	> cat cryptfile_backup00 cryptfile_backup01 cryptfile_backup02 > cryptfile
	# DOS
	> copy /b cryptfile_backup00 + cryptfile_backup01 + cryptfile_backup02 cryptfile

2.4. Using rsync

The characteristics of vfat must also be considered when calling up rsync. As vfat does not support symbolic links, file permissions, owners, groups and devices, usage of the parameter "-a", which considers all of the above will not have any effect (aside from the error messages). Thus it's best to use only the parameters that actually work:

	-r,	--recursive   treat folders recursively
	-v,	--verbose     show transmitted files
	-t,	--times       keep time settings
	-n,	--dry-run     test only
		--exclude     ignore files

As the date of the last file change is very important for rsync, the option "-t" is essential. It's also very wise to test every rsync change with "-n" before.

The last roadblock to a successful synchronization with rsync is the time resolution of the vfat time stamp. It amounts to a little more than one second. You'll have to set the parameter "--modify-window=1" to gain a tolerance of one second, so that rsync isn't "on the dot".

In a nutshell, the command to efficiently transmit all files from an ext3 file system to a vfat file system is:

	> rsync -rvtn --modify-window=1 --exclude "lost+found" /myext3/* /myvfat

3. Problems with large vfat file systems

The rapid growth of hard drive size poses big problems for the rather neglected dosfstools and vfat-driver.

3.1. Lost Clusters

Under Linux Kernel 2.4.x., a limit of the cluster data type results in data loss, as soon as the vfat file system holds around 130GB. In Kernel 2.6.x., this problem was - rather accidently - solved, when many variables were consequently provided with a new type. A detailed description of this bug, including a testsuite and a patch (by Erik Andersen) can be found here. (The patch also allows for file sizes up to 4GB).

If you, however, work with a 2.4.x. Kernel and have a "full" vfat partition, be prepared to lose data: any written file in a new folder will be lost after unmounting the file system. When you mount the file system again, these files have a size of 0 and the clusters are in limbo. You can delete the unassigned clusters via dosfsck.

3.2. dosfsck and high RAM Demand

To conduct file system checks as efficiently as possible, dosfsck copies both FATs to RAM. With a very large file system on a 250GB drive, the high number of clusters yields a very high demand for RAM, that surpassed my 350MB (including swap). Thus dosfsck aborted with a malloc-error. Roman Hodek, the maintainer of dosfsck, proposed to convert the program to "mmap()", but also said that this change would be complex. As long as this situation has not changed, be sure top have sufficient RAM.

3.3. Executing dosfsck

As long as the vfat file system is mounted, dosfsck can be executed, but all repairs silently fail. Thus you should make sure that your partition is not mounted befora using dosfsck. In the following example, an unassigned cluster (due to the bug in Kernel 2.4.x.) is located and deleted. By the way, the command fsck.vfat is a symbolic link to dosfsck.

	> fsck.vfat -vr /dev/sda1
	dosfsck 2.10 (22 Sep 2003)
	dosfsck 2.10, 22 Sep 2003, FAT32, LFN
	Checking we can access the last sector of the filesystem
	Boot sector contents:
	System ID "mkdosfs"
	Media byte 0xf8 (hard disk)
		512 bytes per logical sector
		16384 bytes per cluster
		32 reserved sectors
	First FAT starts at byte 16384 (sector 32)
		2 FATs, 32 bit entries
	39267840 bytes per FAT (= 76695 sectors)
	Root directory start at cluster 2 (arbitrary size)
	Data area starts at byte 78552064 (sector 153422)
		9816944 data clusters (160840810496 bytes)
	63 sectors/track, 255 heads
		0 hidden sectors
		314295660 sectors total
	Checking for unused clusters.
	Reclaimed 1 unused cluster (16384 bytes).
	Checking free cluster summary.
	Free cluster summary wrong (641900 vs. really 641901)
	1) Correct
	2) Don't correct
	? 1 
	Perform changes ? (y/n) y
	/dev/sda1: 143 files, 9175043/9816944 clusters

3.4. Formatting a large vfat file system

When formatting with mkfs.vfat you have to add the option -F 32, so that a 32-bit file system is created. Without this option, a 12-bit or 16-bit file system is created, depending on the partition size, or the formatting process aborts (on an oversized partition). Fat16 only supports file systems up to 2GB, fat32 allows for up to 2TB (terabytes).

	> mkfs.vfat -F 32 /dev/sda1

4. Conclusion

Solving the problems described here cost me a lot of time. But to me, being able to perform my work exclusively with Free Software is a luxury that makes it quite worthwhile. Thus I want to thank all developers of these programs heartily. If the psychological strain of the aforementioned problems grows big enough, there will some volunteers who will approach the remaining problems.

© Torsten Schenk (torsten.scheck@NOSPAM.gmx.de)

License:

This text is subject to the GNU Free Documentation License (FDL). Free spreading in modified or unmodified form is allowed. Modifications must be marked unmistakeably and also distributed under the FDL.

Translated by Mag. Christian Paratschek. More of my work can be found on my website.


If you would like to see your thoughts or experiences with technology published, please consider writing an article for OSNews.
Table of contents
  1. "VFat, Page 1/3"
  2. "VFat, Page 2/3"
  3. "VFat, Page 3/3"
e p (0)    26 Comment(s)

Technology White Papers

See More