du -skh * in / returns vastly different size from df on centos 5.5-Collection of common programming errors

11 years ago
admin
2 minutes

The most common cause of this effect is open files that have been deleted.

The kernel will only free the disk blocks of a deleted file if it is not in use at the time of its deletion. Otherwise that is deferred until the file is closed, or the system is rebooted.

A common Unix-world trick to ensure that no temporary files are left around is the following:

  • A process creates and opens a temporary file

  • While still holding the open file descriptor, the process unlinks (i.e. deletes) the file

  • The process reads and writes to the file normally using the file descriptor

  • The process closes the file descriptor when it’s done, and the kernel frees the space

  • If the process (or the system) terminates unexpectedly, the temporary file is already deleted and no clean-up is necessary.

  • As a bonus, deleting the file reduces the chances of naming collisions when creating temporary files and it also provides an additional layer of obscurity over the running processes – for anyone but the root user, that is.

This behaviour ensures that processes don’t have to deal with files that are suddenly pulled from under their feet, and also that processes don’t have to consult each other in order to delete a file. It is unexpected behaviour for those coming from Windows systems, though, since there you are not normally allowed to delete a file that is in use.

The lsof command, when run as root, will show all open files and it will specifically indicate deleted files that are deleted:

# lsof 2>/dev/null | grep deleted
bootlogd   2024       root    1w      REG                9,3         58     917506 /tmp/init.0W2ARi (deleted)
bootlogd   2024       root    2w      REG                9,3         58     917506 /tmp/init.0W2ARi (deleted)

Stopping and restarting the guilty processes, or just rebooting the server should solve this issue.

Deleted files could also be held open by the kernel if, for example, it’s a mounted filesystem image. In this case unmounting the filesystem or rebooting the server should do the trick.

In your case, judging by the size of the “missing” space I’d look for any references to the file that you used to set up the VPS e.g. the Centos DVD image that you deleted after installing.