The Graham Utilities for OS/2 - Version 2

Appendix G - HPFS Internals

Background Information

The High Performance File System (HPFS) was introduced with OS/2 version 1.2. It was designed by Gordon Letwin, the chief architect of the OS/2 1.x operating systems. HPFS uses different structures, caching and allocation techniques to achieve superior performance over the File Allocation Table (FAT) file system.

Base Structures

What follows is a highly technical discussion of the internals of the HPFS disk structures and organisation.

Boot Blocks

The HPFS has only three items which have a fixed location. The first is the volume boot sector which starts at logical sector 0. Sectors 0 to 15 are reserved for the boot sector and up to 8K of disk bootstrap code. The boot code can access HPFS in a restricted mode; enough to ensure the system is booted and running to the point of being able to load HPFS.IFS - which is the real HPFS driver.

Super Block

Located at logical sector 16 is the Super Block. The Super Block contains the following information:

Super Block Information:

Root Directory FNode: This is the logical sector number of Root Directory FNode.
Size of volume: The size of the volume in logical sectors.
Number of Bad sectors: The total number of bad sectors on the disk.
Bitmap sector lists: This is the sector number of a group of four sectors which form a list of all of the bitmap sectors. Each group of bitmap sectors is four sectors long. The four sectors of each bitmap represents 16,384 bits. Each bit represents the usage of one sector. A 0 means the sector is used and a 1 means that the sector is free. 16,384 sectors is 8Mb, and thus the size of the bands on a HPFS disk. This is an arbitrary limit. However, it may change in future revisions of the HPFS. To maximise the amount of contiguous free space two 8Mb bands are grouped back to back, giving a maximum of 16Mb of contiguous file space. Because the usage of the disk is mapped out by the bitmaps, it allows the HPFS to minimise file fragmentation when a file is created.
Bad sector lists: This is the sector number of a group of four sectors which form the list of currently marked bad sectors. The first four bytes of the list is the sector number of the next bad sector list (zero if there are no more). This leaves room for 511 bad sectors. Bad sectors are marked as used in the bitmaps.
Last CHKDSK /F: This is the date and time at which the last CHKDSK /F was run. GMT is used. The value is zero if CHKDSK has never been run.
Last optimise: This is the date and time of when the last disk optimise was run. GMT is used. The value is zero if the disk has not been optimised.
Size of directory band: The number of sectors which form the directory band. The directory band is located near the seek centre of the disk.
Start, end of dir. band: The starting and ending sector of the directory band.
Bitmap for dir. band: The sector number of the bitmap particularly for the directory band.
ACL Information: There are eight sectors which are allocated near the directory band which is used by HPFS386 to store ACL (Access Control List) information.

The Super Block is only modified by disk maintenance utilities such as CHKDSK HPFS-Bad, HPFSDfrg and HPFSView.

A complete description of all fields in the Super Block is contained in Appendix I - DiskEdit Modules.

Spare Block

The Spare Block is at logical sector 17. It contains the following information:

Volume status flag: The partition status flag reflects the current status of the partition. It also includes the dirty flag which tells the system whether the volume was shut down properly (clean) or if it was improperly stopped and there was information in memory which was still waiting to be written to disk (dirty). When the system boots it checks the state of the dirty flag. If it is dirty, CHKDSK is run to resolve any differences, and if necessary rebuild the necessary disk structures. If the boot volume is dirty the boot procedure stops and you must boot OS/2 from a floppy disk to resolve the problems.
Hotfix list: This is the sector number of the hot fix list - which is four sectors long. This list contains the sector numbers of the sectors which have gone bad and the sector numbers of the good sectors to which they have been remapped. The HPFS checks any real sector reads/writes (as opposed to reads/writes from cached sectors) with this list and substitutes them as necessary.
Number of hotfixes: This is the total number of hotfixes which the volume has.
Number of hotfixes in use: This is the number of hotfixes which are currently used.
Number of spare DIRBLKs: The system maintains a number (generally 20) of spare DIRBLKs. They are usually at the end of the directory band as defined by the Super Block.
Code Page Information: The sector number of the Code Page Information sector for the volume.
Number of Code Pages: The number of Code Pages which are on the volume.
Pointers to spare DIRBLKs: This is a list of the sector numbers of the spare directory DIRBLKs which the system maintains.

The Spare Block is modified, (although infrequently), as the system operates.

A complete description of all fields in the Spare Block is contained in Appendix I - DiskEdit Modules.

As can be inferred from the above information, the layout of the HPFS volume looks similar to the diagram below:

FNodes

The fundamental file system object on an HPFS volume is an FNode (pronounced "eff node"). The FNode contains the true length of the file name, the first fifteen characters of the file name (so that disk recovery utilities such as HPFS-UD can recover deleted files or files from damaged volumes), a pointer to the FNode's parent directory (to enable you to build up path information), extended attribute information and information regarding the position of the data sectors for the file itself. If the file is in eight or less extents, the FNode has room to contain the starting sector and the number of sectors for each sector run.

If the file is in greater than 8 extents, an extra allocation node (or ANode), must be created. There is room for 12 ANodes in the FNode. In this case, the structure of the FNode changes to resemble this:

In this example the ANode contains direct sector runs, not further ANodes.

ALSECs Each ALSEC has room for 40 direct sector runs or 60 pointers to further ALSECs. There is no limit to the number of ALSECs which can be allocated. They will continue to be allocated until they can contain all of the sector runs for the file. Each ALSEC is one sector long.

DIRBLKs

Similar to files, directories are anchored on FNodes. A pointer to the root directory FNode is located in the Super Block. Directories are built up of DIRBLKs. Each DIRBLK is four sectors long They are allocated as four contiguous sectors. Once the directory band is full, they are allocated where ever there is space. The directory band is located near the seek centre of the disk for performance reasons (less head movement). The DIRBLKs are composed of variable length records which represent each directory entry. They are called DIRENTs. Each directory entry contains the following information:

When a file is deleted the complete directory entry is overwritten. Thus, information such as the complete filename and other attributes is lost. The undelete process can only process information found in deleted FNodes; which is why only a limited amount of information about files can be recovered from HPFS volumes.

Extended Attributes and ACLs

There is room for 316 bytes of extended attribute data, or ACL information in the file's FNode. If the amount of extended attribute data exceeds this it can be moved to external sector runs which contain the data; or even an ALSEC to describe heavily fragmented extended attribute data.

There is a bug in the current implementations (both HPFS and HPFS386) of the HPFS.IFS driver. A file's EA's will be lost when the number of extents needed to store the EA exceeds 40.

This is scheduled to be fixed in a future release.