Notes on the storage of attributes.
===================================

Resident attributes
===================

Resident attributes are small enough to be stored in the mft record itself.

When an attribute becomes too big to fit in the mft record or when the number
of attributes grows so large that there is no more space in the mft record, the
largest attribute(s) is(are) made non-resident. (Note, that some attributes
cannot be non-resident.)

Non-resident attributes
=======================

Non-resident attributes contain only their non-resident attribute header in the
mft record and the actual attribute value is stored anywhere on the volume
(but not in other mft records).

The value of the attribute is saved as a sequence of clusters, named virtual
clusters, beginning at virtual cluster number (vcn) zero. Each vcn corresponds
to a cluster on the volume, or a logical cluster number (lcn). The lcns on the
volume begin with lcn zero for the very first cluster on the volume.

The location of the attribute value is described by the mapping pairs array
present in the non-resident attribute header structure. The mapping pairs array
contains, in compressed form, the mapping between the attribute's vcns and the
corresponding lcns on the volume.

When the mapping pairs array of a non-resident attribute becomes too large to
fit in the mft record or when there are so many attributes in the mft record
that, even though all attributes that can be non-resident have been made
non-resident, they still do not fit in the mft record, the larger non-resident
attributes are moved away from the mft record to make space.

This is done by moving whole non-resident attribute header structures to other
mft records and/or by splitting the mapping pairs array of attributes into
several non-resident attribute headers, each containing a chunk of the
original mapping pairs array. These non-resident attribute headers each
describing the same attribute value (but different parts of it) are called
extents.

Attribute list attribute
========================

The attribute list attribute is then added to the base mft record to describe
the location of each attribute.

The attribute list attribute lists all attributes that belong to this mft
record (with the exception of itself). Each entry in the attribute list
describes the attribute listed and in which mft record its attribute header can
be found. For resident attributes this will be the same number as the base mft
record in which the attribute list attribute is located itself. For non-resident
attributes, this will be another mft record, called an extension mft record.
Naturally, all extension mft records point back to their base mft record.

Only one attribute is stored in an extension mft record, even if the attribute
is very small. At least this is the case with Windows NT4 SP6a driver.

Should the mapping pairs array of an attribute become so large as to not fit
into an extenstion mft record, even though its attribute is the only attribute
in this extension record, then the attribute is splitt into several extents.
The first extent starts at vcn 0 and has its lowest vcn value set to zero and
continues up to its highest vcn value. This is determined by splitting up the
mapping pairs array into chunks which just fit into an extension mft record
each. Thus the first mapping pairs array chunk will determine the value of
the highest vcn for the first extent. The attribute list will contain an entry
for this extent. Then, a second extent is created which has its lowest vcn
value set to the highest vcn of the previous extent + 1 and the next chunk of
the mapping pairs array is inserted into this extent. Again, an entry for this
extent is placed into the attribute list, and so on, until the whole
non-resident attribute's mapping pairs array has been accomodated.

Should the attribute list become too big to fit inside the base mft record it
is made non-resident. However, it's maximum value size is determined by the
Windows cache manager to be 256kb (the size of a VACB). Also, the mapping pairs
array of the attribute list has to fit inside the base mft record, as an
attribute list can't be described by itself and attribute lists are not allowed to be nested.

Compressed attributes
=====================

The attribute value of each $DATA attribute can be compressed to save space.
If this is the case, the ATTR_IS_COMPRESSED flag is set.

See the discussion on compression in include/attrib.h for more details.
FIXME: The discussion belongs here! (AIA)

Sparse attributes (NTFS 3.0+)
=============================

The attribute value of each $DATA attribute can be sparse to save space.
If this is the case, the ATTR_IS_SPARSE flag is set. Note, that compressed
attributes are by definition sparse, as well as compressed, without having the
ATTR_IS_SPARSE flag set.

See the discussion on compression in include/attrib.h for more details.
FIXME: The discussion belongs here! (AIA)

Encrypted attributes (NTFS 3.0+)
================================

Since NTFS 3.0, the attribute value of each $DATA attribute can be encrypted,
to protect the contents from spying eyes. If this is the case, the
ATTR_IS_ENCRYPTED flag is set.

FIXME: Write notes on attribute encryption. The discussions from the articles
"Inside the Encrypting File System" in Windows NT magazine (?) are very good
starting points. (AIA)