[−][src]Module xnde::fields
fields
Introduction
NDE (Nullsoft Database Engine) field types. I've only implemented the fields needed to read my muisc library. Contact me if you're using this & have fields in your library that aren't handled here.
Discussion
As mentioned in the xnde
docs, an NDE table consists of a collection of records serialized
very compactly. An index is a list of offsets into the data (the .dat
file on disk) at each
each row begins, in the order corresponding to the index. E.g. in my database, index 255
is the primary index and just gives the record offsets in sequential order, while index 0
lists the exact same set of records, but in lexicographical order of the "filename" field.
In my database, the first record contains the columns for the table, and the second the indicies.
Record Formats
Each record consists of a sequence of fields. Each field has the form:
+----+------+----------+------+------+------+ | ID | type | max_size | prev | next | data | +----+------+----------+------+------+------+
ID is a 8-bit unsigned int; this appears to be unique to each column, and serves to map a field to a column in other records. Types as an 8-bit unsigned int describing the field type. In the NDE implementation, these map to a set of contants:
#define FIELD_UNKNOWN 255
enum
{
FIELD_COLUMN = 0,
FIELD_INDEX = 1,
FIELD_REDIRECTOR = 2,
FIELD_STRING = 3,
FIELD_INTEGER = 4,
FIELD_BOOLEAN = 5,
FIELD_BINARY = 6, // max size 65536
FIELD_GUID = 7,
FIELD_PRIVATE = 8,
FIELD_BITMAP = 6,
FIELD_FLOAT = 9,
FIELD_DATETIME = 10,
FIELD_LENGTH = 11,
FIELD_FILENAME = 12,
FIELD_INT64 = 13,
FIELD_BINARY32 = 14, // binary field, but 32bit sizes instead of 16bit
FIELD_INT128 = 15, // mainly for storing MD5 hashes
};
In this crate, they are represented by the enum FieldType
.
max_size
is a 32-bit little-endian unsigned int containing the size of the data
field
(i.e. the field-specific blob after the common header). If fields were guaranteed to be
sequential within a record, this quantity could be computed from next
, so I can only
assume that is not the case generally, even though it was for my database.
Also, note that this is the serialized size on disk; I have seen fields where the field- specific data took up less than this. I suspect, but haven't verified, that this allows the write implementation to update a field whose new serialized representation happens to be smaller by just writing it in place and not updating the rest of the file to "pack" it more tightly.
prev
is a 32-bit little-endian unsigned int containing the offset of the previous field in
this record, and next
is the same giving the offset of the next. Not all columns need appear
in each record, and the record length is nowhere written down; the reader must simply read one
field after another until encountering one whose next
field is zero. As an aside, this
strongly suggests a buffered read implementation, which the NDE uses.
Field Formats
The following diagrams display field layouts after the common field header.
Column
+----------+------------------+--------------+--------------------+ | type: u8 | unique index: i8 | name len: u8 | name: ASCII string | +----------+------------------+--------------+--------------------+
The column name is a length-prefixed ASCII string (i.e. no trailing nil).
Filename & String
+----+------+ | cb | text | +----+------+
cb
is a sixteen-bit, little-endian unsigned integer containing the number of bytes in the
filename or string. The text may be UTF-16 encoded; in that case we expect a BOM. Else the
reference implementation simply copies the bytes; this implementation assumes UTF-8. Note that
the string is not null-terminated.
Index
+-----+------+----+------+ | pos | type | cb | name | +-----+------+----+------+
pos
is a 32-bit, little endian signed int. I never figured out what it does. type' is also 32-bit, LE, signed int, and refers to the type of field found in this column.
cbis a 32-bit, LE unsigned int describing the length of the
name` field, which is the ASCII text
of the filter name.
Int64
+-----+ | val | +-----+
A 64-bit, little-endian, signed integer.
Datetime, Integer, Length
+-------+ | value | +-------+
value
is a 32-bit little-endian integer. I still haven't unravelled how to interpret it in all
cases, but it is a signed integer (i.e. not a simple Unix-style "seconds-since-epoch" value
for time, or seconds for length).
Structs
ColumnField | NDE FIELD_COLUMN |
DatetimeField | NDE FIELD_DATETIME |
Error | |
FilenameField | NDE FIELD_FILENAME |
IndexField | NDE FIELD_INDEX |
Int64Field | NDE FIELD_INT64 |
IntegerField | NDE FIELD_INTEGER |
LengthField | NDE FIELD_LENGTH |
NdeFieldBase | Common NDE Field attributes: id, next-field, prev-field |
StringField | NDE FIELD_STRING |
UnsupportedNdeField | An NDE Field which we do not know how to parse |
Enums
Cause | |
FieldType | NDE field types, maintaining the associated C numeric constants |
FieldValue |
Traits
NdeField | Common NDE Field behavior |
Functions
field_factory |
Type Definitions
Result |