Crate libdd_profiling_protobuf 

This crate implements Protobuf encoders for profiles which write to a Write. It has encoders for:

• Function
• Label
• Location and Line
• Mapping
• Sample
• ValueType

There is no encoder for a Profile message. It would require borrowing a lot of data, which becomes unwieldy. It also isn’t very compatible with writing a streaming serializer to lower peak memory usage.

Encoding often happens one byte at a time, so a buffered writer should probably be used.

Indices into the string table are represented by StringOffset, which uses a 32-bit number. ID fields are still 64-bit, so the user can control their values, potentially using a 64-bit address for its value.

The types are generally #[repr(C)] so they can be used in FFI one day.

Here is a condensed reference for the parts of protobuf used by profiles:

message    := (tag value)*
tag        := (field << 3) bit-or wire_type;
                encoded as uint32 varint
value      := varint      for wire_type == VARINT,
              len-prefix  for wire_type == LEN,
varint     := int64 | uint64
len-prefix := size (message | string | packed);
                size encoded as int32 varint
string     := valid UTF-8 string;
                max 2GB of bytes
packed     := varint*
                consecutive values of the type specified in `.proto`

A Record represents a Tag and Value pair, where the WireType comes from Value::WIRE_TYPE.

Protos must be smaller than 2 GiB when encoded. Many proto implementations will refuse to encode or decode messages that exceed this limit.

Structs

Function

Represents a function in a profile. Omits the start line because it’s not useful to Datadog right now, so we save the bytes/ops.

Label

A label includes additional context for this sample. It can include things like a thread id, allocation size, etc.

Line

Represents function and line number information. Omits column.

Location

Describes function and line table debug information. This only supports a single Line, whereas protobuf supports zero or more. The is_folding field is not omitted for size/CPU reasons.

Mapping

Represents a mapping in a profile. The boolean fields have been omitted to save bytes/CPU since they are unused in Datadog.

Record

A record is responsible for encoding the field number, wire type and payload. The wire type tells the parser how big the payload after it is. For more details, refer to the Condensed Reference Card.

Sample

Each Sample records values encountered in some program context. The program context is typically a stack trace, perhaps augmented with auxiliary information like the thread-id, some indicator of a higher level request being handled, etc.

StringOffset

Represents an offset into the Profile’s string table. Note that it cannot exceed u32 because an entire protobuf message must not be larger than or equal to 2 GiB. By the time you encode the tag and length prefix for each string, there’s no way to get this many unique-ish strings without first exceeding the protobuf 2 GiB limit.

Tag

A tag is a combination of a wire_type, stored in the least significant three bits, and the field number that is defined in the .proto file.

ValueType

ValueType describes the semantics and measurement units of a value.

Enums

WireType

Represents the wire type for the in-wire protobuf encoding. There are more types than are represented here; these are just the ones used in profiles. See Message Structure for more documentation.

Constants

NO_OPT_ZERO

Intended to be provided to a Record to mean that it shouldn’t optimize for a value of zero. Should be used on fields that should not be zero, such as Mapping.id and for Records which hold arrays, since that would cause the length of the decoded array to change, which is unexpected. Things like sample types shouldn’t get optimized away, since they get used element-wise and this would screw up the pairing.

OPT_ZERO

Intended to be provided to a Record to mean that it should optimize for a value of zero. See also NO_OPT_ZERO.

Traits

Value

A value (or payload) is stored differently depending on the wire_type. In profiles, there two types of payloads: varints and len-prefixed types.