Crate termite_dmg

Expand description

§Termite Data Model Generator

The Termite Data Model Generator is a crate for generating boiler plate code for data models.

§Data Model

The crate consists of two parts, the first is the data model itself. This is preferably imported from a yaml or json file into a DataModel object.

It can also be defined directly in code, however, this is not as readable or easy to write.

A DataModel object consist of header and footer strings and a list of all the data types. The header and footer strings are strings to add to the top and bottom of the generated files like for adding includes in .cpp files. The data types are names and data for the user defined types defined to be generated by termite.

The headers and footers are both using the same format. They are maps of strings with each object in the map referering to the string to add as header/footer for the file refered to by the key. Currently the only supported keys are “cpp-header” and “cpp-source” for the generated “.h” and “.cpp” files.

A namespace can be defined as a list of strings defining the nested namespace, this is only used in c++.

A list of macros can also be defined, these maps a string to a yaml or json structure (string, map or list). Inside any default value definition or a header or footer, if any string is encased in dollar signs then the value is used as the key to find the corresponding macro which is inserted in its place, this also works recursively. A map or list macro can only be inserted if the macro definition is the only thing in the header or default value while a string can be inserted anywhere.

A data type must be given a “name”, type specific “data”, and optionally a “description”. The type specific data defines how the type is implemented and may be types like structs, enums or arrays.

The different types are:

Struct: A normal struct with a number of public fields (like a rust/c++ struct). The “data” must include just a single field called “fields” which is a list of objects describing the fields of the struct. Each object must be given a “name”, “data_type”, and a “default” value description. The “default” description must be either “Required”, “Optional” or “Default”. If it is “Required” then the user must supply the field value when importing a settings file. If it is “Optional” then the internal type of the field in c++ is std::optional<“data_type”> and is set to std::nullopt if the field is not given by the user. If it is Default then it must be followed by a value which is given to the field if the user does not supply a value.

Array: A list of objects of the same type (like a rust/c++ vector). The “data” must include just a single field called “data_type” which is the data type of the elements of the array.

Variant: Can be any of a number of different types, when parsing a value from a user as a variant it will attempt to parse the types from the beginning of the list of types and stops when one is successful (like a c++ variant). The “data” must include just a single field called “data_types” which is a list of all the type names.

Enum: Can be any of a number of predefined enum values, each enum value can optionally wrap a single type to include extra data (like a rust enum). When a user specifies an enum value they must specify the name of the enum value to set along with any data for the wrapped type. Several different enum values can wrap the same type. The “data” must include just a single field called “types” which is a list of all the different enum values. Each element must be given a unique “name”, optionally a “description”, and optionally a wrapped “data_type”.

ConstrainedType: Wraps another type and enforces constraints which only allows parsing if the constraints are respected. When parsing values through a settings file a constrained type does not change the syntax compared to if it was no constraints. The “data” must include two fields. “data_type” which is the data type to be wrapped and “constraints” which is a list of boolean statements which can include the variable “x” where the potential new value is inserted to check if the constraint is true.

§Data Format

No matter what language to generate code for and no matter if the user supplies a YAML or JSON file the user supplied settings file will always have the same format. Only three types of object are used which is common to both YAML and JSON, this is Struct/Map/Object, Array/List/Sequence and Value/String. Beneath is a description of how to write a settings file for each termite type.

Struct: A struct is written as a Map in YAML/JSON. The keys in the Map must be the struct.fields[i].name for the i’th field and the value of the key-value pair must be defined as the type struct.fields[i].data_type. All fields marked as Required must be supplied in the Map while all other fields are not required to be present. Any key in the Map not in the struct.fields is collected in c++ into the field “extra_fields”.

Array: An array is written as a Sequence in YAML/JSON. Each element in the Sequence must be of the type array.data_type

Variant: A variant does not have its own syntax, instead the syntax of one of its types should be used. If multiple of the variant types use the same syntax and has the same valid input then it will be read as the first valid type in the types list.

Enum: An enum has two different syntax. If the enum value does not wrap any type then it is just written as a Value where the Value is the name of the enum type. If the enum value does wrap a type then it is written as a Map with a single key-value pair where the key is the name of the enum type and the value of the key-value pair is the wrapped type.

ConstrainedType: A constrained type also does not have its own syntax, instead it inherits the syntax of its wrapped type as it will just load the wrapped type and then check its constraints afterwards.

§Code Generation

The second part of the Termite crate is generating the code. For now it only supports c++ with the cpp module and JSON schema with the schema module.

To generate the c++ code for the data model, use the .get_header and .get_source methods on the model to generate the strings of the .h and the corresponding .cpp files.

To generate the termite.hpp file use the get_termite_dependency function and save it as “termite.hpp” on the compiler path.

To enable YAML support use the get_yaml_interface function to get the strings of the YAML interface .h and .cpp files. These must be saved on the compiler path as “termite-yaml.h” and “termite-yaml.cpp” respectively.

To enable JSON support use the get_json_interface function to get the strings of the JSON interface .h and .cpp files. These must be saved on the compiler path as “termite-json.h” and “termite-json.cpp” respectively.

To generate the schema generation run the run the .export_schema method to receive the JSON object with the schema.

§Examples

use termite_dmg as termite;
use termite::schema;
use indoc::formatdoc;

let yaml_model = formatdoc!("
  data_types:
  - name: PositiveDouble
    data: !ConstrainedType
      data_type: number
      constraints:
      - x > 0.0
  - name: Point
    description: A point in 2D space
    data: !Struct
      fields:
      - name: x
        data_type: number
        default: !Default 0.0
      - name: y
        data_type: number
        default: !Default $DEFAULT_COORDINATE$
      - name: id
        data_type: integer
        default: Optional
  - name: Size
    description: The size of a box
    data: !Struct
      fields:
      - name: w
        description: The width
        data_type: PositiveDouble
        default: Required
      - name: h
        description: The height
        data_type: PositiveDouble
        default: Required
  - name: SizeVariant
    description: Is either a Size or just a PositiveDouble if it is a square
    data: !Variant
      data_types:
      - PositiveDouble
      - Size
  - name: SizeArray
    data: !Array
      data_type: SizeVariant
  - name: Geometry
    data: !Enum
      types:
      - name: Nothing
        description: No geometry
      - name: Sizes
        description: A number of sizes
        data_type: SizeArray
      - name: Point
        description: A point
        data_type: Point
  - name: NamedGeometry
    data: !Struct
      fields:
      - name: geometry
        description: The geometry data
        data_type: Geometry
        default: !Default
          Point:
            x: 1.0
            id: 0
      - name: name
        description: The name of the geometry
        data_type: string
        default: Required
  headers:
    cpp-header: \"// My .h Header with message: $MESSAGE$\"
    cpp-source: \"// My .cpp Header and this is a dollar sign: $$\"
  footers:
    cpp-header: // My .h Footer
    cpp-source: // My .cpp Footer
  macros:
    DEFAULT_COORDINATE: 0.0
    MESSAGE: This is a macro message
  namespace:
  - my_namespace
");

let model = termite::DataModel::import_yaml(&yaml_model).unwrap();
let cpp_model = termite::cpp::DataModel::new(model.clone()).unwrap();

let termite_hpp = termite::cpp::get_termite_dependency();
let termite_yaml_hpp = termite::cpp::get_yaml_interface();
let model_h = cpp_model.get_header("HEADER_GUARD", 2);
let model_cpp = cpp_model.get_source("model", 2);
let model_schema = model.export_schema("Geometry", "my_schema");

YAML file for loading a my_namespace::PositiveDouble

1.2

YAML file for loading a my_namespace::Point

x: 2.0
y: -3.0
id: 5

Another YAML file for loading a my_namespace::Point

y: -3.0

YAML file for loading a my_namespace::Size

w: 5.2
h: 1.3

YAML file for loading a my_namespace::SizeVariant

1.2

Another YAML file for loading a my_namespace::SizeVariant

w: 5.2
h: 1.3

YAML file for loading a my_namespace::SizeArray

- w: 5.2
  h: 1.3
- 1.2

YAML file for loading a my_namespace::Geometry

Nothing

Another YAML file for loading a my_namespace::Geometry

Sizes:
- w: 5.2
  h: 1.3
- 1.2

Another YAML file for loading a my_namespace::Geometry

Point:
  y: -3.0

YAML file for loading a my_namespace::NamedGeometry

name: A name
geometry:
  Point:
    y: -3.0

Another YAML file for loading a my_namespace::NamedGeometry

name: Another name

§Changelog

§0.6.0

§Major changes

Changed default values to use a serialization model instead of a string to support much more complex default values that works for more than just c++ and are easier to read
Implemented macros to be used with the default values, any value surrounded by $ are interpreted as a macro and are replaced by the contents of that macro, any partial insertion can only be a string insertion and any $$ outside of a macro name is replaced by a single $ in the string

§Minor changes

Fixed minor visual bug in the c++ code generation where a double ;; would be inserted sometimes
Fixed bug where all fields like namespace, footer and header must be included in the data model
Added using statement to termite.hpp for boolean(bool), string(std::string), number(double), and integer(int64_t) to avoid having to add that to all headers
Changed using statements in termite.hpp for default schema types since they sometimes conflicted with using definitions from other libraries

§0.5.0

§Major changes

Added the schema module to generate a JSON schema for a data model

§Minor changes

Changed top-level doc comments to use the README file to make sure the documentation is up to date

§0.4.0

§Major changes

Added helper functions for yaml and json to export to and import from json and yaml strings and files.
Added helper functions to directly import to or export from termite generated types from/to json or yaml nodes, strings or files.

§Minor changes

Fixed bug in yaml and json when attempting to export an empty termite list or map to a json or yaml node. It would export them as null/empty node not as an empty list/map.
Added method to termite::Result called .unwrap which throws an exception if the result is Err, should only be used when you know the result must be Ok.
Updated the README file to be much clearer.

§0.3.0

§Major changes

Added get_json_interface function to add json support for importing data model data. It works just like the get_yaml_interface function.

§Minor changes

§0.2.1

§Major changes

§Minor changes

Split termite-yaml.hpp into a .h and a .cpp file to avoid compilation errors when using multiple compilation units.

§0.2.0

§Major changes

Split the hpp file into a .h and .cpp file to fix linker issues when including in several compilation blocks.
Removed default values from constructor for Structs and instead added static methods for constructors of all the fields with default values.
Added a from_value static template method for termite::Node to convert any data model back into a node.

§Minor changes

Slightly changed the code style of the cpp code.
Fixed bug where default value for a field in a Struct of a type defined in this data model could not comile if namespaces were used.

§0.1.1

§Minor changes

Fixed bug where the namespace was not added to data types in the parsing code when those data types were custom types stopping the c++ code from compiling.
Fixed bug where if a struct field was called x then it could not compile.
Fixed bug where ConstrainedType fields in structs with default values could not compile.

Modules§

cpp: This module handles generation of c++ code to support a data model, it includes the ability to create a header file, (de)serialization and documentation.
schema

Structs§

Array: An array of values of the same data type
ConstrainedType: A constrained type, wraps any other type and adds constraints onto them
DataModel: An entire data model
DataType: Any data type (struct, variant, ect.)
Enum: An enum, includes a number of enum values
EnumType: An enum value, describes a specific enum type
Struct: A struct which has a number of fields
StructField: The data for a single field in a struct
Variant: A variant which can be any of a number of different types, when parsing it will attempt to parse all types from the start until it is successful

Enums§

DataTypeData: Supplies the type specific information for a data type
DefaultType: Describes whether a field is required or optional
SerializationModel: A generic serialization model which can be used to serialize any data model