Expand description
§Done
Changed to SInt and UInt in ele type
Added BufferDescriptor
§TODO
Decided not to replace ByteBuffer with T:AsRef<u8>; this would require clients to still cast Vec<> etc to a something-of-u8
Need to make BufferDataAccessor refer to a BufferDescriptor
Need to add BufferIndexAccessor
Make Vertices have an option
§3D Model library
This library provides structures and functions to support simple and complex 3D objects in a reasonably performant system. Its use cases include 3D modeling tools, games, and 3D user interfaces.
The object model is derived from the Khronos glTF 3D model/scene description (https://github.com/KhronosGroup/glTF), without explicit support for animation or cameras.
§Overview of the model
The 3D model library is designed to provide for the description of 3D objects with simple or sophisticated specifications of their vertices; with materials that might be simple colors or complete PRBS specifications, and so on.
Once models have been described they can be turned into GL-specific instantiable structures. It is quite common for graphics libraries to convert some external data form and to allocate internal memory for rendering - so that the object description buffers used initially are no longer required and their memory can be freed.
When an ‘instantiable’ object exists it can be rendered many times in a single scene, with different transformations and skeleon poses.
Hence this library provides for Instantiable object creation that first requires memory buffers to be allocated, descriptions of an object created; from this Object the Instantiable is made for a particular graphics library context - allowing the freeing of those first memory buffers. This Instantiable can then be instanced a number of times, and these Instance each have their own Transformation and SkeletonPose.
§Backends
The model of programming and drawing for OpenGL and WebGL is to provide buffers to the GPU that separately contain vertex data and index data; the shader program is compiled and linked from separate sources; the individual vertex attributes used by the shader program are named in the program, and need to be mapped to the contents of the vertex buffers that need to be drawn. The vertex buffers thus need a mapping concept for an object’s buffers that need to be drawn; this is the Vertex-Attribute-Object (VAO). Different objects can have different vertex attribute layouts within their buffers.
Hence OpenGL/WebGL have programs that have program-specified named variables of vertex attributes; buffers for different objects can have different vertex layouts; VAOs map parts of buffers to the vertex attributes.
WebGPU follows a different model; the shader program is a single source code (for vertex and fragment), which is compiled into a shader module; the shader module can be used in multiple RenderPipelines; each RenderPipeline specifies the layouts of the buffers, which maps the layout of the vertices in the buffers that are provided to it at draw time to the locations that the shader module expects. There is no need for a VAO; the eventual draw call uses buffers that are bound (prior to the draw; the operation is set pipeline, set buffers, draw).
§Object creation
The first step in using the library is to create the required Object. This requires some binary data buffer(s) containing float vertex data (position, and possible normals, texture coordinates, tangents, relevant bones and weights, and so on); also arrays of unsigned int vertex indices, that indicate how the vertices form strips of triangles, or lines, etc. In some applications this binary data might come from a large file, containing the data for many objecs; as such, different portions of the binary data contain different parts of the object data.
The library requries that a data buffer support the ByteBuffer trait, and then that data buffer can have BufferData views created onto it - portions of the data buffer. For particular vertex data a subset of BufferData is used to create BufferDataAccessors and for indices a BufferIndexAccessor.
A number of BufferData are pulled together to produce a Vertices object - this might describe all the points and drawing indices for a complete object. Subsets of the Vertices object are combined with a Material to help describe a set of elements to render - this might be a TriangleStrip, for example - this is known as a Primitive
A Component of an object is a list of Primitive and a Transformation (the list of Primitive is a Mesh)
An Object has a hierarchy of Component, and optionally a Skeleton; it also must keep references to the data that it uses - so it contains arrays of Vertices and Material references.
Note that the Vertices used by one object may be used by others; as such one might load a single data file that contains many objects for a game, and the objects all refer to the same buffers and even Material and Vertices.
§Buffers
Underlying the data model is the ByteBuffer trait - any data that is used for the models must support this trait, and implementations are provided for slice <> and for Vec<>.
§BufferData
A type that borrows a sub slice ofu8, using an explicit offset and length, and which might have a client reference (e.g. an OpenGL GlBuffer handle). It is similar to a Gltf BufferView, without a ‘stride’.
The base concept for model BufferData is that it is an immutable borrow of a portion of some model data buffer of a type that supports the ByteBuffer trait; the data internally may be floats, ints, etc, or combinations thereof - from which one creates BufferDataAccessors, or BufferIndexAccessors when used as model indices. So it can be the complete data for a whole set of models.
Each BufferData has a related client element (a Renderable::Buffer) which is created when an Object has its client structures created; this may be an Rc of an OpenGL buffer, if the client is an OpenGL renderer.
Each BufferData is use through one or more BufferDataAccessor or BufferIndexAccessor.
§BufferDataAccessor, BufferIndexAccessor
A BufferDataAccessor is an immutable reference to a subset of a BufferData. A BufferDataAccessor may, for example, be the vertex positions for one or more models; it may be texture coordinates; and so on. The BufferData corresponds on the OpenGL side to an ARRAY_BUFFER or (for BufferIndexAccessor) an ELEMENT_ARRAY_BUFFER; hence it expects to have a VBO associated with it.
The BufferDataAccessor and BufferIndexAccessor are similar to a glTF Accessor.
Each BufferDataAccessor has a related client element (a [Renderable::View]) which is created when an Object has its client structures created; this may be the data indicating the subset of the Renderable::Buffer that the view refers to, or perhaps a client buffer of its own.
A set of BufferDataAccessors are borrowed to describe Vertices, each BufferDataAccessor providing one piece of vertex information (position or normal). A single BufferDataAccessor may be used by more than one Vertices object.
A BufferIndexAccessor may be borrowed to describe the indices for a Vertices.
§Vertices
The Vertices type borrows at least one [BufferAccessor] for a vertex indices buffer, and at least one [BufferAccessor] for positions of the vertices; in addition it borrows more [BufferAccessor], one for each attribute VertexAttr that is part of a mesh or set of meshes.
The Vertices object should be considered to be a complete descriptor of a model or set of models within one or more ByteBuffer. In OpenGL a Vertices object becomes a set of OpenGL Buffers (and subsets thereof) and for a particular shader class it can be bound into a VAO.
A Vertices object is a set of related [BufferAccessor]s, with at least a view for indices and a view for vertex positions; it may have more views for additional attributes. It has a lifetime that is no longer than that of the BufferData from which the [BufferAccessor]s are made.
A Renderable::Vertices can be constructed from a Vertices; this is a renderer-specific vertices instance that replaces the use of [BufferAccessor]s with the underlying client types.
§Skeleton and posing
A Skeleton consists of a (possibly multi-rooted) hierarchy of Bones. Each bone has a Transformation, which is the mapping from the coordinate space of its parent to the coordinate space of the bone itself.
Each Bone has a ‘matrix_index’ which indicates which ‘Joints’ matrix the bone is referred to by any ’joint’s attribute entry in a mesh.
An object instance will have a SkeletonPose associated with it; this allows the object contents to be rendered with adjustments to the model, such as to make it appear to walk. A SkeletonPose is an array of BonePose which reflect the associated Bones in the skeleton; each has an associated posed Transformation.
The SkeletonPose can be traversed and for each posed bone an appropriate mesh-to-model-space matrix can be generated; if a mesh is annotated with bone weights that sum to 1 then a mesh vertex coordinate can be converted to a posed-model coordinate by summing the mesh-to-model-space matrices of the bones times their weights times the mesh vertex coordinate.
A Skeleton is similar to a skin in GLTF.
/// Each bone has a transformation with respect to its parent that is /// a translation (its origin relative to its parent origin), scale /// (in each direction, although a common scale for each coordinates /// is best), and an orientation of its contents provided by a /// quaternion (as a rotation). /// /// A point in this bone’s space is then translate(rotate(scale(pt))) /// in its parent’s space. The bone’s children start with this /// transformation too. /// /// From this the bone has a local bone-to-parent transform matrix /// and it has a local parent-to-bone transform matrix /// /// At rest (where a mesh is skinned) there are two rest matrix variants /// Hence bone_relative = ptb * parent_relative /// /// The skinned mesh has points that are parent relative, so /// animated_parent_relative(t) = btp(t) * ptb * parent_relative(skinned) /// /// For a chain of bones Root -> A -> B -> C: /// bone_relative = C.ptb * B.ptb * A.ptb * mesh /// root = A.btp * B.btp * C.btp * C_bone_relative /// animated(t) = A.btp(t) * B.btp(t) * C.btp(t) * C.ptb * B.ptb * A.ptb * mesh
§Textxures
A texture is a 1D, 2D or 3D object that is indexed by a shader to provide one of a scalar, vec2, vec3 or vec4 of byte, short, integer, or float.
§Materials
Materials are types that have the Material trait, and which have the lifetime of the BufferData of the object they belong to; this is because they may contain textures. As such they have an associated Renderable::Material type, which has a lifetime as defined by the Renderable.
Material is a trait that must be supported by materials, which thus
permits different abstract shading models to be used. It has a
MaterialClient parameter, which
Example Material instances are:
A Material has a make_renderable() method that makes it renderable?
§Primitive
A Primitive is a small structure that uses a single subset of Vertices with a Material wih a drawing type (such as TriangleStrip). It does not contain direct references to the vertices or material; rather it uses an index within the arrays of Vertices or Material held by an Object.
A Primitive contains:
-
a Material (from an index within the Object to which the primitive mesh belongs)
-
a set of Vertices - the attributes required by the Mesh and a set of indices, a subset of which are used by the Primitive (from an index within the Object to which the mesh belongs)
-
a drawable element type (PrimitiveType such as
TriangleStrip) -
an index offset (within the Vertices indices)
-
a number of indices
A Primitive does not contain any transformation information - all the Primitive that belong to a Mesh have the same transformation.
§Mesh
A Mesh is an array of Primitive; this is just a way to combine sets of drawn elements, all using the same transformation (other than bone poses).
A mesh is part of a Component that is part of an Object.
§Component of an Object
A Component is part of the hierarchy of an Object and has no meaning without it; the indices and materials used in the Component are provided by the Object. The Component has a Transformation (relative to its parent) and a Mesh.
Note that a hierarchy of object Components is implicitly
renderable as it contains only indices, not actual references to
[BufferAccessor] data structures.
A hierarchy of object Components can be reduced to a RenderRecipe; this is an array of:
-
transformation matrix
-
material index (in a Primitive)
-
vertices index (in a Primitive)
-
drawable element type (in a Primitive)
-
index offset (in a Primitive)
-
index count (in a Primitive)
§Object
An Object has an array of Vertices and [Materials] references that its meshes use; both of these will end up being mapped to arrays of graphic library client handles on a one-to-one basis (each Vertices in the array becomes a graphics library vertices client, for example).
The Object then has a hierarchy of Component; this describes everything that it takes to draw the object.
Additionally the Object has an optional Skeleton.
All of the data from an Object can be used to create an Instantiable; this latter does not refer to any of he data buffers (such as the Vertices) and so the original byte buffers can be dropped once an Instantiable exists - all the data will be in the graphics library.
§Instantiable objects
A 3D model Object consists of:
-
a hierarchy of Components
-
a Skeleton
-
an array of Vertices; each of these is a set of indices within a BufferData and attribute [BufferAccessor]s.
-
an array of Material
Such an object may have a plurality of render views created for it, for use with different visualizers (in OpenGL these could be different shaders, for example).
An object can be turned in to a renderable object within a
Renderable::Context using the into_instantiable method. Once created
(unless the renderable context requires it) the object can be dropped.
The Instantiable is created within a specific renderable context. For simple graphics libraries this probably means that instances of the instantiated object are to be wih a single shader program, whose attribute layout an uniforms is known ahead of time. As such the renerable context might generate a single VAO for the instantiable with appropriate buffer bindings, at the into_instantiable invocation. For more complex applications, where an object may be rendered with more than one layout of attributes, with many different shader program classes, a VAO could be generated per shader program class and (e.g.) a ‘bind vertex buffers’ for the object can be invoked within a VAO prior to the rendering of a number of the instances of the object with the shader program (each presumably with its own ‘uniform’ settings!).
An Instantiable can then be drawn by (theoretically, and given a particular SkeletonPose):
-
Generating the BonePose mesh-to-model-space matrices for each bone in the Skeleton
-
Traversing the hierarchy, keeping a current node Transformation in hand
-
Apply the node’s Transformation
-
Render the node’s Primitives using the Objects material at the correct index, with the Instantiable associated with the Vertices index of the mesh
§Instantiated objects
An instantiated object is created by instantiating an Instantiable.
The Instance has a Transformation, a SkeletonPose, and a set of Material overrides; the Material overrides are an array of optional materials.
For efficient rendering the object instance includes an array of the instance’s SkeletonPose matrices plus the base instance Transformation matrix.
§Rendering an instance
A Vertices object is then used by a number of Primitives; each of these borrows the Vertices object, and it owns an array of Drawables. Each Drawable is an OpenGL element type (such as TriangleStrip), a number of indices, and an indication as to which index within the Vertices object to use as the first index. Each Primitive has a single Material associated with it.
An array of Primitive objects is owned by a Mesh object, which corresponds to the glTF Mesh - hence the Primitive object here corresponds to a glTF Primitive within a glTF Mesh. A Mesh might correspond to a table leg or the table top in a model.
A number of Mesh objects are borrowed to form Object Nodes; each Node has its own Transformation that is applied to its Mesh. Hence a table can consist of four Object Nodes that borrow the same table leg Mesh, and a further Object Node that is the table top. A Node may also have a BoneSet associated with it, to provide for skinned objects.
An Object Node forms part of an Object’s Hierarchy - hence the Object Nodes are owned by the Object. An Object, then, can be a complete table, for example. It might also be a posable robot; in this case one would expect the top level node to have a BoneSet, and there to perhaps be Object Nodes for the body, head, arms and legs, perhaps sharing the same Mesh for the two arms and anoter Mesh for the two legs.
It is worth noting at this point that the lifetime of an Object must be no more than the lifetime of the data buffer containing its data; even though the Object may be passed in to a GPU memory, the data used for building the object then not being required by the CPU (using STATIC_DRAW). It is, then, clearly useful to consider the Object as a model construction type, not necessarily the model draw type.
When it comes to rendering an Object, this requires a Shader. The data required by a Shader to render an Object depends not just on the Object but also on the Shader’s capabilities (such as does it utilize vertex tangents). However, there is some data that is common for all the Shaders that might render a single Object instance - such as the bone poses for the object, and the mesh matrices.
Hence an Object must have two type created for it prior to rendering. The first is a drawable::Instantatiable. This is a drawable object that in itself may have instantiations. The drawable::Instantiable contains an owned copy of the BoneSet for the object, and any transformation data required by the meshes for drawing (given each object node has its own transformation). The drawable::Instantiable is created from the object using its create_instantiable method.
The second type required for rendering is a shader::Instantiable. This is used by binding the data required for a shader for an object to a VAO (Vertex Attribute Object) for the shader; this VAO is used in the rendering of any instances of the shader::Instantiable. The shader::Instantiable borrows the drawable::Instantiable created for the Object; it is created using the Object’s bind_shader method.
With STATIC_DRAW the lifetime of the shader::Instantiable can be shorter than that of the Object data - it can have a lifetime of the rendering.
Once an Object has had all of its drawable::Instantiable and shader::Instantiable types created, the Object may be dropped.
A renderable instance of an object then requires a drawable::Instance
to be created from the drawable::Instantiable; this instance has its
own transformation and bone poses, and it borrows the
drawable::Instantiable. The Instance can be rendered with a particular
shader using that shader::Instantiable’s gl_draw method, which takes
a reference to the Instance. This method then has access to all the
matrices required for the mesh, for the posed bones, and so on.
A Shader is created using standard OpenGL calls. It must have the ShaderClass trait.
An instantiable model consists of abstract mesh data and poses of the elements within its skeleton, if required.
Multiple instances of an instantiable model can be created, each with its own set of poses.
The Hierarchy module provides for a hierarchy of owned elements which are stored in an array inside the Hierachy structure; the rerlationship between nodes in the hierarchy are handled by indices into this array. The Hierarchy is designed to be created, and then immutably interrogated - although the immutability refers to the hierarchy and the node array, not the contents of the nodes - the node content may be updated at will.
§Graphics libraries
In OpenGL we have
VAO = list of binding from VAO attr number to a buffer; it also has an ‘elemeent array buffer’ that is the buffer of indices. Note a buffer here is an offset/stride of a gl buffer.
The VAO is specific to the shader class, as it uses the attribute locations of the shader
A VAO can have the attribute bbuffers bound iini one go with glBindVertexBuffers (or glVertexArrayVertexBuffers to specify the vao without binding it first),
The index buffer can be bound witth glBindElementBuffer.
In theory a single VAO can work for many objects with one shader class as the attribute layour is specific o the shader class. This requires all its buffers to be rebound and the element buffer to be rebound. So it keeps some of the VAO.
Uniforms must be set after useProgram
A VBO glBuffer can be a BufferData; an Accessor is the glBuffer with offset and stride.
§OpenGL 4.0
Program has id; attribute list of (AttribLocation, VertexAttr); uniform list of (UniformLocation, UniformId).
UniformId is either ViewMatrix, ModelMatrix, etc, User(x), or Buffer(x)
§Examples
use model3d::{BufferAccessor, MaterialAspect}; use model3d::example_client::Renderable;
§To do
Optimize primitive to fit within 32 bytes
Make Buffer have a client ‘reproduce me’ element so that if it comes from a file that file could be reloaded if required. This would allow the GPU data for an instantiable to be dropped and reloaded, if the appropriate client code is written. The Buffer would require this element at creation time so that its create client method could could capture it.
Add a String to each component, and extract that for each root component in the hierarchy Maybe have an ‘extract component by name’ from object that creates an Instantiable (requires there to be no skeleton for now)
Make only part of an instantiable be drawn (have a Vec of RenderRecipes, one per component in the root by default)
!
Re-exports§
pub use example_objects::ExampleVertices;
Modules§
- example_
client - Example client
- example_
objects - This module contains example objects
- hierarchy
- This module provides a hierarchy of nodes and iterators over them
Structs§
- Base
Material - Base material that provides simply color and constant metallicness/roughness
- Bone
- Each bone has a transformation with respect to its parent that is a translation (its origin relative to its parent origin), scale (in each direction, although a common scale for each coordinates is best), and an orientation of its contents provided by a quaternion (as a rotation).
- Bone
Pose - A pose of a Bone, referring to it so that many poses can use the same Bone.
- Buffer
Data - A data buffer for use with vertex data. It may be indices or vertex coordinates etc.
- Buffer
Data Accessor - A subset of a
BufferData, used for vertex attributes; hence for use in a vertex attribute pointer. - Buffer
Descriptor - A descriptor of a subset of a
BufferData, used for vertex attributes; hence for use in a vertex attribute pointer. - Buffer
Index Accessor - A subset of a
BufferData, used for vertex attributes; hence for use in a vertex attribute pointer. - Component
- A Component of an object’s hierarchy
- Instance
- A drawable::Instance contains the instance data for an instance of a drawable::Instantiable
- Instantiable
- An Instantiable is a type that is related to a set of Mesh data, which can be instanced for different drawable::Instance’s
- Material
Base Data - The basic data for a material; the most simple material is actually just RGB, but to keep the system simple the BaseData includes an alpha, metallicness and roughness.
- Mesh
- A Mesh provides an array of primitives, that is notionally drawn from first to last
- Object
- A hierarchy of ObjectNode’s
- PbrMaterial
- A physically-based rendered material with full set of textures
- Primitive
- A primitive consisting of a material and a subset of vertices using a particular range of indices
- Render
Recipe - A RenderRecipe for a hierarchy of components
- Short
Index - An optional index used within the model system, that is up to 65000
- Skeleton
- A set of related bones, with one or more roots
- Skeleton
Pose - A pose structure for a complete Skeleton
- Texture
- A texture is managed by the library as a byte slice which has up to three dimensions - minimally a width, with a 2D texture having a non-zero height, and a 3D texture with a non-zero depth
- Transformation
- A transformation corresponds to a translation of a rotation of a scaling
- Vertex
Desc - A descriptor of the contents of one aspect of data for a Vertex
- Vertices
- A set of vertices using one or more crate::BufferData through [BufferAccessor]s.
Enums§
- Buffer
Element Type - The type of an element in a buffer
- Material
Aspect - The aspect of a material
- Primitive
Type - Type of a primitive
- Vertex
Attr - A VertexAttr is a possible vertex attribute that can be used by a renderer; a vertex always has a position attribute, but additional attributes may or maynot be provided by a model
Traits§
- Accessor
Client - Trait supported by a BufferAccessor client
- Buffer
Client - Trait supported by a BufferData client
- Byte
Buffer - A trait for all types that are to be used as sources of data for buffers of, e.g. vertex data, indices, etc
- Descriptor
Client - Trait supported by a BufferDescriptor client
- Material
- A Material provides means to access the data for a material, be it simple of full PBR. A fragment shader may require some aspects of a material to be provided to it for rendering, and this API allows that information to be gathered from any kind of material
- Material
Client - Trait supported by a material client
- Renderable
- The Renderable trait must be implemented by a type that is a client of the 3D model system. It provides associated types for a renderable context (this might be a particular shader program within a OpenGL context, for example), and then its own structures that are used to hold BufferData, textures, materials, and sets of renderable Vertices.
- Texture
Client - The trait that must be supported by a client texture
- Vertices
Client - The trait that must be supported by a client vertices