Module crayon::video

A stateless, layered, multithread video system with OpenGL backends.

Overview and Goals

The management of video effects has become an important topic and key feature of rendering engines. With the increasing number of effects it is not sufficient anymore to only support them, but also to integrate them into the rendering engine in a clean and extensible way.

The goal of this work and simultaneously its main contribution is to design and implement an advanced effects framework. Using this framework it should be easy for further applications to combine several small effects like texture mapping, shading and shadowing in an automated and transparent way and apply them to any 3D model. Additionally, it should be possible to integrate new effects and use the provided framework for rapid prototyping.

Multi Platform

Ideally, crayon should be able to run on macOS, windows and popular mobile-platforms. There still are a huge number of performance and feature limited devices, so this video module will always be limited by lower-end 3D APIs like OpenGL ES2.0.

Stateless Pipeline

Ordinary OpenGL application deals with stateful APIs, which is error-prone. This means whenever you change any state in the API for subsequent draw calls, this state change also affects draw calls submitted at a later point in time. Ideally, submitting a draw call with whatever state we want should not affect any of the other draw calls, even in multi-thread environments.

Modern 3D-APIs like gfx-rs, glium bundles render state and data into a few, precompiled resource objects which are combined into final render pipeline. We should follow the same philosophy.

Multi-thread

In most cases, dividing OpenGL rendering across multiple threads will not result in any performance improvement due the pipeline nature of OpenGL. What we are about to do is actually exploiting parallelism in resource preparation, and provides a set of multi-thread friendly APIs.

The most common solution is by using a double-buffer of commands. This consists of running the renderer backend in a speparate thread, where all draw calls and communication with the OpenGL API are performed. The frontend thread that runs the game logic communicates with the backend renderer via a command double-buffer.

Layered Rendering

Its important to sort video commands (generated by different threads) before submiting them to OpenGL, for the sack of both correctness and performance. For example, to draw transparent objects via blending, we need draw opaque object first, usually from front-to-back, and draw translucents from back-to-front.

The idea here is to assign a integer key to a command which is used for sorting. Depending on where those bits are stored in the integer, you can apply different sorting criteria for the same array of commands, as long as you know how the keys were built.

Resource Objects

Render state and data, which are combined into final render pipeline, are bundled into a few, precompiled resource objects in video module.

All resources types can be created instantly from data in memory, and meshes, textures can also be loaded asynchronously from the filesystem.

And the actual resource objects are usually private and opaque, you will get a Handle immediately for every resource objects you created instead of some kind of reference. Its the unique identifier for the resource, its type-safe and copyable.

When you are done with the created resource objects, its your responsiblity to delete the resource object with Handle to avoid leaks.

For these things loaded from filesystem, it could be safely shared by the Location. We keeps a use-counting internally. It will not be freed really, before all the users deletes its Handle.

Surface Object

Surface object plays as the Layer role we mentioned above, all the commands we submitted in application code is attached to a specific Surface. Commands inside Surface are sorted before submitting to underlying OpenGL.

Surface object also holds references to render target, and wraps rendering operations to it. Likes clearing, offscreen-rendering, MSAA resolve etc..

use crayon::prelude::*;
application::oneshot().unwrap();

// Creates a `SurfaceParams` object.
let mut params = SurfaceParams::default();
/// Sets the attachments of internal frame-buffer. It consists of multiple color attachments
/// and a optional `Depth/DepthStencil` buffer attachment.
///
/// If none attachment is assigned, the default framebuffer generated by the system will be
/// used.
params.set_attachments(&[], None);
// Sets the clear flags for this surface and its underlying framebuffer.
params.set_clear(Color::white(), 1.0, None);

// Creates an surface with `SurfaceParams`.
let surface = video::create_surface(params).unwrap();
// Deletes the surface object.
video::delete_surface(surface);

Shader Object

Shader object is introduced to encapsulate all stateful things we need to configurate video pipeline. This would also enable us to easily change the order of draw calls and get rid of redundant state changes.

use crayon::prelude::*;
application::oneshot().unwrap();

// Declares the uniform variable layouts.
let mut uniforms = UniformVariableLayout::build()
    .with("u_ModelViewMatrix", UniformVariableType::Matrix4f)
    .with("u_MVPMatrix", UniformVariableType::Matrix4f)
    .finish();

// Declares the attributes.
let attributes = AttributeLayout::build()
     .with(Attribute::Position, 3)
     .with(Attribute::Normal, 3)
     .finish();

let mut params = ShaderParams::default();
params.attributes = attributes;
params.uniforms = uniforms;
params.state = RenderState::default();

let vs = "..".into();
let fs = "..".into();

// Create a shader with initial shaders and render state. It encapusulates all the
// informations we need to configurate graphics pipeline before real drawing.
let shader = video::create_shader(params, vs, fs).unwrap();

// Deletes shader object.
video::delete_shader(shader);

Texture Object

A texture object is a container of one or more images. It can be the source of a texture access from a Shader.

use crayon::prelude::*;
application::oneshot().unwrap();

let mut params = TextureParams::default();

// Create a texture object with optional data. You can fill it later with `update_texture`.
let texture = video::create_texture(params, None).unwrap();

// Deletes the texture object.
video::delete_texture(texture);

Compressed Texture Format

TODO: Cube texture. TODO: 3D texture.

Mesh Object

use crayon::prelude::*;
application::oneshot().unwrap();

let mut params = MeshParams::default();

// Create a mesh object with optional data. You can fill it later with `update_mesh`.
let mesh = video::create_mesh(params, None).unwrap();

// Deletes the mesh object.
video::delete_mesh(mesh);

Commands

TODO: CommandBuffer TODO: DrawCommandBuffer

Modules

assets

command

errors

prelude

Constants

MAX_FRAMEBUFFER_ATTACHMENTS

Maximum number of attachments in framebuffer.

MAX_UNIFORM_TEXTURE_SLOTS

Maximum number of textures in shader.

MAX_UNIFORM_VARIABLES

Maximum number of uniform variables in shader.

MAX_VERTEX_ATTRIBUTES

Maximum number of attributes in vertex layout.

Functions

create_mesh

Create a new mesh object.

create_mesh_from

Creates a mesh object from file asynchronously.

create_mesh_from_uuid

Creates a mesh object from file asynchronously.

create_render_texture

Create render texture object, which could be attached with a framebuffer.

create_shader

Create a shader with initial shaders and render state. It encapusulates all the informations we need to configurate graphics pipeline before real drawing.

create_surface

Creates an surface with SurfaceParams.

create_texture

Create texture object. A texture is an image loaded in video memory, which can be sampled in shaders.

create_texture_from

Creates a texture object from file asynchronously.

create_texture_from_uuid

Creates a texture object from file asynchronously.

delete_mesh

Delete mesh object.

delete_render_texture

Delete the render texture object.

delete_shader

Delete shader state object.

delete_surface

Deletes surface object.

delete_texture

Delete the texture object.

mesh

Gets the MeshParams if available.

mesh_state

Get the resource state of specified mesh.

render_texture

Gets the RenderTextureParams if available.

render_texture_state

Get the resource state of specified render texture.

shader

Gets the ShaderParams if available.

shader_state

Get the resource state of specified shader.

surface

Gets the SurfaceParams if available.

surface_state

Get the resource state of specified surface.

texture_state

Get the resource state of specified texture.

update_index_buffer

Update a subset of dynamic index buffer. Use offset specifies the offset into the buffer object’s data store where data replacement will begin, measured in bytes.

update_texture

Update a contiguous subregion of an existing two-dimensional texture object.

update_vertex_buffer

Update a subset of dynamic vertex buffer. Use offset specifies the offset into the buffer object’s data store where data replacement will begin, measured in bytes.