Struct EntityDb 

pub struct EntityDb {
    pub data_source: Option<LogSource>,
    pub estimated_application_overhead_bytes: Option<u64>,
    /* private fields */
}

An in-memory database built from a stream of LogMsges.

NOTE: all mutation is to be done via public functions!

Fields

data_source: Option<LogSource>

Set by whomever created this EntityDb.

Clones of an EntityDb gets a None source.

estimated_application_overhead_bytes: Option<u64>

How much RAM the whole application uses beyond the raw physical chunks in this recording.

This is estimated by the viewer after a GC pass, when there is only one recording loaded. Includes primary and secondary indices, (purged) caches, fonts, icons, and other overhead.

Implementations

impl EntityDb

pub fn new(store_id: StoreId) -> Self

Secondary viewer indexes are enabled by default.

Use Self::with_store_config for more control.

pub fn with_store_config( store_id: StoreId, enable_viewer_indexes: bool, store_config: ChunkStoreConfig, ) -> Self

pub fn tree(&self) -> &EntityTree

pub fn format_with_components(&self) -> String

Formats the entity tree into a human-readable text representation with component schema information.

pub fn storage_engine(&self) -> StorageEngineReadGuard<'_>

Returns a read-only guard to the backing StorageEngine.

pub fn num_physical_chunks(&self) -> usize

Number of physical chunks currently loaded.

pub fn byte_size_of_physical_chunks(&self) -> u64

The total size of the physical chunks currently loaded in memory.

This is the evictable data — the raw arrow chunk data that can be GC’d.

pub fn rrd_manifest_index_mut_and_storage_engine( &mut self, ) -> (&mut RrdManifestIndex, StorageEngineReadGuard<'_>)

pub unsafe fn storage_engine_raw(&self) -> &StorageEngine

Returns a reference to the backing StorageEngine.

This can be used to obtain a clone of the StorageEngine.

Safety

Trying to lock the StorageEngine (whether read or write) while the computation of a viewer’s frame is already in progress will lead to data inconsistencies, livelocks and deadlocks. The viewer runs a synchronous work-stealing scheduler (rayon) as well as an asynchronous one (tokio): when and where locks are taken is entirely non-deterministic (even unwanted reentrancy is a possibility).

Don’t use this unless you know what you’re doing. Use Self::storage_engine instead.

pub fn storage_engine_arc(&self) -> StorageEngineArcReadGuard

Returns a read-only guard to the backing StorageEngine.

That guard can be cloned at will and has a static lifetime.

It is not possible to insert any more data in this EntityDb until the returned guard, and any clones, have been dropped.

pub fn rrd_manifest_index(&self) -> &RrdManifestIndex

pub fn rrd_manifest_index_mut(&mut self) -> &mut RrdManifestIndex

pub fn can_fetch_chunks_from_redap(&self) -> bool

Are we connected to redap, and can fetch missing chunks?

pub fn is_currently_downloading_manifest(&self) -> bool

Are we currently in the process of downloading the RRD Manifest?

pub fn is_buffering(&self) -> bool

True if we’re are currently waiting for necessary data to be loaded before we can show it.

pub fn store_info_msg(&self) -> Option<&SetStoreInfo>

pub fn store_info(&self) -> Option<&StoreInfo>

pub fn application_id(&self) -> &ApplicationId

pub fn recording_id(&self) -> &RecordingId

pub fn store_kind(&self) -> StoreKind

pub fn store_id(&self) -> &StoreId

pub fn redap_uri(&self) -> Option<&DatasetSegmentUri>

What redap URI does this thing live on?

pub fn store_class(&self) -> EntityDbClass<'_>

Returns the EntityDbClass of this entity db.

pub fn recording_info_property<C: Component>( &self, component: ComponentIdentifier, ) -> Option<C>

Read one of the built-in RecordingInfo properties.

pub fn set_recording_property<Component: Component>( &mut self, entity_path: EntityPath, component_descr: ComponentDescriptor, value: &Component, ) -> Result<(), Error>

Use can use this both for setting the built-in RecordingInfo components, and for setting custom properties on the recording.

pub fn timeline_type(&self, timeline_name: &TimelineName) -> TimeType

pub fn latest_at( &self, query: &LatestAtQuery, entity_path: &EntityPath, components: impl IntoIterator<Item = ComponentIdentifier>, ) -> LatestAtResults

Queries for the given components using latest-at semantics.

See re_query::LatestAtResults for more information about how to handle the results.

This is a cached API – data will be lazily cached upon access.

pub fn latest_at_component<C: Component>( &self, entity_path: &EntityPath, query: &LatestAtQuery, component: ComponentIdentifier, ) -> Option<((TimeInt, RowId), C)>

Get the latest index and value for a given dense re_types_core::Component.

This assumes that the row we get from the store contains at most one instance for this component; it will log a warning otherwise.

This should only be used for “mono-components” such as Transform and Tensor.

This is a best-effort helper, it will merely log errors on failure.

pub fn latest_at_component_quiet<C: Component>( &self, entity_path: &EntityPath, query: &LatestAtQuery, component: ComponentIdentifier, ) -> Option<((TimeInt, RowId), C)>

Get the latest index and value for a given dense re_types_core::Component.

This assumes that the row we get from the store contains at most one instance for this component; it will log a warning otherwise.

This should only be used for “mono-components” such as Transform and Tensor.

This is a best-effort helper, and will quietly swallow any errors.

pub fn latest_at_component_at_closest_ancestor<C: Component>( &self, entity_path: &EntityPath, query: &LatestAtQuery, component: ComponentIdentifier, ) -> Option<(EntityPath, (TimeInt, RowId), C)>

pub fn can_load_more(&self) -> bool

Can we load this recording more right now?

pub fn has_fully_loaded_query( &self, entity_path: &EntityPath, component: ComponentIdentifier, query: &LatestAtQuery, ) -> bool

Check if we have all loaded chunk for the given entity and component at query.at().

pub fn cloned_from(&self) -> Option<&StoreId>

If this entity db is the result of a clone, which store was it cloned from?

A cloned store always gets a new unique ID.

We currently only use entity db cloning for blueprints: when we activate a default blueprint that was received on the wire (e.g. from a recording), we clone it and make the clone the active blueprint. This means all active blueprints are clones.

pub fn timelines(&self) -> BTreeMap<TimelineName, Timeline>

pub fn timeline_histograms(&self) -> &TimeHistogramPerTimeline

When do we have data on each timeline?

pub fn time_range_for( &self, timeline: &TimelineName, ) -> Option<AbsoluteTimeRange>

Returns the time range of data on the given timeline, ignoring any static times.

pub fn time_histogram(&self, timeline: &TimelineName) -> Option<&TimeHistogram>

Histogram of all events on the timeeline, of all entities.

pub fn generation(&self) -> ChunkStoreGeneration

Return the current ChunkStoreGeneration. This can be used to determine whether the database has been modified since the last time it was queried.

pub fn last_modified_at(&self) -> Instant

pub fn latest_row_id(&self) -> Option<RowId>

The highest RowId in the store, which corresponds to the last edit time. Ignores deletions.

pub fn sorted_entity_paths(&self) -> impl Iterator<Item = &EntityPath>

A sorted list of all the entity paths in this database.

pub fn ingestion_stats(&self) -> &IngestionStatistics

pub fn entity_path_from_hash( &self, entity_path_hash: &EntityPathHash, ) -> Option<&EntityPath>

pub fn is_known_entity(&self, entity_path: &EntityPath) -> bool

Returns true also for entities higher up in the hierarchy.

pub fn is_logged_entity(&self, entity_path: &EntityPath) -> bool

If you log world/points, then that is a logged entity, but world is not, unless you log something to world too.

pub fn add_rrd_manifest_message(&mut self, rrd_manifest: Arc<RrdManifest>)

pub fn add_log_msg( &mut self, msg: &LogMsg, ) -> Result<Vec<ChunkStoreEvent>, Error>

Insert new data into the store.

pub fn add_record_batch( &mut self, record_batch: &RecordBatch, ) -> Result<Vec<ChunkStoreEvent>, Error>

Insert a chunk (encoded as a record batch) into the store.

pub fn add_chunk( &mut self, chunk: &Arc<Chunk>, ) -> Result<Vec<ChunkStoreEvent>, Error>

Insert new data into the store.

pub fn set_store_info(&mut self, store_info: SetStoreInfo)

pub fn gc_with_target( &mut self, target: GarbageCollectionTarget, time_cursor: Option<TimelinePoint>, ) -> Vec<ChunkStoreEvent>

Free up some RAM by forgetting the older parts of all timelines.

pub fn gc(&self, gc_options: &GarbageCollectionOptions) -> Vec<ChunkStoreEvent>

The chunk store events are not handled within this function!

pub fn drop_time_range( &mut self, timeline: &TimelineName, drop_range: AbsoluteTimeRange, ) -> Vec<ChunkStoreEvent>

Drop all events in the given time range from the given timeline.

Used to implement undo (erase the last event from the blueprint db).

pub fn drop_entity_path(&mut self, entity_path: &EntityPath)

Unconditionally drops all the data for a given EntityPath .

This is not recursive. Children of this entity will not be affected.

To drop the entire subtree below an entity, see: Self::drop_entity_path_recursive.

pub fn drop_entity_path_recursive(&mut self, entity_path: &EntityPath)

Unconditionally drops all the data for a given EntityPath and all its children.

pub fn to_messages( &self, time_selection: Option<(TimelineName, AbsoluteTimeRangeF)>, ) -> impl Iterator<Item = ChunkResult<LogMsg>> + '_

Export the contents of the current database to a sequence of messages.

If time_selection is specified, then only data for that specific timeline over that specific time range will be accounted for.

pub fn clone_with_new_id(&self, new_id: StoreId) -> Result<Self, Error>

Make a clone of this EntityDb, assigning it a new StoreId.

impl EntityDb

Stats

pub fn subtree_stats_static( &self, engine: &StorageEngineReadGuard<'_>, entity_path: &EntityPath, ) -> ChunkStoreChunkStats

Returns the stats for the static store of the entity and all its children, recursively.

This excludes temporal data.

pub fn subtree_stats_on_timeline( &self, engine: &StorageEngineReadGuard<'_>, entity_path: &EntityPath, timeline: &TimelineName, ) -> ChunkStoreChunkStats

Returns the stats for the entity and all its children on the given timeline, recursively.

This excludes static data.

pub fn subtree_has_data_on_timeline( &self, engine: &StorageEngineReadGuard<'_>, timeline: &TimelineName, entity_path: &EntityPath, ) -> bool

Returns true if an entity or any of its children have any data on the given timeline.

This includes static data.

pub fn subtree_has_temporal_data_on_timeline( &self, engine: &StorageEngineReadGuard<'_>, timeline: &TimelineName, entity_path: &EntityPath, ) -> bool

Returns true if an entity or any of its children have any temporal data on the given timeline.

This ignores static data.

pub fn entity_has_temporal_data_on_timeline( &self, engine: &StorageEngineReadGuard<'_>, timeline: &TimelineName, entity_path: &EntityPath, ) -> bool

Returns true if an entity has any temporal data on the given timeline.

This ignores static data.

pub fn entity_has_temporal_data_on_timeline_for_component( &self, engine: &StorageEngineReadGuard<'_>, timeline: &TimelineName, entity_path: &EntityPath, component: ComponentIdentifier, ) -> bool

Returns true if an entity has data for the given component on the given timeline at any point in time.

This ignores static data. This is a more fine grained version of Self::entity_has_temporal_data_on_timeline.

Trait Implementations

impl Clone for EntityDb

fn clone(&self) -> EntityDb

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for EntityDb

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl MemUsageTreeCapture for EntityDb

fn capture_mem_usage_tree(&self) -> MemUsageTree

impl SizeBytes for EntityDb

fn heap_size_bytes(&self) -> u64

Returns how many bytes self uses on the heap.

fn total_size_bytes(&self) -> u64

Returns the total size of self in bytes, accounting for both stack and heap space.

fn stack_size_bytes(&self) -> u64

Returns the total size of self on the stack, in bytes.

fn is_pod() -> bool

Is Self just plain old data?