Crate carbon_core

Carbon Core

carbon-core is a framework designed for building customizable and extensible indexers tailored to Solana blockchain data. It facilitates efficient data ingestion, transformation, and processing, supporting a wide range of use cases from transaction parsing to complex instruction analysis. This crate includes modular components that enable users to process blockchain data flexibly and with ease.

The true power of this framework lies when utilizing all it’s components in combination with one another.

Modules Overview

• account: Manages account data processing, including decoding and updates. Account data is processed through pipes that support custom decoders and processors.

• account_deletion: Handles the deletion of accounts and processes these events in the pipeline.

• collection: Defines collections for instruction decoding, allowing for customized instruction parsers that handle specific instruction sets.

• datasource: Provides data ingestion capabilities, enabling the integration of external data sources into the pipeline. Supports Solana-specific data structures.

• deserialize: Contains utilities for data deserialization, including helper functions for parsing Solana transactions and other binary data formats.

• error: Defines error types used throughout the crate, providing consistent error handling for the framework.

• instruction: Supports instruction parsing and processing within transactions. This module includes structures and traits for decoding and handling transaction instructions.

• metrics: Facilitates performance monitoring and metric recording within the pipeline. Metrics can be customized and are recorded at each processing stage for monitoring and debugging purposes.

• pipeline: Represents the core of the framework, defining the main pipeline structure that manages data flow and processing. The pipeline integrates data sources, processing pipes, and metrics to provide a complete data processing solution.

• processor: Contains traits and implementations for processing data in the pipeline. This module allows for the creation of custom data processors that can be integrated into various stages of the pipeline.

• [schema]: Defines transaction schemas, allowing for structured parsing and validation of transaction data based on specified rules. Supports complex nested instruction matching for comprehensive transaction analysis.

• transaction: Manages transaction data, including metadata extraction and parsing. This module supports transaction validation and processing, enabling detailed transaction insights.

• transformers: Provides utility functions for transforming and restructuring data. This module includes functions for converting Solana transaction data into formats suitable for processing within the framework.

Quick Start

To create a new carbon-core pipeline, start by configuring data sources, processing pipes, and metrics in the pipeline::PipelineBuilder. Below is a basic example demonstrating how to set up a pipeline:

use std::sync::Arc;

carbon_core::pipeline::Pipeline::builder()
.datasource(transaction_crawler)
.metrics(Arc::new(LogMetrics::new()))
.metrics(Arc::new(PrometheusMetrics::new()))
.instruction(
   TestProgramDecoder,
   TestProgramProcessor
)
.account(
    TestProgramDecoder,
    TestProgramAccountProcessor
)
.transaction(TEST_SCHEMA.clone(), TestProgramTransactionProcessor)
.account_deletions(TestProgramAccountDeletionProcessor)
.build()?
.run()
.await?;

Crate Features

• Modular Design: Components can be easily added or replaced, allowing for a high degree of customization.
• Concurrency Support: Built with asynchronous Rust, enabling efficient data processing in parallel.
• Solana-Specific: Tailored to handle Solana blockchain data structures, making it ideal for blockchain data analysis and transaction processing.

Notes

• carbon-core integrates with Solana’s SDK, leveraging types and data structures specific to the Solana blockchain.
• This framework is designed for advanced use cases, such as blockchain indexing, transaction monitoring, and custom data analysis.

Explore each module in detail to understand their individual functions and to learn how to customize and extend carbon-core to suit your specific data processing requirements.

Re-exports

pub use borsh;

Modules

account

Provides structures and traits for processing and decoding Solana accounts within the pipeline.

account_deletion

Provides structures and traits for handling account deletion events within the pipeline.

collection

Provides traits and utilities for decoding and managing collections of Solana instructions.

datasource

Provides traits and structures for managing and consuming data updates from various sources.

deserialize

Provides traits and utility functions for deserialization and account arrangement within the carbon-core framework.

error

Defines the Error enum and CarbonResult type used for error handling in the carbon-core framework.

instruction

Provides structures and traits for decoding and processing instructions within transactions.

metrics

A trait for collecting and managing performance metrics within the pipeline.

pipeline

Deprocess(ines the Pipeline struct and related components for processing blockchain data updates.

processor

Defines the Processor trait for processing data within the carbon-core pipeline.

schema

Defines the structures and functions for constructing and matching transaction schemas in carbon-core.

schemas

Schema Module

transaction

Provides types and traits for handling transaction processing in the carbon-core framework. It also provides utilities for matching transactions to schemas and executing custom processing logic on matched data.

transformers

Provides utility functions to transform transaction data into various representations within the carbon-core framework.

try_decode_ixs

Instruction Decoding Module

Macros

instruction_decoder_collection

Generates a collection of instruction decoders and associated enums.

schema

Constructs a TransactionSchema from provided tokens.

schema_inner

Recursively constructs schema nodes within the schema! macro.

try_decode_instructions

Attempts to decode an instruction into a specific variant type.

Derive Macros

CarbonDeserialize

Automatically generates an implementation of the CarbonDeserialize trait.

InstructionType

Derives a corresponding InstructionType enum for a given enum.