[−][src]Module exonum::runtime::migrations

Data migration tools.

Stability

Since migrations are tightly related to unstable Runtime trait, the entirety of this module is considered unstable.

Migrations Overview

The goal of a data migration is to prepare data of an Exonum service for use with an updated version of the service business logic. In this sense, migrations fulfil the same role as migrations in traditional database management systems.

Migrations are performed via MigrationScripts, which are essentially wrappers around a closure. A script takes data of a service and uses the database capabilities to transform it to a new version. Migration is non-destructive, i.e., does not remove the old versions of migrated indexes. Instead, new indexes are created in a separate namespace, and atomically replace the old data when the migration is flushed. (See database docs for more details.)

The problems solved by the migration workflow are:

Allowing for migration to be performed in background, while the node continues to process transactions and other requests.
Ensuring that migrations finish at finite time (i.e., at some blockchain height).
Allowing concurrent migrations for different services.
Ensuring that all nodes in the network have arrived at the same data after migration is completed.

Similar to other service lifecycle events, data migrations are managed by the dispatcher, but are controlled by the supervisor service.

Migration Types

Exonum recognizes two kinds of migrations:

Fast-forward migrations synchronously change the version of the artifact associated with the service. A fast-forward migration is performed if the updated artifact signals that it is compatible with the old service data by returning Ok(None) from Runtime::migrate().
Migrations that require changing data layout via MigrationScripts are referred to as async migrations.

For a migration to start, the targeted service must be stopped or frozen, and a newer version of the service artifact needs to be deployed across the network.

Fast-Forward Migration Workflow

Fast-forward migrations do not require any special workflow to agree migration outcome among nodes; indeed, the outcome is agreed upon via the consensus algorithm. The artifact associated with the service instance is changed instantly. The service status is changed to stopped, regardless of the status before the migration. This is because a new artifact might want to prepare service data before the artifact can use it.

Async Migration Workflow

Migration is initiated by a call from a supervisor. Once a block with this call is merged, all nodes in the network retrieve the migration script via Runtime::migrate() and start executing it in a background thread. The script may execute at varying speed on different nodes. Service status changes to Migrating.
After the script is finished on a node, its result becomes available using the local_migration_result() method of the dispatcher schema. Nodes synchronize these results using supervisor capabilities (e.g., via broadcasting transactions).
Once the consensus is built up around migration, its result is either committed or the migration is rolled back. Right below, we consider commitment workflow; the rollback workflow will be described slightly later.
Committing a migration works similarly to artifact commitment. It means that any node in the network starting from a specific blockchain height must have migration completed with a specific outcome (i.e., hash of the migrated data). A node that does not have migration script completed by this moment will block until the script is completed. If the local migration outcome differs from the committed one, the node will be unable to continue participating in the network.
After migration commitment, migration can be flushed, which will replace old service data with the migrated one. Flushing is a separate call to the dispatcher; it can occur at any block after the migration commitment (since at this point, we guarantee that the migration data is available and is the same on all nodes).
After the migration is flushed, the service returns to the Stopped status. The service can then be resumed with the new data, or more migrations could be applied to it.

If the migration is rolled back on step 3, the migrated data is erased, and the service returns to the Stopped status. The local migration result is ignored; if the migration script has not completed locally, it is aborted.

Note that commitment and flushing are separate operations and must be performed in different blocks. When a migration is flushed, the migrated data needs to have a definite state, which is ensured by an earlier commitment acting as a filter. The requirement for different blocks is more nuanced and is related to implementation details of the database backend. Namely, the flushing operation needs to be performed on a fork which contains the final migration state; not doing this may break state aggregation.

Deciding when it is appropriate to commit or roll back a migration is the responsibility of the supervisor service. For example, it may commit the migration once all validators have submitted identical migration results, and roll back a migration if at least one validator has reported an error during migration or there is divergence among reported migration results.

Structs

InstanceMigration	Information about a migration of a service instance.
LinearMigrations	Linearly ordered migrations.
MigrationContext	Context of a migration.
MigrationScript	Atomic migration script.
MigrationStatus	Result of execution of a migration script.

Enums

InitMigrationError	Errors that can occur when initiating a data migration. This error indicates that the migration cannot be started.
MigrationError	Errors that can occur in a migration script.
MigrationType	Types of data migrations.

Traits

MigrateData

Encapsulates data migration logic.