Crate rattler_lock

Expand description

Definitions for a lock-file format that stores information about pinned dependencies from both the Conda and Pypi ecosystem.

The crate is structured in two API levels.

The top level API accessible through the LockFile type that exposes high level access to the lock-file. This API is intended to be relatively stable and is the preferred way to interact with the lock-file.
The *Data types. These are lower level types that expose more of the internal data structures used in the crate. These types are not intended to be stable and are subject to change over time. These types are used internally by the top level API. Also note that only a subset of the *Data types are exposed. See [crate::PyPiPackageData], [crate::CondaPackageData] for examples.

§Design goals

The goal of the lock-file format is:

To be complete. The lock-file should contain all the information needed to recreate environments even years after it was created. As long as the package data persists that a lock-file refers to, it should be possible to recreate the environment.
To be human readable. Although lock-files are not intended to be edited by hand, they should be relatively easy to read and understand. So that when a lock-file is checked into version control and someone looks at the diff, they can understand what changed.
To be easily parse-able. It should be fairly straightforward to create a parser for the format so that it can be used in other tools.
To reduce diff size when the content changes. The order of content in the serialized lock-file should be fixed to ensure that the diff size is minimized when the content changes.
To be reproducible. Recreating the lock-file with the exact same input (including externally fetched data) should yield the same lock-file byte-for-byte.
To be statically verifiable. Given the specifications of the packages that went into a lock-file it should be possible to cheaply verify whether or not the specifications are still satisfied by the packages stored in the lock-file.
Backward compatible. Older version of lock-files should still be readable by never versions of this crate.

§Relation to conda-lock

Initially the lock-file format was based on conda-lock but over time significant changes have been made compared to the original conda-lock format. Conda-lock files (e.g. conda-lock.yml files) can still be parsed by this crate but the serialization format changed significantly. This means files created by this crate are not compatible with conda-lock.

Conda-lock stores a lot of metadata to be able to verify if the lock-file is still valid given the sources/inputs. For example conda-lock contains a content-hash which is a hash of all the input data of the lock-file. This crate approaches this differently by storing enough information in the lock-file to be able to verify if the lock-file still satisfies an input/source without requiring additional input (e.g. network requests) or expensive solves. We call this static satisfiability verification.

Conda-lock stores a custom partial representation of a rattler_conda_types::RepoDataRecord in the lock-file. This poses a problem when incrementally updating an environment. To only partially update packages in the lock-file without completely recreating it, the records stored in the lock-file need to be passed to the solver as “preferred” packages. Since rattler_conda_types::MatchSpec can match on any field present in a rattler_conda_types::PackageRecord we need to store all fields in the lock-file not just a subset. To that end this crate stores the full rattler_conda_types::PackageRecord in the lock-file. This allows completely recreating the record that was read from repodata when the lock-file was created which will allow a correct incremental update.

Conda-lock requires users to create multiple lock-files when they want to store multiple environments. This crate allows storing multiple environments for different platforms and with different channels in a single lock-file. This allows storing production- and test environments in a single file.

Re-exports§

pub use options::PypiPrereleaseMode;
pub use options::SolveOptions;

Modules§

options: Defines SolveOptions and reexports from rattler_solve that are used.
source: Provides data types that are used to describe the location of a source package.

Structs§

Channel: The conda channel that was used for the dependency
CondaBinaryData: Information about a binary conda package stored in the lock-file.
CondaSourceData: Information about a source package stored in the lock-file.
Environment: Information about a specific environment in the lock-file.
EnvironmentPackages: A deduplicated set of package references, each stored as a PackageIndex paired with its package id.
InconsistentInsertError: Error returned by EnvironmentPackages::insert when the requested handle conflicts with an existing entry.
InputHash: A record of input files that were used to define the metadata of the package.
LockFile: Represents a lock-file for both Conda packages and Pypi packages.
LockFileBuilder: A struct to incrementally build a lock-file.
OwnedEnvironment: An owned version of an Environment.
OwnedPlatform: An owned version of a Platform.
PackageHandle: An opaque handle to a package stored in a LockFile, bundling its PackageIndex with a package id used to refer to it in the lockfile format.
PackageIndex: An package used in an environment. Selects a type of package based on the enum and might contain additional data that is specific to the environment. For instance different environments might select the same Pypi package but with different extras.
PartialSourceMetadata: Metadata for a source package without a fully evaluated package record.
Platform: A reference to a platform in a lock file.
PlatformData: Represents a package with platform-specific information.
PlatformName: A valid name for a platform
PypiDistributionData: Data for a wheel package (index-served or local .whl file).
PypiIndexes: Defines the pypi indexes that were used during solving.
PypiSourceData: Data for a local source directory package.
PypiSourceTreeHashable: A struct that wraps the hashable part of a source package.
RepoDataRecord: Information about a package from repodata. It includes a crate::PackageRecord but it also stores the source of the data (like the url and the channel).
SourceData: The packages needed to build a source package.
SourceIdentifier: A unique identifier for a source package in the lock file.
Verbatim: A helper to keep the user-provided input in addition to the evaluated value, behaving like the inner value T as much as possible.

Enums§

CondaPackageData: A locked conda dependency can be either a binary package or a source package.
ConversionError: Error used when converting from repo_data module to conda lock module
FileFormatVersion: The different version of the lock-file format.
FindLinksUrlOrPath: A flat index is a static source of
FromSelectorIdsError: Error returned when parsing selector-id strings for an EnvironmentPackages set, reporting either a failed lookup from the resolver closure or an internal inconsistency among the resolved entries.
GitShallowSpec: Shallow git specification for tracking the originally requested reference.
InvalidPackageHandleError: Error returned by PackageHandle::get when the handle does not consistently refer to a package in the provided list.
LockedPackage: Information about a single locked package in an environment.
PackageBuildSource: Package build source location for reproducible builds.
PackageHashes: This implementation of the Deserialize trait for the PackageHashes struct
ParseCondaLockError
ParsePlatformError: An error that can occur when parsing a platform from a string.
ParseSourceIdentifierError: Error type for parsing a SourceIdentifier from a string.
PypiPackageData: A pinned PyPI package, either a wheel (immutable artifact) or a source directory (mutable local path).
RegisterSourcePackageError: Error returned by LockFileBuilder::register_conda_source_package and LockFileBuilder::register_pypi_source_package.
SourceMetadata: Metadata for a source package, either partial (name-only) or full (evaluated record + sources).
UrlOrPath: Represents either a URL or a path.
VariantValue: Represents a conda-build variant value.

Constants§

DEFAULT_ENVIRONMENT_NAME: The name of the default environment in a LockFile. This is the environment name that is used when no explicit environment name is specified.

Traits§

Matches: A trait that defines the behavior of matching a spec against a record.