pub struct MegatronInstaller { /* private fields */ }Expand description
The Megatron-LM installer.
Implementations§
Source§impl MegatronInstaller
impl MegatronInstaller
Sourcepub fn new(config: MegatronConfig) -> Self
pub fn new(config: MegatronConfig) -> Self
Create a new Megatron installer with the given config.
Sourcepub fn with_defaults() -> Self
pub fn with_defaults() -> Self
Create with default config.
Sourcepub fn dependencies(&self) -> &[&str]
pub fn dependencies(&self) -> &[&str]
Get the list of required dependencies.
Megatron-LM depends on:
- PyTorch (ROCm-enabled)
- MPI4Py
Sourcepub fn rocm_env(&self, rocm_env: &RocmEnv) -> Vec<(String, String)>
pub fn rocm_env(&self, rocm_env: &RocmEnv) -> Vec<(String, String)>
Get the ROCm environment variables needed for Megatron.
The original script sets:
- AMD_LOG_LEVEL=0
- HIP_VISIBLE_DEVICES
- ROCR_VISIBLE_DEVICES
- HSA_OVERRIDE_GFX_VERSION=11.0.0
- PYTORCH_ROCM_ARCH=gfx1100
- ROCM_PATH
- HSA_TOOLS_LIB
- PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True
Sourcepub fn build_git_clone_command(&self) -> ShellCommand
pub fn build_git_clone_command(&self) -> ShellCommand
Construct the git clone command for Megatron-LM.
The original script clones:
git clone https://github.com/NVIDIA/Megatron-LM.git <dir>
It retries up to 3 times on failure.
Sourcepub fn max_clone_retries(&self) -> u32
pub fn max_clone_retries(&self) -> u32
Get the maximum number of clone retries.
Sourcepub fn build_pip_install_command(&self) -> ShellCommand
pub fn build_pip_install_command(&self) -> ShellCommand
Construct the pip install command for Megatron-LM.
The original script installs with:
pip install -e . --no-deps
The editable install is done from the cloned directory. Dependencies are resolved separately (safe requirements filtering).
§Pip Target Directory
The working directory is set to install_dir(), which is the git clone
target directory. The NVIDIA/Megatron-LM repository has setup.py at the
repository root (not in a subdirectory), so pip install -e . works
correctly from the clone root.
Sourcepub fn build_dep_install_command(&self, package: &str) -> ShellCommand
pub fn build_dep_install_command(&self, package: &str) -> ShellCommand
Construct the pip install command for a specific dependency.
Used for installing individual safe dependencies.
Sourcepub fn build_requirements_install_command(
&self,
requirements_path: &str,
) -> ShellCommand
pub fn build_requirements_install_command( &self, requirements_path: &str, ) -> ShellCommand
Construct a pip install command for a requirements file.
Sourcepub fn is_safe_package(&self, package_name: &str) -> bool
pub fn is_safe_package(&self, package_name: &str) -> bool
Check if a Python package is safe to install (non-CUDA/non-NVIDIA).
Delegates to the single canonical blocklist
crate::installers::common::nvidia_blocklist::is_cuda_nvidia_package
(Stage 2: one unified rule for the “no nvidia deps ever” tenet).
Auto Trait Implementations§
impl Freeze for MegatronInstaller
impl RefUnwindSafe for MegatronInstaller
impl Send for MegatronInstaller
impl Sync for MegatronInstaller
impl Unpin for MegatronInstaller
impl UnsafeUnpin for MegatronInstaller
impl UnwindSafe for MegatronInstaller
Blanket Implementations§
Source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
Source§impl<T> Instrument for T
impl<T> Instrument for T
Source§fn instrument(self, span: Span) -> Instrumented<Self> ⓘ
fn instrument(self, span: Span) -> Instrumented<Self> ⓘ
Source§fn in_current_span(self) -> Instrumented<Self> ⓘ
fn in_current_span(self) -> Instrumented<Self> ⓘ
Source§impl<T> IntoEither for T
impl<T> IntoEither for T
Source§fn into_either(self, into_left: bool) -> Either<Self, Self> ⓘ
fn into_either(self, into_left: bool) -> Either<Self, Self> ⓘ
self into a Left variant of Either<Self, Self>
if into_left is true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read moreSource§fn into_either_with<F>(self, into_left: F) -> Either<Self, Self> ⓘ
fn into_either_with<F>(self, into_left: F) -> Either<Self, Self> ⓘ
self into a Left variant of Either<Self, Self>
if into_left(&self) returns true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read more