Function run_suite_as 

pub fn run_suite_as(
    dir: &Path,
    only: Option<&str>,
    provider: &Arc<dyn ModelProvider>,
    secret: Option<&str>,
    keep: Option<&Path>,
    repeat: usize,
    executor: Option<&str>,
) -> CommandResult

run_suite with every case’s attempts made by an external executor (claude, codex), to compare it with the native loop on the same cases and oracles.