[INFO] cloning repository https://github.com/folkol/llm-inference-benchmark
[INFO] running `Command { std: "git" "-c" "credential.helper=" "-c" "credential.helper=/workspace/cargo-home/bin/git-credential-null" "clone" "--bare" "https://github.com/folkol/llm-inference-benchmark" "/workspace/cache/git-repos/https%3A%2F%2Fgithub.com%2Ffolkol%2Fllm-inference-benchmark", kill_on_drop: false }`
[INFO] [stderr] Cloning into bare repository '/workspace/cache/git-repos/https%3A%2F%2Fgithub.com%2Ffolkol%2Fllm-inference-benchmark'...
[INFO] running `Command { std: "git" "rev-parse" "HEAD", kill_on_drop: false }`
[INFO] [stdout] 2ec705387e920ca8ffd44909fc1d7aa48a3a861b
[INFO] testing folkol/llm-inference-benchmark against 1.98.0-beta.1 for beta-1.98-1
[INFO] running `Command { std: "git" "clone" "/workspace/cache/git-repos/https%3A%2F%2Fgithub.com%2Ffolkol%2Fllm-inference-benchmark" "/workspace/builds/worker-6-tc2/source", kill_on_drop: false }`
[INFO] [stderr] Cloning into '/workspace/builds/worker-6-tc2/source'...
[INFO] [stderr] done.
[INFO] removed /workspace/builds/worker-6-tc2/source/.cargo/config.toml
[INFO] started tweaking git repo https://github.com/folkol/llm-inference-benchmark
[INFO] finished tweaking git repo https://github.com/folkol/llm-inference-benchmark
[INFO] tweaked toml for git repo https://github.com/folkol/llm-inference-benchmark written to /workspace/builds/worker-6-tc2/source/Cargo.toml
[INFO] validating manifest of git repo https://github.com/folkol/llm-inference-benchmark on toolchain 1.98.0-beta.1
[INFO] running `Command { std: CARGO_HOME="/workspace/cargo-home" RUSTUP_HOME="/workspace/rustup-home" "/workspace/cargo-home/bin/cargo" "+1.98.0-beta.1" "metadata" "--manifest-path" "Cargo.toml" "--no-deps", kill_on_drop: false }`
[INFO] crate git repo https://github.com/folkol/llm-inference-benchmark already has a lockfile, it will not be regenerated
[INFO] running `Command { std: CARGO_HOME="/workspace/cargo-home" RUSTUP_HOME="/workspace/rustup-home" "/workspace/cargo-home/bin/cargo" "+1.98.0-beta.1" "fetch" "--manifest-path" "Cargo.toml", kill_on_drop: false }`
[INFO] running `Command { std: "docker" "create" "-v" "/var/lib/crater-agent-workspace/builds/worker-6-tc2/source:/opt/rustwide/workdir:ro,Z" "-v" "/var/lib/crater-agent-workspace/builds/worker-6-tc2/target:/opt/rustwide/target:rw,Z" "-v" "/var/lib/crater-agent-workspace/cargo-home:/opt/rustwide/cargo-home:ro,Z" "-v" "/var/lib/crater-agent-workspace/rustup-home:/opt/rustwide/rustup-home:ro,Z" "-m" "1610612736" "--network" "none" "ghcr.io/rust-lang/crates-build-env/linux@sha256:3d5ced03c013a94a2f102a4510f48a6e9184255caf5fd8244f58017bde7f5210" "sleep" "infinity", kill_on_drop: false }`
[INFO] [stdout] 29405f2eb4212d6a3b5596b52740a9042f41c76d021c856f904ae87ccc2d4f66
[INFO] running `Command { std: "docker" "start" "29405f2eb4212d6a3b5596b52740a9042f41c76d021c856f904ae87ccc2d4f66", kill_on_drop: false }`
[INFO] running `Command { std: "docker" "inspect" "29405f2eb4212d6a3b5596b52740a9042f41c76d021c856f904ae87ccc2d4f66", kill_on_drop: false }`
[INFO] running `Command { std: "docker" "exec" "-e" "SOURCE_DIR=/opt/rustwide/workdir" "-e" "CARGO_HOME=/opt/rustwide/cargo-home" "-e" "RUSTUP_HOME=/opt/rustwide/rustup-home" "-e" "CARGO_TARGET_DIR=/opt/rustwide/target" "-w" "/opt/rustwide/workdir" "--user" "0:0" "29405f2eb4212d6a3b5596b52740a9042f41c76d021c856f904ae87ccc2d4f66" "/opt/rustwide/cargo-home/bin/cargo" "+1.98.0-beta.1" "metadata" "--no-deps" "--format-version=1", kill_on_drop: false }`
[INFO] running `Command { std: "docker" "inspect" "29405f2eb4212d6a3b5596b52740a9042f41c76d021c856f904ae87ccc2d4f66", kill_on_drop: false }`
[INFO] running `Command { std: "docker" "exec" "-e" "SOURCE_DIR=/opt/rustwide/workdir" "-e" "CARGO_HOME=/opt/rustwide/cargo-home" "-e" "RUSTUP_HOME=/opt/rustwide/rustup-home" "-e" "CARGO_TARGET_DIR=/opt/rustwide/target" "-e" "CARGO_INCREMENTAL=0" "-e" "RUST_BACKTRACE=full" "-e" "RUSTFLAGS=--cap-lints=warn" "-e" "RUSTDOCFLAGS=--cap-lints=warn" "-w" "/opt/rustwide/workdir" "--user" "0:0" "29405f2eb4212d6a3b5596b52740a9042f41c76d021c856f904ae87ccc2d4f66" "/opt/rustwide/cargo-home/bin/cargo" "+1.98.0-beta.1" "build" "--frozen" "--message-format=json", kill_on_drop: false }`
[INFO] [stderr]    Compiling serde_core v1.0.228
[INFO] [stderr]    Compiling pkg-config v0.3.33
[INFO] [stderr]    Compiling cc v1.2.62
[INFO] [stderr]    Compiling vcpkg v0.2.15
[INFO] [stderr]    Compiling syn v2.0.117
[INFO] [stderr]    Compiling serde v1.0.228
[INFO] [stderr]    Compiling memchr v2.8.0
[INFO] [stderr]    Compiling parking_lot_core v0.9.12
[INFO] [stderr]    Compiling bitflags v2.11.1
[INFO] [stderr]    Compiling parking_lot v0.12.5
[INFO] [stderr]    Compiling socket2 v0.6.3
[INFO] [stderr]    Compiling mio v1.2.0
[INFO] [stderr]    Compiling crossbeam-utils v0.8.21
[INFO] [stderr]    Compiling foreign-types-shared v0.1.1
[INFO] [stderr]    Compiling log v0.4.29
[INFO] [stderr]    Compiling openssl v0.10.79
[INFO] [stderr]    Compiling http v0.2.12
[INFO] [stderr]    Compiling foreign-types v0.3.2
[INFO] [stderr]    Compiling tracing-core v0.1.36
[INFO] [stderr]    Compiling futures-io v0.3.32
[INFO] [stderr]    Compiling native-tls v0.2.18
[INFO] [stderr]    Compiling typenum v1.20.0
[INFO] [stderr]    Compiling crc32fast v1.5.0
[INFO] [stderr]    Compiling futures-util v0.3.32
[INFO] [stderr]    Compiling tracing v0.1.44
[INFO] [stderr]    Compiling crossbeam-epoch v0.9.18
[INFO] [stderr]    Compiling openssl-sys v0.9.115
[INFO] [stderr]    Compiling utf8parse v0.2.2
[INFO] [stderr]    Compiling simd-adler32 v0.3.9
[INFO] [stderr]    Compiling generic-array v0.14.7
[INFO] [stderr]    Compiling rustix v1.1.4
[INFO] [stderr]    Compiling rayon-core v1.13.0
[INFO] [stderr]    Compiling crossbeam-deque v0.8.6
[INFO] [stderr]    Compiling http-body v0.4.6
[INFO] [stderr]    Compiling num-traits v0.2.19
[INFO] [stderr]    Compiling anstyle-parse v1.0.0
[INFO] [stderr]    Compiling futures-channel v0.3.32
[INFO] [stderr]    Compiling socket2 v0.5.10
[INFO] [stderr]    Compiling anstyle v1.0.14
[INFO] [stderr]    Compiling colorchoice v1.0.5
[INFO] [stderr]    Compiling anstyle-query v1.1.5
[INFO] [stderr]    Compiling serde_json v1.0.149
[INFO] [stderr]    Compiling linux-raw-sys v0.12.1
[INFO] [stderr]    Compiling portable-atomic v1.13.1
[INFO] [stderr]    Compiling is_terminal_polyfill v1.70.2
[INFO] [stderr]    Compiling miniz_oxide v0.8.9
[INFO] [stderr]    Compiling anstream v1.0.0
[INFO] [stderr]    Compiling block-buffer v0.10.4
[INFO] [stderr]    Compiling crypto-common v0.1.7
[INFO] [stderr]    Compiling winnow v0.7.15
[INFO] [stderr]    Compiling base64 v0.21.7
[INFO] [stderr]    Compiling clap_lex v1.1.0
[INFO] [stderr]    Compiling bumpalo v3.20.2
[INFO] [stderr]    Compiling unicode-width v0.2.2
[INFO] [stderr]    Compiling option-ext v0.2.0
[INFO] [stderr]    Compiling toml_write v0.1.2
[INFO] [stderr]    Compiling strsim v0.11.1
[INFO] [stderr]    Compiling zip v2.4.2
[INFO] [stderr]    Compiling either v1.15.0
[INFO] [stderr]    Compiling zopfli v0.8.3
[INFO] [stderr]    Compiling rayon v1.12.0
[INFO] [stderr]    Compiling clap_builder v4.6.0
[INFO] [stderr]    Compiling console v0.15.11
[INFO] [stderr]    Compiling dirs-sys v0.4.1
[INFO] [stderr]    Compiling synstructure v0.13.2
[INFO] [stderr]    Compiling xattr v1.6.1
[INFO] [stderr]    Compiling rustls-pemfile v1.0.4
[INFO] [stderr]    Compiling flate2 v1.1.9
[INFO] [stderr]    Compiling digest v0.10.7
[INFO] [stderr]    Compiling csv-core v0.1.13
[INFO] [stderr]    Compiling filetime v0.2.29
[INFO] [stderr]    Compiling number_prefix v0.4.0
[INFO] [stderr]    Compiling sync_wrapper v0.1.2
[INFO] [stderr]    Compiling colored v2.2.0
[INFO] [stderr]    Compiling sha2 v0.10.9
[INFO] [stderr]    Compiling tar v0.4.45
[INFO] [stderr]    Compiling indicatif v0.17.11
[INFO] [stderr]    Compiling csv v1.4.0
[INFO] [stderr]    Compiling dirs v5.0.1
[INFO] [stderr]    Compiling bytesize v1.3.3
[INFO] [stderr]    Compiling sysinfo v0.29.11
[INFO] [stderr]    Compiling zerofrom-derive v0.1.7
[INFO] [stderr]    Compiling yoke-derive v0.8.2
[INFO] [stderr]    Compiling displaydoc v0.2.5
[INFO] [stderr]    Compiling zerovec-derive v0.11.3
[INFO] [stderr]    Compiling serde_derive v1.0.228
[INFO] [stderr]    Compiling tokio-macros v2.7.0
[INFO] [stderr]    Compiling openssl-macros v0.1.1
[INFO] [stderr]    Compiling tokio v1.52.3
[INFO] [stderr]    Compiling thiserror-impl v2.0.18
[INFO] [stderr]    Compiling zerofrom v0.1.8
[INFO] [stderr]    Compiling yoke v0.8.2
[INFO] [stderr]    Compiling clap_derive v4.6.1
[INFO] [stderr]    Compiling zerovec v0.11.6
[INFO] [stderr]    Compiling zerotrie v0.2.4
[INFO] [stderr]    Compiling tinystr v0.8.3
[INFO] [stderr]    Compiling icu_locale_core v2.2.0
[INFO] [stderr]    Compiling potential_utf v0.1.5
[INFO] [stderr]    Compiling thiserror v2.0.18
[INFO] [stderr]    Compiling icu_collections v2.2.0
[INFO] [stderr]    Compiling clap v4.6.1
[INFO] [stderr]    Compiling icu_provider v2.2.0
[INFO] [stderr]    Compiling icu_normalizer v2.2.0
[INFO] [stderr]    Compiling icu_properties v2.2.0
[INFO] [stderr]    Compiling toml_datetime v0.6.11
[INFO] [stderr]    Compiling serde_spanned v0.6.9
[INFO] [stderr]    Compiling serde_urlencoded v0.7.1
[INFO] [stderr]    Compiling chrono v0.4.44
[INFO] [stderr]    Compiling toml_edit v0.22.27
[INFO] [stderr]    Compiling idna_adapter v1.2.2
[INFO] [stderr]    Compiling idna v1.1.0
[INFO] [stderr]    Compiling url v2.5.8
[INFO] [stderr]    Compiling tokio-util v0.7.18
[INFO] [stderr]    Compiling tokio-native-tls v0.3.1
[INFO] [stderr]    Compiling toml v0.8.23
[INFO] [stderr]    Compiling h2 v0.3.27
[INFO] [stderr]    Compiling hyper v0.14.32
[INFO] [stderr]    Compiling hyper-tls v0.5.0
[INFO] [stderr]    Compiling reqwest v0.11.27
[INFO] [stderr]    Compiling llmb v0.1.0 (/opt/rustwide/workdir)
[INFO] [stderr]     Finished `dev` profile [unoptimized + debuginfo] target(s) in 2m 09s
[INFO] running `Command { std: "docker" "inspect" "29405f2eb4212d6a3b5596b52740a9042f41c76d021c856f904ae87ccc2d4f66", kill_on_drop: false }`
[INFO] running `Command { std: "docker" "exec" "-e" "SOURCE_DIR=/opt/rustwide/workdir" "-e" "CARGO_HOME=/opt/rustwide/cargo-home" "-e" "RUSTUP_HOME=/opt/rustwide/rustup-home" "-e" "CARGO_TARGET_DIR=/opt/rustwide/target" "-e" "CARGO_INCREMENTAL=0" "-e" "RUST_BACKTRACE=full" "-e" "RUSTFLAGS=--cap-lints=warn" "-e" "RUSTDOCFLAGS=--cap-lints=warn" "-w" "/opt/rustwide/workdir" "--user" "0:0" "29405f2eb4212d6a3b5596b52740a9042f41c76d021c856f904ae87ccc2d4f66" "/opt/rustwide/cargo-home/bin/cargo" "+1.98.0-beta.1" "test" "--frozen" "--no-run" "--message-format=json", kill_on_drop: false }`
[INFO] [stderr]    Compiling llmb v0.1.0 (/opt/rustwide/workdir)
[INFO] [stderr]     Finished `test` profile [unoptimized + debuginfo] target(s) in 3.81s
[INFO] running `Command { std: "docker" "inspect" "29405f2eb4212d6a3b5596b52740a9042f41c76d021c856f904ae87ccc2d4f66", kill_on_drop: false }`
[INFO] running `Command { std: "docker" "exec" "-e" "SOURCE_DIR=/opt/rustwide/workdir" "-e" "CARGO_HOME=/opt/rustwide/cargo-home" "-e" "RUSTUP_HOME=/opt/rustwide/rustup-home" "-e" "CARGO_TARGET_DIR=/opt/rustwide/target" "-e" "CARGO_INCREMENTAL=0" "-e" "RUST_BACKTRACE=full" "-e" "RUSTFLAGS=--cap-lints=warn" "-e" "RUSTDOCFLAGS=--cap-lints=warn" "-w" "/opt/rustwide/workdir" "--user" "0:0" "29405f2eb4212d6a3b5596b52740a9042f41c76d021c856f904ae87ccc2d4f66" "/opt/rustwide/cargo-home/bin/cargo" "+1.98.0-beta.1" "test" "--frozen", kill_on_drop: false }`
[INFO] [stderr]     Finished `test` profile [unoptimized + debuginfo] target(s) in 0.30s
[INFO] [stderr]      Running unittests src/lib.rs (/opt/rustwide/target/debug/deps/llmb-1541a368efc85b1c)
[INFO] [stdout] 
[INFO] [stdout] running 7 tests
[INFO] [stdout] test hardware::tests::report_path_label_without_gpu ... ok
[INFO] [stdout] test setup::tests::find_asset_prefers_plain_macos_arm64_tarball ... ok
[INFO] [stdout] test setup::tests::find_asset_rejects_unrelated_archives ... ok
[INFO] [stdout] test setup::tests::find_windows_cuda_prefers_newer_explicit_toolkit ... ok
[INFO] [stdout] test setup::tests::find_windows_cudart_matches_llama_zip_toolkit_version ... ok
[INFO] [stdout] test hardware::tests::report_path_label_matches_known_snapshot_host ... ok
[INFO] [stdout] test setup::tests::find_asset_accepts_zip_and_tar_gz ... ok
[INFO] [stdout] 
[INFO] [stdout] test result: ok. 7 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s
[INFO] [stdout] 
[INFO] [stdout] 
[INFO] [stdout] running 0 tests
[INFO] [stdout] 
[INFO] [stdout] test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s
[INFO] [stdout] 
[INFO] [stderr]      Running unittests src/main.rs (/opt/rustwide/target/debug/deps/llmb-548c58a2e6283f11)
[INFO] [stderr]      Running tests/e2e.rs (/opt/rustwide/target/debug/deps/e2e-9ba5403a16e1ced0)
[INFO] [stdout] 
[INFO] [stdout] running 5 tests
[INFO] [stdout] test test_e2e_tokens_per_sec_plausible ... FAILED
[INFO] [stderr] error: test failed, to rerun pass `--test e2e`
[INFO] [stdout] test test_e2e_different_seeds_both_complete ... FAILED
[INFO] [stdout] test test_e2e_same_seed_same_token_count ... FAILED
[INFO] [stdout] test test_e2e_aggregated_metrics_are_sane ... FAILED
[INFO] [stdout] test test_e2e_cold_inference_succeeds ... FAILED
[INFO] [stdout] 
[INFO] [stdout] failures:
[INFO] [stdout] 
[INFO] [stdout] ---- test_e2e_tokens_per_sec_plausible stdout ----
[INFO] [stdout] 
[INFO] [stdout] thread 'test_e2e_tokens_per_sec_plausible' (1943) panicked at tests/e2e.rs:26:10:
[INFO] [stdout] llama-cli not found. Install llama.cpp and add it to PATH.
[INFO] [stdout] Download: https://github.com/ggerganov/llama.cpp/releases: Cannot find 'llama-cli'. Install llama.cpp and ensure it is on PATH, or download prebuilt binaries to /root/.cache/llm-benchmark/llama-bin .
[INFO] [stdout] See: https://github.com/ggerganov/llama.cpp/releases
[INFO] [stdout] 
[INFO] [stdout] Stack backtrace:
[INFO] [stdout]    0: <anyhow::Error>::msg::<alloc::string::String>
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/anyhow-1.0.102/src/backtrace.rs:10:14
[INFO] [stdout]    1: llmb::assets::find_exe
[INFO] [stdout]              at ./src/assets.rs:59:5
[INFO] [stdout]    2: llmb::assets::find_llama_cli
[INFO] [stdout]              at ./src/assets.rs:42:5
[INFO] [stdout]    3: e2e::llama_cli
[INFO] [stdout]              at ./tests/e2e.rs:25:5
[INFO] [stdout]    4: e2e::test_e2e_tokens_per_sec_plausible
[INFO] [stdout]              at ./tests/e2e.rs:199:15
[INFO] [stdout]    5: e2e::test_e2e_tokens_per_sec_plausible::{closure#0}
[INFO] [stdout]              at ./tests/e2e.rs:198:39
[INFO] [stdout]    6: <e2e::test_e2e_tokens_per_sec_plausible::{closure#0} as core::ops::function::FnOnce<()>>::call_once
[INFO] [stdout]              at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/core/src/ops/function.rs:250:5
[INFO] [stdout]    7: <fn() -> core::result::Result<(), alloc::string::String> as core::ops::function::FnOnce<()>>::call_once
[INFO] [stdout]              at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/core/src/ops/function.rs:250:5
[INFO] [stdout]    8: test::__rust_begin_short_backtrace::<core::result::Result<(), alloc::string::String>, fn() -> core::result::Result<(), alloc::string::String>>
[INFO] [stdout]              at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/test/src/lib.rs:724:18
[INFO] [stdout]    9: test::run_test_in_process::{closure#0}
[INFO] [stdout]              at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/test/src/lib.rs:747:74
[INFO] [stdout]   10: <core::panic::unwind_safe::AssertUnwindSafe<test::run_test_in_process::{closure#0}> as core::ops::function::FnOnce<()>>::call_once
[INFO] [stdout]              at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/core/src/panic/unwind_safe.rs:275:9
[INFO] [stdout]   11: std::panicking::catch_unwind::do_call::<core::panic::unwind_safe::AssertUnwindSafe<test::run_test_in_process::{closure#0}>, core::result::Result<(), alloc::string::String>>
[INFO] [stdout]              at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/std/src/panicking.rs:576:43
[INFO] [stdout]   12: std::panicking::catch_unwind::<core::result::Result<(), alloc::string::String>, core::panic::unwind_safe::AssertUnwindSafe<test::run_test_in_process::{closure#0}>>
[INFO] [stdout]              at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/std/src/panicking.rs:544:19
[INFO] [stdout]   13: std::panic::catch_unwind::<core::panic::unwind_safe::AssertUnwindSafe<test::run_test_in_process::{closure#0}>, core::result::Result<(), alloc::string::String>>
[INFO] [stdout]              at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/std/src/panic.rs:359:14
[INFO] [stdout]   14: test::run_test_in_process
[INFO] [stdout]              at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/test/src/lib.rs:747:27
[INFO] [stdout]   15: test::run_test::{closure#0}
[INFO] [stdout]              at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/test/src/lib.rs:668:43
[INFO] [stdout]   16: test::run_test::{closure#1}
[INFO] [stdout]              at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/test/src/lib.rs:698:41
[INFO] [stdout]   17: std::sys::backtrace::__rust_begin_short_backtrace::<test::run_test::{closure#1}, ()>
[INFO] [stdout]              at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/std/src/sys/backtrace.rs:166:18
[INFO] [stdout]   18: std::thread::lifecycle::spawn_unchecked::<test::run_test::{closure#1}, ()>::{closure#1}::{closure#0}
[INFO] [stdout]              at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/std/src/thread/lifecycle.rs:70:13
[INFO] [stdout]   19: <core::panic::unwind_safe::AssertUnwindSafe<std::thread::lifecycle::spawn_unchecked<test::run_test::{closure#1}, ()>::{closure#1}::{closure#0}> as core::ops::function::FnOnce<()>>::call_once
[INFO] [stdout]              at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/core/src/panic/unwind_safe.rs:275:9
[INFO] [stdout]   20: std::panicking::catch_unwind::do_call::<core::panic::unwind_safe::AssertUnwindSafe<std::thread::lifecycle::spawn_unchecked<test::run_test::{closure#1}, ()>::{closure#1}::{closure#0}>, ()>
[INFO] [stdout]              at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/std/src/panicking.rs:576:43
[INFO] [stdout]   21: std::panicking::catch_unwind::<(), core::panic::unwind_safe::AssertUnwindSafe<std::thread::lifecycle::spawn_unchecked<test::run_test::{closure#1}, ()>::{closure#1}::{closure#0}>>
[INFO] [stdout]              at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/std/src/panicking.rs:544:19
[INFO] [stdout]   22: std::panic::catch_unwind::<core::panic::unwind_safe::AssertUnwindSafe<std::thread::lifecycle::spawn_unchecked<test::run_test::{closure#1}, ()>::{closure#1}::{closure#0}>, ()>
[INFO] [stdout]              at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/std/src/panic.rs:359:14
[INFO] [stdout]   23: std::thread::lifecycle::spawn_unchecked::<test::run_test::{closure#1}, ()>::{closure#1}
[INFO] [stdout]              at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/std/src/thread/lifecycle.rs:68:26
[INFO] [stdout]   24: <std::thread::lifecycle::spawn_unchecked<test::run_test::{closure#1}, ()>::{closure#1} as core::ops::function::FnOnce<()>>::call_once::{shim:vtable#0}
[INFO] [stdout]              at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/core/src/ops/function.rs:250:5
[INFO] [stdout]   25: <alloc::boxed::Box<dyn core::ops::function::FnOnce<(), Output = ()> + core::marker::Send> as core::ops::function::FnOnce<()>>::call_once
[INFO] [stdout]              at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/alloc/src/boxed.rs:2319:9
[INFO] [stdout]   26: <std::sys::thread::unix::Thread>::new::thread_start
[INFO] [stdout]              at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/std/src/sys/thread/unix.rs:123:17
[INFO] [stdout]   27: <unknown>
[INFO] [stdout]   28: clone
[INFO] [stdout] stack backtrace:
[INFO] [stdout]    0:     0x5d3f63f2ace1 - std[73adb7dc35730857]::backtrace_rs::backtrace::libunwind::trace
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/std/src/../../backtrace/src/backtrace/libunwind.rs:117:9
[INFO] [stdout]    1:     0x5d3f63f2ace1 - std[73adb7dc35730857]::backtrace_rs::backtrace::trace_unsynchronized::<std[73adb7dc35730857]::sys::backtrace::_print_fmt::{closure#1}>
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/std/src/../../backtrace/src/backtrace/mod.rs:66:14
[INFO] [stdout]    2:     0x5d3f63f2ace1 - std[73adb7dc35730857]::sys::backtrace::_print_fmt
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/std/src/sys/backtrace.rs:74:9
[INFO] [stdout]    3:     0x5d3f63f2ace1 - <<std[73adb7dc35730857]::sys::backtrace::BacktraceLock>::print::DisplayBacktrace as core[6883ba1bc0fe4ed1]::fmt::Display>::fmt
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/std/src/sys/backtrace.rs:44:26
[INFO] [stdout]    4:     0x5d3f63f3f7ba - <core[6883ba1bc0fe4ed1]::fmt::rt::Argument>::fmt
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/core/src/fmt/rt.rs:152:76
[INFO] [stdout]    5:     0x5d3f63f3f7ba - core[6883ba1bc0fe4ed1]::fmt::write
[INFO] [stdout]    6:     0x5d3f63f2f6bc - std[73adb7dc35730857]::io::default_write_fmt::<alloc[55a36b64bcbf2c0d]::vec::Vec<u8>>
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/std/src/io/mod.rs:626:11
[INFO] [stdout]    7:     0x5d3f63f2f6bc - <alloc[55a36b64bcbf2c0d]::vec::Vec<u8> as std[73adb7dc35730857]::io::Write>::write_fmt
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/std/src/io/mod.rs:1730:13
[INFO] [stdout]    8:     0x5d3f63f07ed6 - <std[73adb7dc35730857]::sys::backtrace::BacktraceLock>::print
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/std/src/sys/backtrace.rs:47:9
[INFO] [stdout]    9:     0x5d3f63f07ed6 - std[73adb7dc35730857]::panicking::default_hook::{closure#0}
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/std/src/panicking.rs:292:27
[INFO] [stdout]   10:     0x5d3f63f22e29 - std[73adb7dc35730857]::panicking::default_hook
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/std/src/panicking.rs:316:9
[INFO] [stdout]   11:     0x5d3f63e6fa20 - <alloc[55a36b64bcbf2c0d]::boxed::Box<dyn for<'a, 'b> core[6883ba1bc0fe4ed1]::ops::function::Fn<(&'a std[73adb7dc35730857]::panic::PanicHookInfo<'b>,), Output = ()> + core[6883ba1bc0fe4ed1]::marker::Send + core[6883ba1bc0fe4ed1]::marker::Sync> as core[6883ba1bc0fe4ed1]::ops::function::Fn<(&std[73adb7dc35730857]::panic::PanicHookInfo,)>>::call
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/alloc/src/boxed.rs:2333:9
[INFO] [stdout]   12:     0x5d3f63e6fa20 - test[980ffaebd391d06d]::test_main_inner::<test[980ffaebd391d06d]::test_main_static::{closure#0}>::{closure#0}
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/test/src/lib.rs:155:21
[INFO] [stdout]   13:     0x5d3f63f22fe2 - <alloc[55a36b64bcbf2c0d]::boxed::Box<dyn for<'a, 'b> core[6883ba1bc0fe4ed1]::ops::function::Fn<(&'a std[73adb7dc35730857]::panic::PanicHookInfo<'b>,), Output = ()> + core[6883ba1bc0fe4ed1]::marker::Send + core[6883ba1bc0fe4ed1]::marker::Sync> as core[6883ba1bc0fe4ed1]::ops::function::Fn<(&std[73adb7dc35730857]::panic::PanicHookInfo,)>>::call
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/alloc/src/boxed.rs:2333:9
[INFO] [stdout]   14:     0x5d3f63f22fe2 - std[73adb7dc35730857]::panicking::panic_with_hook
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/std/src/panicking.rs:823:13
[INFO] [stdout]   15:     0x5d3f63f07f82 - std[73adb7dc35730857]::panicking::panic_handler::{closure#0}
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/std/src/panicking.rs:688:13
[INFO] [stdout]   16:     0x5d3f63eff229 - std[73adb7dc35730857]::sys::backtrace::__rust_end_short_backtrace::<std[73adb7dc35730857]::panicking::panic_handler::{closure#0}, !>
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/std/src/sys/backtrace.rs:182:18
[INFO] [stdout]   17:     0x5d3f63f08c8d - __rustc[a7b7b02e776dd976]::rust_begin_unwind
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/std/src/panicking.rs:679:5
[INFO] [stdout]   18:     0x5d3f63f3ff8c - core[6883ba1bc0fe4ed1]::panicking::panic_fmt
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/core/src/panicking.rs:80:14
[INFO] [stdout]   19:     0x5d3f63f3fd02 - core[6883ba1bc0fe4ed1]::result::unwrap_failed
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/core/src/result.rs:1870:5
[INFO] [stdout]   20:     0x5d3f63e621a3 - <core[6883ba1bc0fe4ed1]::result::Result<std[73adb7dc35730857]::path::PathBuf, anyhow[3b9a07a6948cd063]::Error>>::expect
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/core/src/result.rs:1183:23
[INFO] [stdout]   21:     0x5d3f63e60b69 - e2e[f0516541adf4547c]::llama_cli
[INFO] [stdout]                                at /opt/rustwide/workdir/tests/e2e.rs:26:10
[INFO] [stdout]   22:     0x5d3f63e60ed1 - e2e[f0516541adf4547c]::test_e2e_tokens_per_sec_plausible
[INFO] [stdout]                                at /opt/rustwide/workdir/tests/e2e.rs:199:15
[INFO] [stdout]   23:     0x5d3f63e5fef7 - e2e[f0516541adf4547c]::test_e2e_tokens_per_sec_plausible::{closure#0}
[INFO] [stdout]                                at /opt/rustwide/workdir/tests/e2e.rs:198:39
[INFO] [stdout]   24:     0x5d3f63e62bf6 - <e2e[f0516541adf4547c]::test_e2e_tokens_per_sec_plausible::{closure#0} as core[6883ba1bc0fe4ed1]::ops::function::FnOnce<()>>::call_once
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/core/src/ops/function.rs:250:5
[INFO] [stdout]   25:     0x5d3f63e62d5b - <fn() -> core[6883ba1bc0fe4ed1]::result::Result<(), alloc[55a36b64bcbf2c0d]::string::String> as core[6883ba1bc0fe4ed1]::ops::function::FnOnce<()>>::call_once
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/core/src/ops/function.rs:250:5
[INFO] [stdout]   26:     0x5d3f63e62d5b - test[980ffaebd391d06d]::__rust_begin_short_backtrace::<core[6883ba1bc0fe4ed1]::result::Result<(), alloc[55a36b64bcbf2c0d]::string::String>, fn() -> core[6883ba1bc0fe4ed1]::result::Result<(), alloc[55a36b64bcbf2c0d]::string::String>>
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/test/src/lib.rs:724:18
[INFO] [stdout]   27:     0x5d3f63e703a5 - test[980ffaebd391d06d]::run_test_in_process::{closure#0}
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/test/src/lib.rs:747:74
[INFO] [stdout]   28:     0x5d3f63e703a5 - <core[6883ba1bc0fe4ed1]::panic::unwind_safe::AssertUnwindSafe<test[980ffaebd391d06d]::run_test_in_process::{closure#0}> as core[6883ba1bc0fe4ed1]::ops::function::FnOnce<()>>::call_once
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/core/src/panic/unwind_safe.rs:275:9
[INFO] [stdout]   29:     0x5d3f63e703a5 - std[73adb7dc35730857]::panicking::catch_unwind::do_call::<core[6883ba1bc0fe4ed1]::panic::unwind_safe::AssertUnwindSafe<test[980ffaebd391d06d]::run_test_in_process::{closure#0}>, core[6883ba1bc0fe4ed1]::result::Result<(), alloc[55a36b64bcbf2c0d]::string::String>>
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/std/src/panicking.rs:576:43
[INFO] [stdout]   30:     0x5d3f63e703a5 - std[73adb7dc35730857]::panicking::catch_unwind::<core[6883ba1bc0fe4ed1]::result::Result<(), alloc[55a36b64bcbf2c0d]::string::String>, core[6883ba1bc0fe4ed1]::panic::unwind_safe::AssertUnwindSafe<test[980ffaebd391d06d]::run_test_in_process::{closure#0}>>
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/std/src/panicking.rs:544:19
[INFO] [stdout]   31:     0x5d3f63e703a5 - std[73adb7dc35730857]::panic::catch_unwind::<core[6883ba1bc0fe4ed1]::panic::unwind_safe::AssertUnwindSafe<test[980ffaebd391d06d]::run_test_in_process::{closure#0}>, core[6883ba1bc0fe4ed1]::result::Result<(), alloc[55a36b64bcbf2c0d]::string::String>>
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/std/src/panic.rs:359:14
[INFO] [stdout]   32:     0x5d3f63e703a5 - test[980ffaebd391d06d]::run_test_in_process
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/test/src/lib.rs:747:27
[INFO] [stdout]   33:     0x5d3f63e703a5 - test[980ffaebd391d06d]::run_test::{closure#0}
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/test/src/lib.rs:668:43
[INFO] [stdout]   34:     0x5d3f63e6ae54 - test[980ffaebd391d06d]::run_test::{closure#1}
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/test/src/lib.rs:698:41
[INFO] [stdout]   35:     0x5d3f63e6ae54 - std[73adb7dc35730857]::sys::backtrace::__rust_begin_short_backtrace::<test[980ffaebd391d06d]::run_test::{closure#1}, ()>
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/std/src/sys/backtrace.rs:166:18
[INFO] [stdout]   36:     0x5d3f63e734f2 - std[73adb7dc35730857]::thread::lifecycle::spawn_unchecked::<test[980ffaebd391d06d]::run_test::{closure#1}, ()>::{closure#1}::{closure#0}
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/std/src/thread/lifecycle.rs:70:13
[INFO] [stdout]   37:     0x5d3f63e734f2 - <core[6883ba1bc0fe4ed1]::panic::unwind_safe::AssertUnwindSafe<std[73adb7dc35730857]::thread::lifecycle::spawn_unchecked<test[980ffaebd391d06d]::run_test::{closure#1}, ()>::{closure#1}::{closure#0}> as core[6883ba1bc0fe4ed1]::ops::function::FnOnce<()>>::call_once
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/core/src/panic/unwind_safe.rs:275:9
[INFO] [stdout]   38:     0x5d3f63e734f2 - std[73adb7dc35730857]::panicking::catch_unwind::do_call::<core[6883ba1bc0fe4ed1]::panic::unwind_safe::AssertUnwindSafe<std[73adb7dc35730857]::thread::lifecycle::spawn_unchecked<test[980ffaebd391d06d]::run_test::{closure#1}, ()>::{closure#1}::{closure#0}>, ()>
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/std/src/panicking.rs:576:43
[INFO] [stdout]   39:     0x5d3f63e734f2 - std[73adb7dc35730857]::panicking::catch_unwind::<(), core[6883ba1bc0fe4ed1]::panic::unwind_safe::AssertUnwindSafe<std[73adb7dc35730857]::thread::lifecycle::spawn_unchecked<test[980ffaebd391d06d]::run_test::{closure#1}, ()>::{closure#1}::{closure#0}>>
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/std/src/panicking.rs:544:19
[INFO] [stdout]   40:     0x5d3f63e734f2 - std[73adb7dc35730857]::panic::catch_unwind::<core[6883ba1bc0fe4ed1]::panic::unwind_safe::AssertUnwindSafe<std[73adb7dc35730857]::thread::lifecycle::spawn_unchecked<test[980ffaebd391d06d]::run_test::{closure#1}, ()>::{closure#1}::{closure#0}>, ()>
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/std/src/panic.rs:359:14
[INFO] [stdout]   41:     0x5d3f63e734f2 - std[73adb7dc35730857]::thread::lifecycle::spawn_unchecked::<test[980ffaebd391d06d]::run_test::{closure#1}, ()>::{closure#1}
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/std/src/thread/lifecycle.rs:68:26
[INFO] [stdout]   42:     0x5d3f63e734f2 - <std[73adb7dc35730857]::thread::lifecycle::spawn_unchecked<test[980ffaebd391d06d]::run_test::{closure#1}, ()>::{closure#1} as core[6883ba1bc0fe4ed1]::ops::function::FnOnce<()>>::call_once::{shim:vtable#0}
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/core/src/ops/function.rs:250:5
[INFO] [stdout]   43:     0x5d3f63f2a56f - <alloc[55a36b64bcbf2c0d]::boxed::Box<dyn core[6883ba1bc0fe4ed1]::ops::function::FnOnce<(), Output = ()> + core[6883ba1bc0fe4ed1]::marker::Send> as core[6883ba1bc0fe4ed1]::ops::function::FnOnce<()>>::call_once
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/alloc/src/boxed.rs:2319:9
[INFO] [stdout]   44:     0x5d3f63f2a56f - <std[73adb7dc35730857]::sys::thread::unix::Thread>::new::thread_start
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/std/src/sys/thread/unix.rs:123:17
[INFO] [stdout]   45:     0x7fac29384aa4 - <unknown>
[INFO] [stdout]   46:     0x7fac29411a64 - clone
[INFO] [stdout]   47:                0x0 - <unknown>
[INFO] [stdout] 
[INFO] [stdout] ---- test_e2e_different_seeds_both_complete stdout ----
[INFO] [stdout] 
[INFO] [stdout] thread 'test_e2e_different_seeds_both_complete' (1938) panicked at tests/e2e.rs:26:10:
[INFO] [stdout] llama-cli not found. Install llama.cpp and add it to PATH.
[INFO] [stdout] Download: https://github.com/ggerganov/llama.cpp/releases: Cannot find 'llama-cli'. Install llama.cpp and ensure it is on PATH, or download prebuilt binaries to /root/.cache/llm-benchmark/llama-bin .
[INFO] [stdout] See: https://github.com/ggerganov/llama.cpp/releases
[INFO] [stdout] 
[INFO] [stdout] Stack backtrace:
[INFO] [stdout]    0: <anyhow::Error>::msg::<alloc::string::String>
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/anyhow-1.0.102/src/backtrace.rs:10:14
[INFO] [stdout]    1: llmb::assets::find_exe
[INFO] [stdout]              at ./src/assets.rs:59:5
[INFO] [stdout]    2: llmb::assets::find_llama_cli
[INFO] [stdout]              at ./src/assets.rs:42:5
[INFO] [stdout]    3: e2e::llama_cli
[INFO] [stdout]              at ./tests/e2e.rs:25:5
[INFO] [stdout]    4: e2e::test_e2e_different_seeds_both_complete
[INFO] [stdout]              at ./tests/e2e.rs:141:15
[INFO] [stdout]    5: e2e::test_e2e_different_seeds_both_complete::{closure#0}
[INFO] [stdout]              at ./tests/e2e.rs:140:44
[INFO] [stdout]    6: <e2e::test_e2e_different_seeds_both_complete::{closure#0} as core::ops::function::FnOnce<()>>::call_once
[INFO] [stdout]              at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/core/src/ops/function.rs:250:5
[INFO] [stdout]    7: <fn() -> core::result::Result<(), alloc::string::String> as core::ops::function::FnOnce<()>>::call_once
[INFO] [stdout]              at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/core/src/ops/function.rs:250:5
[INFO] [stdout]    8: test::__rust_begin_short_backtrace::<core::result::Result<(), alloc::string::String>, fn() -> core::result::Result<(), alloc::string::String>>
[INFO] [stdout]              at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/test/src/lib.rs:724:18
[INFO] [stdout]    9: test::run_test_in_process::{closure#0}
[INFO] [stdout]              at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/test/src/lib.rs:747:74
[INFO] [stdout]   10: <core::panic::unwind_safe::AssertUnwindSafe<test::run_test_in_process::{closure#0}> as core::ops::function::FnOnce<()>>::call_once
[INFO] [stdout]              at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/core/src/panic/unwind_safe.rs:275:9
[INFO] [stdout]   11: std::panicking::catch_unwind::do_call::<core::panic::unwind_safe::AssertUnwindSafe<test::run_test_in_process::{closure#0}>, core::result::Result<(), alloc::string::String>>
[INFO] [stdout]              at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/std/src/panicking.rs:576:43
[INFO] [stdout]   12: std::panicking::catch_unwind::<core::result::Result<(), alloc::string::String>, core::panic::unwind_safe::AssertUnwindSafe<test::run_test_in_process::{closure#0}>>
[INFO] [stdout]              at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/std/src/panicking.rs:544:19
[INFO] [stdout]   13: std::panic::catch_unwind::<core::panic::unwind_safe::AssertUnwindSafe<test::run_test_in_process::{closure#0}>, core::result::Result<(), alloc::string::String>>
[INFO] [stdout]              at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/std/src/panic.rs:359:14
[INFO] [stdout]   14: test::run_test_in_process
[INFO] [stdout]              at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/test/src/lib.rs:747:27
[INFO] [stdout]   15: test::run_test::{closure#0}
[INFO] [stdout]              at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/test/src/lib.rs:668:43
[INFO] [stdout]   16: test::run_test::{closure#1}
[INFO] [stdout]              at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/test/src/lib.rs:698:41
[INFO] [stdout]   17: std::sys::backtrace::__rust_begin_short_backtrace::<test::run_test::{closure#1}, ()>
[INFO] [stdout]              at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/std/src/sys/backtrace.rs:166:18
[INFO] [stdout]   18: std::thread::lifecycle::spawn_unchecked::<test::run_test::{closure#1}, ()>::{closure#1}::{closure#0}
[INFO] [stdout]              at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/std/src/thread/lifecycle.rs:70:13
[INFO] [stdout]   19: <core::panic::unwind_safe::AssertUnwindSafe<std::thread::lifecycle::spawn_unchecked<test::run_test::{closure#1}, ()>::{closure#1}::{closure#0}> as core::ops::function::FnOnce<()>>::call_once
[INFO] [stdout]              at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/core/src/panic/unwind_safe.rs:275:9
[INFO] [stdout]   20: std::panicking::catch_unwind::do_call::<core::panic::unwind_safe::AssertUnwindSafe<std::thread::lifecycle::spawn_unchecked<test::run_test::{closure#1}, ()>::{closure#1}::{closure#0}>, ()>
[INFO] [stdout]              at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/std/src/panicking.rs:576:43
[INFO] [stdout]   21: std::panicking::catch_unwind::<(), core::panic::unwind_safe::AssertUnwindSafe<std::thread::lifecycle::spawn_unchecked<test::run_test::{closure#1}, ()>::{closure#1}::{closure#0}>>
[INFO] [stdout]              at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/std/src/panicking.rs:544:19
[INFO] [stdout]   22: std::panic::catch_unwind::<core::panic::unwind_safe::AssertUnwindSafe<std::thread::lifecycle::spawn_unchecked<test::run_test::{closure#1}, ()>::{closure#1}::{closure#0}>, ()>
[INFO] [stdout]              at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/std/src/panic.rs:359:14
[INFO] [stdout]   23: std::thread::lifecycle::spawn_unchecked::<test::run_test::{closure#1}, ()>::{closure#1}
[INFO] [stdout]              at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/std/src/thread/lifecycle.rs:68:26
[INFO] [stdout]   24: <std::thread::lifecycle::spawn_unchecked<test::run_test::{closure#1}, ()>::{closure#1} as core::ops::function::FnOnce<()>>::call_once::{shim:vtable#0}
[INFO] [stdout]              at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/core/src/ops/function.rs:250:5
[INFO] [stdout]   25: <alloc::boxed::Box<dyn core::ops::function::FnOnce<(), Output = ()> + core::marker::Send> as core::ops::function::FnOnce<()>>::call_once
[INFO] [stdout]              at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/alloc/src/boxed.rs:2319:9
[INFO] [stdout]   26: <std::sys::thread::unix::Thread>::new::thread_start
[INFO] [stdout]              at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/std/src/sys/thread/unix.rs:123:17
[INFO] [stdout]   27: <unknown>
[INFO] [stdout]   28: clone
[INFO] [stdout] stack backtrace:
[INFO] [stdout]    0:     0x5d3f63f2ace1 - std[73adb7dc35730857]::backtrace_rs::backtrace::libunwind::trace
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/std/src/../../backtrace/src/backtrace/libunwind.rs:117:9
[INFO] [stdout]    1:     0x5d3f63f2ace1 - std[73adb7dc35730857]::backtrace_rs::backtrace::trace_unsynchronized::<std[73adb7dc35730857]::sys::backtrace::_print_fmt::{closure#1}>
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/std/src/../../backtrace/src/backtrace/mod.rs:66:14
[INFO] [stdout]    2:     0x5d3f63f2ace1 - std[73adb7dc35730857]::sys::backtrace::_print_fmt
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/std/src/sys/backtrace.rs:74:9
[INFO] [stdout]    3:     0x5d3f63f2ace1 - <<std[73adb7dc35730857]::sys::backtrace::BacktraceLock>::print::DisplayBacktrace as core[6883ba1bc0fe4ed1]::fmt::Display>::fmt
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/std/src/sys/backtrace.rs:44:26
[INFO] [stdout]    4:     0x5d3f63f3f7ba - <core[6883ba1bc0fe4ed1]::fmt::rt::Argument>::fmt
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/core/src/fmt/rt.rs:152:76
[INFO] [stdout]    5:     0x5d3f63f3f7ba - core[6883ba1bc0fe4ed1]::fmt::write
[INFO] [stdout]    6:     0x5d3f63f2f6bc - std[73adb7dc35730857]::io::default_write_fmt::<alloc[55a36b64bcbf2c0d]::vec::Vec<u8>>
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/std/src/io/mod.rs:626:11
[INFO] [stdout]    7:     0x5d3f63f2f6bc - <alloc[55a36b64bcbf2c0d]::vec::Vec<u8> as std[73adb7dc35730857]::io::Write>::write_fmt
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/std/src/io/mod.rs:1730:13
[INFO] [stdout]    8:     0x5d3f63f07ed6 - <std[73adb7dc35730857]::sys::backtrace::BacktraceLock>::print
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/std/src/sys/backtrace.rs:47:9
[INFO] [stdout]    9:     0x5d3f63f07ed6 - std[73adb7dc35730857]::panicking::default_hook::{closure#0}
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/std/src/panicking.rs:292:27
[INFO] [stdout]   10:     0x5d3f63f22e29 - std[73adb7dc35730857]::panicking::default_hook
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/std/src/panicking.rs:316:9
[INFO] [stdout]   11:     0x5d3f63e6fa20 - <alloc[55a36b64bcbf2c0d]::boxed::Box<dyn for<'a, 'b> core[6883ba1bc0fe4ed1]::ops::function::Fn<(&'a std[73adb7dc35730857]::panic::PanicHookInfo<'b>,), Output = ()> + core[6883ba1bc0fe4ed1]::marker::Send + core[6883ba1bc0fe4ed1]::marker::Sync> as core[6883ba1bc0fe4ed1]::ops::function::Fn<(&std[73adb7dc35730857]::panic::PanicHookInfo,)>>::call
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/alloc/src/boxed.rs:2333:9
[INFO] [stdout]   12:     0x5d3f63e6fa20 - test[980ffaebd391d06d]::test_main_inner::<test[980ffaebd391d06d]::test_main_static::{closure#0}>::{closure#0}
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/test/src/lib.rs:155:21
[INFO] [stdout]   13:     0x5d3f63f22fe2 - <alloc[55a36b64bcbf2c0d]::boxed::Box<dyn for<'a, 'b> core[6883ba1bc0fe4ed1]::ops::function::Fn<(&'a std[73adb7dc35730857]::panic::PanicHookInfo<'b>,), Output = ()> + core[6883ba1bc0fe4ed1]::marker::Send + core[6883ba1bc0fe4ed1]::marker::Sync> as core[6883ba1bc0fe4ed1]::ops::function::Fn<(&std[73adb7dc35730857]::panic::PanicHookInfo,)>>::call
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/alloc/src/boxed.rs:2333:9
[INFO] [stdout]   14:     0x5d3f63f22fe2 - std[73adb7dc35730857]::panicking::panic_with_hook
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/std/src/panicking.rs:823:13
[INFO] [stdout]   15:     0x5d3f63f07f82 - std[73adb7dc35730857]::panicking::panic_handler::{closure#0}
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/std/src/panicking.rs:688:13
[INFO] [stdout]   16:     0x5d3f63eff229 - std[73adb7dc35730857]::sys::backtrace::__rust_end_short_backtrace::<std[73adb7dc35730857]::panicking::panic_handler::{closure#0}, !>
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/std/src/sys/backtrace.rs:182:18
[INFO] [stdout]   17:     0x5d3f63f08c8d - __rustc[a7b7b02e776dd976]::rust_begin_unwind
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/std/src/panicking.rs:679:5
[INFO] [stdout]   18:     0x5d3f63f3ff8c - core[6883ba1bc0fe4ed1]::panicking::panic_fmt
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/core/src/panicking.rs:80:14
[INFO] [stdout]   19:     0x5d3f63f3fd02 - core[6883ba1bc0fe4ed1]::result::unwrap_failed
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/core/src/result.rs:1870:5
[INFO] [stdout]   20:     0x5d3f63e621a3 - <core[6883ba1bc0fe4ed1]::result::Result<std[73adb7dc35730857]::path::PathBuf, anyhow[3b9a07a6948cd063]::Error>>::expect
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/core/src/result.rs:1183:23
[INFO] [stdout]   21:     0x5d3f63e60b69 - e2e[f0516541adf4547c]::llama_cli
[INFO] [stdout]                                at /opt/rustwide/workdir/tests/e2e.rs:26:10
[INFO] [stdout]   22:     0x5d3f63e61ce4 - e2e[f0516541adf4547c]::test_e2e_different_seeds_both_complete
[INFO] [stdout]                                at /opt/rustwide/workdir/tests/e2e.rs:141:15
[INFO] [stdout]   23:     0x5d3f63e5ff87 - e2e[f0516541adf4547c]::test_e2e_different_seeds_both_complete::{closure#0}
[INFO] [stdout]                                at /opt/rustwide/workdir/tests/e2e.rs:140:44
[INFO] [stdout]   24:     0x5d3f63e62cb6 - <e2e[f0516541adf4547c]::test_e2e_different_seeds_both_complete::{closure#0} as core[6883ba1bc0fe4ed1]::ops::function::FnOnce<()>>::call_once
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/core/src/ops/function.rs:250:5
[INFO] [stdout]   25:     0x5d3f63e62d5b - <fn() -> core[6883ba1bc0fe4ed1]::result::Result<(), alloc[55a36b64bcbf2c0d]::string::String> as core[6883ba1bc0fe4ed1]::ops::function::FnOnce<()>>::call_once
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/core/src/ops/function.rs:250:5
[INFO] [stdout]   26:     0x5d3f63e62d5b - test[980ffaebd391d06d]::__rust_begin_short_backtrace::<core[6883ba1bc0fe4ed1]::result::Result<(), alloc[55a36b64bcbf2c0d]::string::String>, fn() -> core[6883ba1bc0fe4ed1]::result::Result<(), alloc[55a36b64bcbf2c0d]::string::String>>
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/test/src/lib.rs:724:18
[INFO] [stdout]   27:     0x5d3f63e703a5 - test[980ffaebd391d06d]::run_test_in_process::{closure#0}
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/test/src/lib.rs:747:74
[INFO] [stdout]   28:     0x5d3f63e703a5 - <core[6883ba1bc0fe4ed1]::panic::unwind_safe::AssertUnwindSafe<test[980ffaebd391d06d]::run_test_in_process::{closure#0}> as core[6883ba1bc0fe4ed1]::ops::function::FnOnce<()>>::call_once
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/core/src/panic/unwind_safe.rs:275:9
[INFO] [stdout]   29:     0x5d3f63e703a5 - std[73adb7dc35730857]::panicking::catch_unwind::do_call::<core[6883ba1bc0fe4ed1]::panic::unwind_safe::AssertUnwindSafe<test[980ffaebd391d06d]::run_test_in_process::{closure#0}>, core[6883ba1bc0fe4ed1]::result::Result<(), alloc[55a36b64bcbf2c0d]::string::String>>
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/std/src/panicking.rs:576:43
[INFO] [stdout]   30:     0x5d3f63e703a5 - std[73adb7dc35730857]::panicking::catch_unwind::<core[6883ba1bc0fe4ed1]::result::Result<(), alloc[55a36b64bcbf2c0d]::string::String>, core[6883ba1bc0fe4ed1]::panic::unwind_safe::AssertUnwindSafe<test[980ffaebd391d06d]::run_test_in_process::{closure#0}>>
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/std/src/panicking.rs:544:19
[INFO] [stdout]   31:     0x5d3f63e703a5 - std[73adb7dc35730857]::panic::catch_unwind::<core[6883ba1bc0fe4ed1]::panic::unwind_safe::AssertUnwindSafe<test[980ffaebd391d06d]::run_test_in_process::{closure#0}>, core[6883ba1bc0fe4ed1]::result::Result<(), alloc[55a36b64bcbf2c0d]::string::String>>
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/std/src/panic.rs:359:14
[INFO] [stdout]   32:     0x5d3f63e703a5 - test[980ffaebd391d06d]::run_test_in_process
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/test/src/lib.rs:747:27
[INFO] [stdout]   33:     0x5d3f63e703a5 - test[980ffaebd391d06d]::run_test::{closure#0}
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/test/src/lib.rs:668:43
[INFO] [stdout]   34:     0x5d3f63e6ae54 - test[980ffaebd391d06d]::run_test::{closure#1}
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/test/src/lib.rs:698:41
[INFO] [stdout]   35:     0x5d3f63e6ae54 - std[73adb7dc35730857]::sys::backtrace::__rust_begin_short_backtrace::<test[980ffaebd391d06d]::run_test::{closure#1}, ()>
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/std/src/sys/backtrace.rs:166:18
[INFO] [stdout]   36:     0x5d3f63e734f2 - std[73adb7dc35730857]::thread::lifecycle::spawn_unchecked::<test[980ffaebd391d06d]::run_test::{closure#1}, ()>::{closure#1}::{closure#0}
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/std/src/thread/lifecycle.rs:70:13
[INFO] [stdout]   37:     0x5d3f63e734f2 - <core[6883ba1bc0fe4ed1]::panic::unwind_safe::AssertUnwindSafe<std[73adb7dc35730857]::thread::lifecycle::spawn_unchecked<test[980ffaebd391d06d]::run_test::{closure#1}, ()>::{closure#1}::{closure#0}> as core[6883ba1bc0fe4ed1]::ops::function::FnOnce<()>>::call_once
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/core/src/panic/unwind_safe.rs:275:9
[INFO] [stdout]   38:     0x5d3f63e734f2 - std[73adb7dc35730857]::panicking::catch_unwind::do_call::<core[6883ba1bc0fe4ed1]::panic::unwind_safe::AssertUnwindSafe<std[73adb7dc35730857]::thread::lifecycle::spawn_unchecked<test[980ffaebd391d06d]::run_test::{closure#1}, ()>::{closure#1}::{closure#0}>, ()>
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/std/src/panicking.rs:576:43
[INFO] [stdout]   39:     0x5d3f63e734f2 - std[73adb7dc35730857]::panicking::catch_unwind::<(), core[6883ba1bc0fe4ed1]::panic::unwind_safe::AssertUnwindSafe<std[73adb7dc35730857]::thread::lifecycle::spawn_unchecked<test[980ffaebd391d06d]::run_test::{closure#1}, ()>::{closure#1}::{closure#0}>>
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/std/src/panicking.rs:544:19
[INFO] [stdout]   40:     0x5d3f63e734f2 - std[73adb7dc35730857]::panic::catch_unwind::<core[6883ba1bc0fe4ed1]::panic::unwind_safe::AssertUnwindSafe<std[73adb7dc35730857]::thread::lifecycle::spawn_unchecked<test[980ffaebd391d06d]::run_test::{closure#1}, ()>::{closure#1}::{closure#0}>, ()>
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/std/src/panic.rs:359:14
[INFO] [stdout]   41:     0x5d3f63e734f2 - std[73adb7dc35730857]::thread::lifecycle::spawn_unchecked::<test[980ffaebd391d06d]::run_test::{closure#1}, ()>::{closure#1}
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/std/src/thread/lifecycle.rs:68:26
[INFO] [stdout]   42:     0x5d3f63e734f2 - <std[73adb7dc35730857]::thread::lifecycle::spawn_unchecked<test[980ffaebd391d06d]::run_test::{closure#1}, ()>::{closure#1} as core[6883ba1bc0fe4ed1]::ops::function::FnOnce<()>>::call_once::{shim:vtable#0}
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/core/src/ops/function.rs:250:5
[INFO] [stdout]   43:     0x5d3f63f2a56f - <alloc[55a36b64bcbf2c0d]::boxed::Box<dyn core[6883ba1bc0fe4ed1]::ops::function::FnOnce<(), Output = ()> + core[6883ba1bc0fe4ed1]::marker::Send> as core[6883ba1bc0fe4ed1]::ops::function::FnOnce<()>>::call_once
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/alloc/src/boxed.rs:2319:9
[INFO] [stdout]   44:     0x5d3f63f2a56f - <std[73adb7dc35730857]::sys::thread::unix::Thread>::new::thread_start
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/std/src/sys/thread/unix.rs:123:17
[INFO] [stdout]   45:     0x7fac29384aa4 - <unknown>
[INFO] [stdout]   46:     0x7fac29411a64 - clone
[INFO] [stdout]   47:                0x0 - <unknown>
[INFO] [stdout] 
[INFO] [stdout] ---- test_e2e_same_seed_same_token_count stdout ----
[INFO] [stdout] 
[INFO] [stdout] thread 'test_e2e_same_seed_same_token_count' (1939) panicked at tests/e2e.rs:26:10:
[INFO] [stdout] llama-cli not found. Install llama.cpp and add it to PATH.
[INFO] [stdout] Download: https://github.com/ggerganov/llama.cpp/releases: Cannot find 'llama-cli'. Install llama.cpp and ensure it is on PATH, or download prebuilt binaries to /root/.cache/llm-benchmark/llama-bin .
[INFO] [stdout] See: https://github.com/ggerganov/llama.cpp/releases
[INFO] [stdout] 
[INFO] [stdout] Stack backtrace:
[INFO] [stdout]    0: <anyhow::Error>::msg::<alloc::string::String>
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/anyhow-1.0.102/src/backtrace.rs:10:14
[INFO] [stdout]    1: llmb::assets::find_exe
[INFO] [stdout]              at ./src/assets.rs:59:5
[INFO] [stdout]    2: llmb::assets::find_llama_cli
[INFO] [stdout]              at ./src/assets.rs:42:5
[INFO] [stdout]    3: e2e::llama_cli
[INFO] [stdout]              at ./tests/e2e.rs:25:5
[INFO] [stdout]    4: e2e::test_e2e_same_seed_same_token_count
[INFO] [stdout]              at ./tests/e2e.rs:120:15
[INFO] [stdout]    5: e2e::test_e2e_same_seed_same_token_count::{closure#0}
[INFO] [stdout]              at ./tests/e2e.rs:119:41
[INFO] [stdout]    6: <e2e::test_e2e_same_seed_same_token_count::{closure#0} as core::ops::function::FnOnce<()>>::call_once
[INFO] [stdout]              at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/core/src/ops/function.rs:250:5
[INFO] [stdout]    7: <fn() -> core::result::Result<(), alloc::string::String> as core::ops::function::FnOnce<()>>::call_once
[INFO] [stdout]              at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/core/src/ops/function.rs:250:5
[INFO] [stdout]    8: test::__rust_begin_short_backtrace::<core::result::Result<(), alloc::string::String>, fn() -> core::result::Result<(), alloc::string::String>>
[INFO] [stdout]              at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/test/src/lib.rs:724:18
[INFO] [stdout]    9: test::run_test_in_process::{closure#0}
[INFO] [stdout]              at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/test/src/lib.rs:747:74
[INFO] [stdout]   10: <core::panic::unwind_safe::AssertUnwindSafe<test::run_test_in_process::{closure#0}> as core::ops::function::FnOnce<()>>::call_once
[INFO] [stdout]              at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/core/src/panic/unwind_safe.rs:275:9
[INFO] [stdout]   11: std::panicking::catch_unwind::do_call::<core::panic::unwind_safe::AssertUnwindSafe<test::run_test_in_process::{closure#0}>, core::result::Result<(), alloc::string::String>>
[INFO] [stdout]              at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/std/src/panicking.rs:576:43
[INFO] [stdout]   12: std::panicking::catch_unwind::<core::result::Result<(), alloc::string::String>, core::panic::unwind_safe::AssertUnwindSafe<test::run_test_in_process::{closure#0}>>
[INFO] [stdout]              at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/std/src/panicking.rs:544:19
[INFO] [stdout]   13: std::panic::catch_unwind::<core::panic::unwind_safe::AssertUnwindSafe<test::run_test_in_process::{closure#0}>, core::result::Result<(), alloc::string::String>>
[INFO] [stdout]              at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/std/src/panic.rs:359:14
[INFO] [stdout]   14: test::run_test_in_process
[INFO] [stdout]              at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/test/src/lib.rs:747:27
[INFO] [stdout]   15: test::run_test::{closure#0}
[INFO] [stdout]              at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/test/src/lib.rs:668:43
[INFO] [stdout]   16: test::run_test::{closure#1}
[INFO] [stdout]              at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/test/src/lib.rs:698:41
[INFO] [stdout]   17: std::sys::backtrace::__rust_begin_short_backtrace::<test::run_test::{closure#1}, ()>
[INFO] [stdout]              at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/std/src/sys/backtrace.rs:166:18
[INFO] [stdout]   18: std::thread::lifecycle::spawn_unchecked::<test::run_test::{closure#1}, ()>::{closure#1}::{closure#0}
[INFO] [stdout]              at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/std/src/thread/lifecycle.rs:70:13
[INFO] [stdout]   19: <core::panic::unwind_safe::AssertUnwindSafe<std::thread::lifecycle::spawn_unchecked<test::run_test::{closure#1}, ()>::{closure#1}::{closure#0}> as core::ops::function::FnOnce<()>>::call_once
[INFO] [stdout]              at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/core/src/panic/unwind_safe.rs:275:9
[INFO] [stdout]   20: std::panicking::catch_unwind::do_call::<core::panic::unwind_safe::AssertUnwindSafe<std::thread::lifecycle::spawn_unchecked<test::run_test::{closure#1}, ()>::{closure#1}::{closure#0}>, ()>
[INFO] [stdout]              at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/std/src/panicking.rs:576:43
[INFO] [stdout]   21: std::panicking::catch_unwind::<(), core::panic::unwind_safe::AssertUnwindSafe<std::thread::lifecycle::spawn_unchecked<test::run_test::{closure#1}, ()>::{closure#1}::{closure#0}>>
[INFO] [stdout]              at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/std/src/panicking.rs:544:19
[INFO] [stdout]   22: std::panic::catch_unwind::<core::panic::unwind_safe::AssertUnwindSafe<std::thread::lifecycle::spawn_unchecked<test::run_test::{closure#1}, ()>::{closure#1}::{closure#0}>, ()>
[INFO] [stdout]              at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/std/src/panic.rs:359:14
[INFO] [stdout]   23: std::thread::lifecycle::spawn_unchecked::<test::run_test::{closure#1}, ()>::{closure#1}
[INFO] [stdout]              at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/std/src/thread/lifecycle.rs:68:26
[INFO] [stdout]   24: <std::thread::lifecycle::spawn_unchecked<test::run_test::{closure#1}, ()>::{closure#1} as core::ops::function::FnOnce<()>>::call_once::{shim:vtable#0}
[INFO] [stdout]              at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/core/src/ops/function.rs:250:5
[INFO] [stdout]   25: <alloc::boxed::Box<dyn core::ops::function::FnOnce<(), Output = ()> + core::marker::Send> as core::ops::function::FnOnce<()>>::call_once
[INFO] [stdout]              at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/alloc/src/boxed.rs:2319:9
[INFO] [stdout]   26: <std::sys::thread::unix::Thread>::new::thread_start
[INFO] [stdout]              at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/std/src/sys/thread/unix.rs:123:17
[INFO] [stdout]   27: <unknown>
[INFO] [stdout]   28: clone
[INFO] [stdout] stack backtrace:
[INFO] [stdout]    0:     0x5d3f63f2ace1 - std[73adb7dc35730857]::backtrace_rs::backtrace::libunwind::trace
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/std/src/../../backtrace/src/backtrace/libunwind.rs:117:9
[INFO] [stdout]    1:     0x5d3f63f2ace1 - std[73adb7dc35730857]::backtrace_rs::backtrace::trace_unsynchronized::<std[73adb7dc35730857]::sys::backtrace::_print_fmt::{closure#1}>
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/std/src/../../backtrace/src/backtrace/mod.rs:66:14
[INFO] [stdout]    2:     0x5d3f63f2ace1 - std[73adb7dc35730857]::sys::backtrace::_print_fmt
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/std/src/sys/backtrace.rs:74:9
[INFO] [stdout]    3:     0x5d3f63f2ace1 - <<std[73adb7dc35730857]::sys::backtrace::BacktraceLock>::print::DisplayBacktrace as core[6883ba1bc0fe4ed1]::fmt::Display>::fmt
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/std/src/sys/backtrace.rs:44:26
[INFO] [stdout]    4:     0x5d3f63f3f7ba - <core[6883ba1bc0fe4ed1]::fmt::rt::Argument>::fmt
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/core/src/fmt/rt.rs:152:76
[INFO] [stdout]    5:     0x5d3f63f3f7ba - core[6883ba1bc0fe4ed1]::fmt::write
[INFO] [stdout]    6:     0x5d3f63f2f6bc - std[73adb7dc35730857]::io::default_write_fmt::<alloc[55a36b64bcbf2c0d]::vec::Vec<u8>>
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/std/src/io/mod.rs:626:11
[INFO] [stdout]    7:     0x5d3f63f2f6bc - <alloc[55a36b64bcbf2c0d]::vec::Vec<u8> as std[73adb7dc35730857]::io::Write>::write_fmt
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/std/src/io/mod.rs:1730:13
[INFO] [stdout]    8:     0x5d3f63f07ed6 - <std[73adb7dc35730857]::sys::backtrace::BacktraceLock>::print
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/std/src/sys/backtrace.rs:47:9
[INFO] [stdout]    9:     0x5d3f63f07ed6 - std[73adb7dc35730857]::panicking::default_hook::{closure#0}
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/std/src/panicking.rs:292:27
[INFO] [stdout]   10:     0x5d3f63f22e29 - std[73adb7dc35730857]::panicking::default_hook
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/std/src/panicking.rs:316:9
[INFO] [stdout]   11:     0x5d3f63e6fa20 - <alloc[55a36b64bcbf2c0d]::boxed::Box<dyn for<'a, 'b> core[6883ba1bc0fe4ed1]::ops::function::Fn<(&'a std[73adb7dc35730857]::panic::PanicHookInfo<'b>,), Output = ()> + core[6883ba1bc0fe4ed1]::marker::Send + core[6883ba1bc0fe4ed1]::marker::Sync> as core[6883ba1bc0fe4ed1]::ops::function::Fn<(&std[73adb7dc35730857]::panic::PanicHookInfo,)>>::call
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/alloc/src/boxed.rs:2333:9
[INFO] [stdout]   12:     0x5d3f63e6fa20 - test[980ffaebd391d06d]::test_main_inner::<test[980ffaebd391d06d]::test_main_static::{closure#0}>::{closure#0}
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/test/src/lib.rs:155:21
[INFO] [stdout]   13:     0x5d3f63f22fe2 - <alloc[55a36b64bcbf2c0d]::boxed::Box<dyn for<'a, 'b> core[6883ba1bc0fe4ed1]::ops::function::Fn<(&'a std[73adb7dc35730857]::panic::PanicHookInfo<'b>,), Output = ()> + core[6883ba1bc0fe4ed1]::marker::Send + core[6883ba1bc0fe4ed1]::marker::Sync> as core[6883ba1bc0fe4ed1]::ops::function::Fn<(&std[73adb7dc35730857]::panic::PanicHookInfo,)>>::call
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/alloc/src/boxed.rs:2333:9
[INFO] [stdout]   14:     0x5d3f63f22fe2 - std[73adb7dc35730857]::panicking::panic_with_hook
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/std/src/panicking.rs:823:13
[INFO] [stdout]   15:     0x5d3f63f07f82 - std[73adb7dc35730857]::panicking::panic_handler::{closure#0}
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/std/src/panicking.rs:688:13
[INFO] [stdout]   16:     0x5d3f63eff229 - std[73adb7dc35730857]::sys::backtrace::__rust_end_short_backtrace::<std[73adb7dc35730857]::panicking::panic_handler::{closure#0}, !>
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/std/src/sys/backtrace.rs:182:18
[INFO] [stdout]   17:     0x5d3f63f08c8d - __rustc[a7b7b02e776dd976]::rust_begin_unwind
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/std/src/panicking.rs:679:5
[INFO] [stdout]   18:     0x5d3f63f3ff8c - core[6883ba1bc0fe4ed1]::panicking::panic_fmt
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/core/src/panicking.rs:80:14
[INFO] [stdout]   19:     0x5d3f63f3fd02 - core[6883ba1bc0fe4ed1]::result::unwrap_failed
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/core/src/result.rs:1870:5
[INFO] [stdout]   20:     0x5d3f63e621a3 - <core[6883ba1bc0fe4ed1]::result::Result<std[73adb7dc35730857]::path::PathBuf, anyhow[3b9a07a6948cd063]::Error>>::expect
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/core/src/result.rs:1183:23
[INFO] [stdout]   21:     0x5d3f63e60b69 - e2e[f0516541adf4547c]::llama_cli
[INFO] [stdout]                                at /opt/rustwide/workdir/tests/e2e.rs:26:10
[INFO] [stdout]   22:     0x5d3f63e61284 - e2e[f0516541adf4547c]::test_e2e_same_seed_same_token_count
[INFO] [stdout]                                at /opt/rustwide/workdir/tests/e2e.rs:120:15
[INFO] [stdout]   23:     0x5d3f63e5ff27 - e2e[f0516541adf4547c]::test_e2e_same_seed_same_token_count::{closure#0}
[INFO] [stdout]                                at /opt/rustwide/workdir/tests/e2e.rs:119:41
[INFO] [stdout]   24:     0x5d3f63e62c36 - <e2e[f0516541adf4547c]::test_e2e_same_seed_same_token_count::{closure#0} as core[6883ba1bc0fe4ed1]::ops::function::FnOnce<()>>::call_once
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/core/src/ops/function.rs:250:5
[INFO] [stdout]   25:     0x5d3f63e62d5b - <fn() -> core[6883ba1bc0fe4ed1]::result::Result<(), alloc[55a36b64bcbf2c0d]::string::String> as core[6883ba1bc0fe4ed1]::ops::function::FnOnce<()>>::call_once
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/core/src/ops/function.rs:250:5
[INFO] [stdout]   26:     0x5d3f63e62d5b - test[980ffaebd391d06d]::__rust_begin_short_backtrace::<core[6883ba1bc0fe4ed1]::result::Result<(), alloc[55a36b64bcbf2c0d]::string::String>, fn() -> core[6883ba1bc0fe4ed1]::result::Result<(), alloc[55a36b64bcbf2c0d]::string::String>>
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/test/src/lib.rs:724:18
[INFO] [stdout]   27:     0x5d3f63e703a5 - test[980ffaebd391d06d]::run_test_in_process::{closure#0}
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/test/src/lib.rs:747:74
[INFO] [stdout]   28:     0x5d3f63e703a5 - <core[6883ba1bc0fe4ed1]::panic::unwind_safe::AssertUnwindSafe<test[980ffaebd391d06d]::run_test_in_process::{closure#0}> as core[6883ba1bc0fe4ed1]::ops::function::FnOnce<()>>::call_once
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/core/src/panic/unwind_safe.rs:275:9
[INFO] [stdout]   29:     0x5d3f63e703a5 - std[73adb7dc35730857]::panicking::catch_unwind::do_call::<core[6883ba1bc0fe4ed1]::panic::unwind_safe::AssertUnwindSafe<test[980ffaebd391d06d]::run_test_in_process::{closure#0}>, core[6883ba1bc0fe4ed1]::result::Result<(), alloc[55a36b64bcbf2c0d]::string::String>>
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/std/src/panicking.rs:576:43
[INFO] [stdout]   30:     0x5d3f63e703a5 - std[73adb7dc35730857]::panicking::catch_unwind::<core[6883ba1bc0fe4ed1]::result::Result<(), alloc[55a36b64bcbf2c0d]::string::String>, core[6883ba1bc0fe4ed1]::panic::unwind_safe::AssertUnwindSafe<test[980ffaebd391d06d]::run_test_in_process::{closure#0}>>
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/std/src/panicking.rs:544:19
[INFO] [stdout]   31:     0x5d3f63e703a5 - std[73adb7dc35730857]::panic::catch_unwind::<core[6883ba1bc0fe4ed1]::panic::unwind_safe::AssertUnwindSafe<test[980ffaebd391d06d]::run_test_in_process::{closure#0}>, core[6883ba1bc0fe4ed1]::result::Result<(), alloc[55a36b64bcbf2c0d]::string::String>>
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/std/src/panic.rs:359:14
[INFO] [stdout]   32:     0x5d3f63e703a5 - test[980ffaebd391d06d]::run_test_in_process
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/test/src/lib.rs:747:27
[INFO] [stdout]   33:     0x5d3f63e703a5 - test[980ffaebd391d06d]::run_test::{closure#0}
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/test/src/lib.rs:668:43
[INFO] [stdout]   34:     0x5d3f63e6ae54 - test[980ffaebd391d06d]::run_test::{closure#1}
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/test/src/lib.rs:698:41
[INFO] [stdout]   35:     0x5d3f63e6ae54 - std[73adb7dc35730857]::sys::backtrace::__rust_begin_short_backtrace::<test[980ffaebd391d06d]::run_test::{closure#1}, ()>
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/std/src/sys/backtrace.rs:166:18
[INFO] [stdout]   36:     0x5d3f63e734f2 - std[73adb7dc35730857]::thread::lifecycle::spawn_unchecked::<test[980ffaebd391d06d]::run_test::{closure#1}, ()>::{closure#1}::{closure#0}
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/std/src/thread/lifecycle.rs:70:13
[INFO] [stdout]   37:     0x5d3f63e734f2 - <core[6883ba1bc0fe4ed1]::panic::unwind_safe::AssertUnwindSafe<std[73adb7dc35730857]::thread::lifecycle::spawn_unchecked<test[980ffaebd391d06d]::run_test::{closure#1}, ()>::{closure#1}::{closure#0}> as core[6883ba1bc0fe4ed1]::ops::function::FnOnce<()>>::call_once
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/core/src/panic/unwind_safe.rs:275:9
[INFO] [stdout]   38:     0x5d3f63e734f2 - std[73adb7dc35730857]::panicking::catch_unwind::do_call::<core[6883ba1bc0fe4ed1]::panic::unwind_safe::AssertUnwindSafe<std[73adb7dc35730857]::thread::lifecycle::spawn_unchecked<test[980ffaebd391d06d]::run_test::{closure#1}, ()>::{closure#1}::{closure#0}>, ()>
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/std/src/panicking.rs:576:43
[INFO] [stdout]   39:     0x5d3f63e734f2 - std[73adb7dc35730857]::panicking::catch_unwind::<(), core[6883ba1bc0fe4ed1]::panic::unwind_safe::AssertUnwindSafe<std[73adb7dc35730857]::thread::lifecycle::spawn_unchecked<test[980ffaebd391d06d]::run_test::{closure#1}, ()>::{closure#1}::{closure#0}>>
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/std/src/panicking.rs:544:19
[INFO] [stdout]   40:     0x5d3f63e734f2 - std[73adb7dc35730857]::panic::catch_unwind::<core[6883ba1bc0fe4ed1]::panic::unwind_safe::AssertUnwindSafe<std[73adb7dc35730857]::thread::lifecycle::spawn_unchecked<test[980ffaebd391d06d]::run_test::{closure#1}, ()>::{closure#1}::{closure#0}>, ()>
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/std/src/panic.rs:359:14
[INFO] [stdout]   41:     0x5d3f63e734f2 - std[73adb7dc35730857]::thread::lifecycle::spawn_unchecked::<test[980ffaebd391d06d]::run_test::{closure#1}, ()>::{closure#1}
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/std/src/thread/lifecycle.rs:68:26
[INFO] [stdout]   42:     0x5d3f63e734f2 - <std[73adb7dc35730857]::thread::lifecycle::spawn_unchecked<test[980ffaebd391d06d]::run_test::{closure#1}, ()>::{closure#1} as core[6883ba1bc0fe4ed1]::ops::function::FnOnce<()>>::call_once::{shim:vtable#0}
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/core/src/ops/function.rs:250:5
[INFO] [stdout]   43:     0x5d3f63f2a56f - <alloc[55a36b64bcbf2c0d]::boxed::Box<dyn core[6883ba1bc0fe4ed1]::ops::function::FnOnce<(), Output = ()> + core[6883ba1bc0fe4ed1]::marker::Send> as core[6883ba1bc0fe4ed1]::ops::function::FnOnce<()>>::call_once
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/alloc/src/boxed.rs:2319:9
[INFO] [stdout]   44:     0x5d3f63f2a56f - <std[73adb7dc35730857]::sys::thread::unix::Thread>::new::thread_start
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/std/src/sys/thread/unix.rs:123:17
[INFO] [stdout]   45:     0x7fac29384aa4 - <unknown>
[INFO] [stdout]   46:     0x7fac29411a64 - clone
[INFO] [stdout]   47:                0x0 - <unknown>
[INFO] [stdout] 
[INFO] [stdout] ---- test_e2e_aggregated_metrics_are_sane stdout ----
[INFO] [stdout] 
[INFO] [stdout] thread 'test_e2e_aggregated_metrics_are_sane' (1936) panicked at tests/e2e.rs:26:10:
[INFO] [stdout] llama-cli not found. Install llama.cpp and add it to PATH.
[INFO] [stdout] Download: https://github.com/ggerganov/llama.cpp/releases: Cannot find 'llama-cli'. Install llama.cpp and ensure it is on PATH, or download prebuilt binaries to /root/.cache/llm-benchmark/llama-bin .
[INFO] [stdout] See: https://github.com/ggerganov/llama.cpp/releases
[INFO] [stdout] 
[INFO] [stdout] Stack backtrace:
[INFO] [stdout]    0: <anyhow::Error>::msg::<alloc::string::String>
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/anyhow-1.0.102/src/backtrace.rs:10:14
[INFO] [stdout]    1: llmb::assets::find_exe
[INFO] [stdout]              at ./src/assets.rs:59:5
[INFO] [stdout]    2: llmb::assets::find_llama_cli
[INFO] [stdout]              at ./src/assets.rs:42:5
[INFO] [stdout]    3: e2e::llama_cli
[INFO] [stdout]              at ./tests/e2e.rs:25:5
[INFO] [stdout]    4: e2e::test_e2e_aggregated_metrics_are_sane
[INFO] [stdout]              at ./tests/e2e.rs:158:15
[INFO] [stdout]    5: e2e::test_e2e_aggregated_metrics_are_sane::{closure#0}
[INFO] [stdout]              at ./tests/e2e.rs:157:42
[INFO] [stdout]    6: <e2e::test_e2e_aggregated_metrics_are_sane::{closure#0} as core::ops::function::FnOnce<()>>::call_once
[INFO] [stdout]              at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/core/src/ops/function.rs:250:5
[INFO] [stdout]    7: <fn() -> core::result::Result<(), alloc::string::String> as core::ops::function::FnOnce<()>>::call_once
[INFO] [stdout]              at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/core/src/ops/function.rs:250:5
[INFO] [stdout]    8: test::__rust_begin_short_backtrace::<core::result::Result<(), alloc::string::String>, fn() -> core::result::Result<(), alloc::string::String>>
[INFO] [stdout]              at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/test/src/lib.rs:724:18
[INFO] [stdout]    9: test::run_test_in_process::{closure#0}
[INFO] [stdout]              at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/test/src/lib.rs:747:74
[INFO] [stdout]   10: <core::panic::unwind_safe::AssertUnwindSafe<test::run_test_in_process::{closure#0}> as core::ops::function::FnOnce<()>>::call_once
[INFO] [stdout]              at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/core/src/panic/unwind_safe.rs:275:9
[INFO] [stdout]   11: std::panicking::catch_unwind::do_call::<core::panic::unwind_safe::AssertUnwindSafe<test::run_test_in_process::{closure#0}>, core::result::Result<(), alloc::string::String>>
[INFO] [stdout]              at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/std/src/panicking.rs:576:43
[INFO] [stdout]   12: std::panicking::catch_unwind::<core::result::Result<(), alloc::string::String>, core::panic::unwind_safe::AssertUnwindSafe<test::run_test_in_process::{closure#0}>>
[INFO] [stdout]              at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/std/src/panicking.rs:544:19
[INFO] [stdout]   13: std::panic::catch_unwind::<core::panic::unwind_safe::AssertUnwindSafe<test::run_test_in_process::{closure#0}>, core::result::Result<(), alloc::string::String>>
[INFO] [stdout]              at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/std/src/panic.rs:359:14
[INFO] [stdout]   14: test::run_test_in_process
[INFO] [stdout]              at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/test/src/lib.rs:747:27
[INFO] [stdout]   15: test::run_test::{closure#0}
[INFO] [stdout]              at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/test/src/lib.rs:668:43
[INFO] [stdout]   16: test::run_test::{closure#1}
[INFO] [stdout]              at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/test/src/lib.rs:698:41
[INFO] [stdout]   17: std::sys::backtrace::__rust_begin_short_backtrace::<test::run_test::{closure#1}, ()>
[INFO] [stdout]              at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/std/src/sys/backtrace.rs:166:18
[INFO] [stdout]   18: std::thread::lifecycle::spawn_unchecked::<test::run_test::{closure#1}, ()>::{closure#1}::{closure#0}
[INFO] [stdout]              at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/std/src/thread/lifecycle.rs:70:13
[INFO] [stdout]   19: <core::panic::unwind_safe::AssertUnwindSafe<std::thread::lifecycle::spawn_unchecked<test::run_test::{closure#1}, ()>::{closure#1}::{closure#0}> as core::ops::function::FnOnce<()>>::call_once
[INFO] [stdout]              at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/core/src/panic/unwind_safe.rs:275:9
[INFO] [stdout]   20: std::panicking::catch_unwind::do_call::<core::panic::unwind_safe::AssertUnwindSafe<std::thread::lifecycle::spawn_unchecked<test::run_test::{closure#1}, ()>::{closure#1}::{closure#0}>, ()>
[INFO] [stdout]              at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/std/src/panicking.rs:576:43
[INFO] [stdout]   21: std::panicking::catch_unwind::<(), core::panic::unwind_safe::AssertUnwindSafe<std::thread::lifecycle::spawn_unchecked<test::run_test::{closure#1}, ()>::{closure#1}::{closure#0}>>
[INFO] [stdout]              at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/std/src/panicking.rs:544:19
[INFO] [stdout]   22: std::panic::catch_unwind::<core::panic::unwind_safe::AssertUnwindSafe<std::thread::lifecycle::spawn_unchecked<test::run_test::{closure#1}, ()>::{closure#1}::{closure#0}>, ()>
[INFO] [stdout]              at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/std/src/panic.rs:359:14
[INFO] [stdout]   23: std::thread::lifecycle::spawn_unchecked::<test::run_test::{closure#1}, ()>::{closure#1}
[INFO] [stdout]              at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/std/src/thread/lifecycle.rs:68:26
[INFO] [stdout]   24: <std::thread::lifecycle::spawn_unchecked<test::run_test::{closure#1}, ()>::{closure#1} as core::ops::function::FnOnce<()>>::call_once::{shim:vtable#0}
[INFO] [stdout]              at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/core/src/ops/function.rs:250:5
[INFO] [stdout]   25: <alloc::boxed::Box<dyn core::ops::function::FnOnce<(), Output = ()> + core::marker::Send> as core::ops::function::FnOnce<()>>::call_once
[INFO] [stdout]              at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/alloc/src/boxed.rs:2319:9
[INFO] [stdout]   26: <std::sys::thread::unix::Thread>::new::thread_start
[INFO] [stdout]              at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/std/src/sys/thread/unix.rs:123:17
[INFO] [stdout]   27: <unknown>
[INFO] [stdout]   28: clone
[INFO] [stdout] stack backtrace:
[INFO] [stdout]    0:     0x5d3f63f2ace1 - std[73adb7dc35730857]::backtrace_rs::backtrace::libunwind::trace
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/std/src/../../backtrace/src/backtrace/libunwind.rs:117:9
[INFO] [stdout]    1:     0x5d3f63f2ace1 - std[73adb7dc35730857]::backtrace_rs::backtrace::trace_unsynchronized::<std[73adb7dc35730857]::sys::backtrace::_print_fmt::{closure#1}>
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/std/src/../../backtrace/src/backtrace/mod.rs:66:14
[INFO] [stdout]    2:     0x5d3f63f2ace1 - std[73adb7dc35730857]::sys::backtrace::_print_fmt
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/std/src/sys/backtrace.rs:74:9
[INFO] [stdout]    3:     0x5d3f63f2ace1 - <<std[73adb7dc35730857]::sys::backtrace::BacktraceLock>::print::DisplayBacktrace as core[6883ba1bc0fe4ed1]::fmt::Display>::fmt
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/std/src/sys/backtrace.rs:44:26
[INFO] [stdout]    4:     0x5d3f63f3f7ba - <core[6883ba1bc0fe4ed1]::fmt::rt::Argument>::fmt
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/core/src/fmt/rt.rs:152:76
[INFO] [stdout]    5:     0x5d3f63f3f7ba - core[6883ba1bc0fe4ed1]::fmt::write
[INFO] [stdout]    6:     0x5d3f63f2f6bc - std[73adb7dc35730857]::io::default_write_fmt::<alloc[55a36b64bcbf2c0d]::vec::Vec<u8>>
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/std/src/io/mod.rs:626:11
[INFO] [stdout]    7:     0x5d3f63f2f6bc - <alloc[55a36b64bcbf2c0d]::vec::Vec<u8> as std[73adb7dc35730857]::io::Write>::write_fmt
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/std/src/io/mod.rs:1730:13
[INFO] [stdout]    8:     0x5d3f63f07ed6 - <std[73adb7dc35730857]::sys::backtrace::BacktraceLock>::print
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/std/src/sys/backtrace.rs:47:9
[INFO] [stdout]    9:     0x5d3f63f07ed6 - std[73adb7dc35730857]::panicking::default_hook::{closure#0}
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/std/src/panicking.rs:292:27
[INFO] [stdout]   10:     0x5d3f63f22e29 - std[73adb7dc35730857]::panicking::default_hook
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/std/src/panicking.rs:316:9
[INFO] [stdout]   11:     0x5d3f63e6fa20 - <alloc[55a36b64bcbf2c0d]::boxed::Box<dyn for<'a, 'b> core[6883ba1bc0fe4ed1]::ops::function::Fn<(&'a std[73adb7dc35730857]::panic::PanicHookInfo<'b>,), Output = ()> + core[6883ba1bc0fe4ed1]::marker::Send + core[6883ba1bc0fe4ed1]::marker::Sync> as core[6883ba1bc0fe4ed1]::ops::function::Fn<(&std[73adb7dc35730857]::panic::PanicHookInfo,)>>::call
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/alloc/src/boxed.rs:2333:9
[INFO] [stdout]   12:     0x5d3f63e6fa20 - test[980ffaebd391d06d]::test_main_inner::<test[980ffaebd391d06d]::test_main_static::{closure#0}>::{closure#0}
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/test/src/lib.rs:155:21
[INFO] [stdout]   13:     0x5d3f63f22fe2 - <alloc[55a36b64bcbf2c0d]::boxed::Box<dyn for<'a, 'b> core[6883ba1bc0fe4ed1]::ops::function::Fn<(&'a std[73adb7dc35730857]::panic::PanicHookInfo<'b>,), Output = ()> + core[6883ba1bc0fe4ed1]::marker::Send + core[6883ba1bc0fe4ed1]::marker::Sync> as core[6883ba1bc0fe4ed1]::ops::function::Fn<(&std[73adb7dc35730857]::panic::PanicHookInfo,)>>::call
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/alloc/src/boxed.rs:2333:9
[INFO] [stdout]   14:     0x5d3f63f22fe2 - std[73adb7dc35730857]::panicking::panic_with_hook
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/std/src/panicking.rs:823:13
[INFO] [stdout]   15:     0x5d3f63f07f82 - std[73adb7dc35730857]::panicking::panic_handler::{closure#0}
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/std/src/panicking.rs:688:13
[INFO] [stdout]   16:     0x5d3f63eff229 - std[73adb7dc35730857]::sys::backtrace::__rust_end_short_backtrace::<std[73adb7dc35730857]::panicking::panic_handler::{closure#0}, !>
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/std/src/sys/backtrace.rs:182:18
[INFO] [stdout]   17:     0x5d3f63f08c8d - __rustc[a7b7b02e776dd976]::rust_begin_unwind
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/std/src/panicking.rs:679:5
[INFO] [stdout]   18:     0x5d3f63f3ff8c - core[6883ba1bc0fe4ed1]::panicking::panic_fmt
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/core/src/panicking.rs:80:14
[INFO] [stdout]   19:     0x5d3f63f3fd02 - core[6883ba1bc0fe4ed1]::result::unwrap_failed
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/core/src/result.rs:1870:5
[INFO] [stdout]   20:     0x5d3f63e621a3 - <core[6883ba1bc0fe4ed1]::result::Result<std[73adb7dc35730857]::path::PathBuf, anyhow[3b9a07a6948cd063]::Error>>::expect
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/core/src/result.rs:1183:23
[INFO] [stdout]   21:     0x5d3f63e60b69 - e2e[f0516541adf4547c]::llama_cli
[INFO] [stdout]                                at /opt/rustwide/workdir/tests/e2e.rs:26:10
[INFO] [stdout]   22:     0x5d3f63e61781 - e2e[f0516541adf4547c]::test_e2e_aggregated_metrics_are_sane
[INFO] [stdout]                                at /opt/rustwide/workdir/tests/e2e.rs:158:15
[INFO] [stdout]   23:     0x5d3f63e5ff57 - e2e[f0516541adf4547c]::test_e2e_aggregated_metrics_are_sane::{closure#0}
[INFO] [stdout]                                at /opt/rustwide/workdir/tests/e2e.rs:157:42
[INFO] [stdout]   24:     0x5d3f63e62c76 - <e2e[f0516541adf4547c]::test_e2e_aggregated_metrics_are_sane::{closure#0} as core[6883ba1bc0fe4ed1]::ops::function::FnOnce<()>>::call_once
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/core/src/ops/function.rs:250:5
[INFO] [stdout]   25:     0x5d3f63e62d5b - <fn() -> core[6883ba1bc0fe4ed1]::result::Result<(), alloc[55a36b64bcbf2c0d]::string::String> as core[6883ba1bc0fe4ed1]::ops::function::FnOnce<()>>::call_once
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/core/src/ops/function.rs:250:5
[INFO] [stdout]   26:     0x5d3f63e62d5b - test[980ffaebd391d06d]::__rust_begin_short_backtrace::<core[6883ba1bc0fe4ed1]::result::Result<(), alloc[55a36b64bcbf2c0d]::string::String>, fn() -> core[6883ba1bc0fe4ed1]::result::Result<(), alloc[55a36b64bcbf2c0d]::string::String>>
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/test/src/lib.rs:724:18
[INFO] [stdout]   27:     0x5d3f63e703a5 - test[980ffaebd391d06d]::run_test_in_process::{closure#0}
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/test/src/lib.rs:747:74
[INFO] [stdout]   28:     0x5d3f63e703a5 - <core[6883ba1bc0fe4ed1]::panic::unwind_safe::AssertUnwindSafe<test[980ffaebd391d06d]::run_test_in_process::{closure#0}> as core[6883ba1bc0fe4ed1]::ops::function::FnOnce<()>>::call_once
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/core/src/panic/unwind_safe.rs:275:9
[INFO] [stdout]   29:     0x5d3f63e703a5 - std[73adb7dc35730857]::panicking::catch_unwind::do_call::<core[6883ba1bc0fe4ed1]::panic::unwind_safe::AssertUnwindSafe<test[980ffaebd391d06d]::run_test_in_process::{closure#0}>, core[6883ba1bc0fe4ed1]::result::Result<(), alloc[55a36b64bcbf2c0d]::string::String>>
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/std/src/panicking.rs:576:43
[INFO] [stdout]   30:     0x5d3f63e703a5 - std[73adb7dc35730857]::panicking::catch_unwind::<core[6883ba1bc0fe4ed1]::result::Result<(), alloc[55a36b64bcbf2c0d]::string::String>, core[6883ba1bc0fe4ed1]::panic::unwind_safe::AssertUnwindSafe<test[980ffaebd391d06d]::run_test_in_process::{closure#0}>>
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/std/src/panicking.rs:544:19
[INFO] [stdout]   31:     0x5d3f63e703a5 - std[73adb7dc35730857]::panic::catch_unwind::<core[6883ba1bc0fe4ed1]::panic::unwind_safe::AssertUnwindSafe<test[980ffaebd391d06d]::run_test_in_process::{closure#0}>, core[6883ba1bc0fe4ed1]::result::Result<(), alloc[55a36b64bcbf2c0d]::string::String>>
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/std/src/panic.rs:359:14
[INFO] [stdout]   32:     0x5d3f63e703a5 - test[980ffaebd391d06d]::run_test_in_process
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/test/src/lib.rs:747:27
[INFO] [stdout]   33:     0x5d3f63e703a5 - test[980ffaebd391d06d]::run_test::{closure#0}
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/test/src/lib.rs:668:43
[INFO] [stdout]   34:     0x5d3f63e6ae54 - test[980ffaebd391d06d]::run_test::{closure#1}
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/test/src/lib.rs:698:41
[INFO] [stdout]   35:     0x5d3f63e6ae54 - std[73adb7dc35730857]::sys::backtrace::__rust_begin_short_backtrace::<test[980ffaebd391d06d]::run_test::{closure#1}, ()>
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/std/src/sys/backtrace.rs:166:18
[INFO] [stdout]   36:     0x5d3f63e734f2 - std[73adb7dc35730857]::thread::lifecycle::spawn_unchecked::<test[980ffaebd391d06d]::run_test::{closure#1}, ()>::{closure#1}::{closure#0}
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/std/src/thread/lifecycle.rs:70:13
[INFO] [stdout]   37:     0x5d3f63e734f2 - <core[6883ba1bc0fe4ed1]::panic::unwind_safe::AssertUnwindSafe<std[73adb7dc35730857]::thread::lifecycle::spawn_unchecked<test[980ffaebd391d06d]::run_test::{closure#1}, ()>::{closure#1}::{closure#0}> as core[6883ba1bc0fe4ed1]::ops::function::FnOnce<()>>::call_once
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/core/src/panic/unwind_safe.rs:275:9
[INFO] [stdout]   38:     0x5d3f63e734f2 - std[73adb7dc35730857]::panicking::catch_unwind::do_call::<core[6883ba1bc0fe4ed1]::panic::unwind_safe::AssertUnwindSafe<std[73adb7dc35730857]::thread::lifecycle::spawn_unchecked<test[980ffaebd391d06d]::run_test::{closure#1}, ()>::{closure#1}::{closure#0}>, ()>
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/std/src/panicking.rs:576:43
[INFO] [stdout]   39:     0x5d3f63e734f2 - std[73adb7dc35730857]::panicking::catch_unwind::<(), core[6883ba1bc0fe4ed1]::panic::unwind_safe::AssertUnwindSafe<std[73adb7dc35730857]::thread::lifecycle::spawn_unchecked<test[980ffaebd391d06d]::run_test::{closure#1}, ()>::{closure#1}::{closure#0}>>
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/std/src/panicking.rs:544:19
[INFO] [stdout]   40:     0x5d3f63e734f2 - std[73adb7dc35730857]::panic::catch_unwind::<core[6883ba1bc0fe4ed1]::panic::unwind_safe::AssertUnwindSafe<std[73adb7dc35730857]::thread::lifecycle::spawn_unchecked<test[980ffaebd391d06d]::run_test::{closure#1}, ()>::{closure#1}::{closure#0}>, ()>
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/std/src/panic.rs:359:14
[INFO] [stdout]   41:     0x5d3f63e734f2 - std[73adb7dc35730857]::thread::lifecycle::spawn_unchecked::<test[980ffaebd391d06d]::run_test::{closure#1}, ()>::{closure#1}
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/std/src/thread/lifecycle.rs:68:26
[INFO] [stdout]   42:     0x5d3f63e734f2 - <std[73adb7dc35730857]::thread::lifecycle::spawn_unchecked<test[980ffaebd391d06d]::run_test::{closure#1}, ()>::{closure#1} as core[6883ba1bc0fe4ed1]::ops::function::FnOnce<()>>::call_once::{shim:vtable#0}
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/core/src/ops/function.rs:250:5
[INFO] [stdout]   43:     0x5d3f63f2a56f - <alloc[55a36b64bcbf2c0d]::boxed::Box<dyn core[6883ba1bc0fe4ed1]::ops::function::FnOnce<(), Output = ()> + core[6883ba1bc0fe4ed1]::marker::Send> as core[6883ba1bc0fe4ed1]::ops::function::FnOnce<()>>::call_once
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/alloc/src/boxed.rs:2319:9
[INFO] [stdout]   44:     0x5d3f63f2a56f - <std[73adb7dc35730857]::sys::thread::unix::Thread>::new::thread_start
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/std/src/sys/thread/unix.rs:123:17
[INFO] [stdout]   45:     0x7fac29384aa4 - <unknown>
[INFO] [stdout]   46:     0x7fac29411a64 - clone
[INFO] [stdout]   47:                0x0 - <unknown>
[INFO] [stdout] 
[INFO] [stdout] ---- test_e2e_cold_inference_succeeds stdout ----
[INFO] [stdout] 
[INFO] [stdout] thread 'test_e2e_cold_inference_succeeds' (1937) panicked at tests/e2e.rs:26:10:
[INFO] [stdout] llama-cli not found. Install llama.cpp and add it to PATH.
[INFO] [stdout] Download: https://github.com/ggerganov/llama.cpp/releases: Cannot find 'llama-cli'. Install llama.cpp and ensure it is on PATH, or download prebuilt binaries to /root/.cache/llm-benchmark/llama-bin .
[INFO] [stdout] See: https://github.com/ggerganov/llama.cpp/releases
[INFO] [stdout] 
[INFO] [stdout] Stack backtrace:
[INFO] [stdout]    0: <anyhow::Error>::msg::<alloc::string::String>
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/anyhow-1.0.102/src/backtrace.rs:10:14
[INFO] [stdout]    1: llmb::assets::find_exe
[INFO] [stdout]              at ./src/assets.rs:59:5
[INFO] [stdout]    2: llmb::assets::find_llama_cli
[INFO] [stdout]              at ./src/assets.rs:42:5
[INFO] [stdout]    3: e2e::llama_cli
[INFO] [stdout]              at ./tests/e2e.rs:25:5
[INFO] [stdout]    4: e2e::test_e2e_cold_inference_succeeds
[INFO] [stdout]              at ./tests/e2e.rs:101:15
[INFO] [stdout]    5: e2e::test_e2e_cold_inference_succeeds::{closure#0}
[INFO] [stdout]              at ./tests/e2e.rs:100:38
[INFO] [stdout]    6: <e2e::test_e2e_cold_inference_succeeds::{closure#0} as core::ops::function::FnOnce<()>>::call_once
[INFO] [stdout]              at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/core/src/ops/function.rs:250:5
[INFO] [stdout]    7: <fn() -> core::result::Result<(), alloc::string::String> as core::ops::function::FnOnce<()>>::call_once
[INFO] [stdout]              at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/core/src/ops/function.rs:250:5
[INFO] [stdout]    8: test::__rust_begin_short_backtrace::<core::result::Result<(), alloc::string::String>, fn() -> core::result::Result<(), alloc::string::String>>
[INFO] [stdout]              at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/test/src/lib.rs:724:18
[INFO] [stdout]    9: test::run_test_in_process::{closure#0}
[INFO] [stdout]              at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/test/src/lib.rs:747:74
[INFO] [stdout]   10: <core::panic::unwind_safe::AssertUnwindSafe<test::run_test_in_process::{closure#0}> as core::ops::function::FnOnce<()>>::call_once
[INFO] [stdout]              at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/core/src/panic/unwind_safe.rs:275:9
[INFO] [stdout]   11: std::panicking::catch_unwind::do_call::<core::panic::unwind_safe::AssertUnwindSafe<test::run_test_in_process::{closure#0}>, core::result::Result<(), alloc::string::String>>
[INFO] [stdout]              at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/std/src/panicking.rs:576:43
[INFO] [stdout]   12: std::panicking::catch_unwind::<core::result::Result<(), alloc::string::String>, core::panic::unwind_safe::AssertUnwindSafe<test::run_test_in_process::{closure#0}>>
[INFO] [stdout]              at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/std/src/panicking.rs:544:19
[INFO] [stdout]   13: std::panic::catch_unwind::<core::panic::unwind_safe::AssertUnwindSafe<test::run_test_in_process::{closure#0}>, core::result::Result<(), alloc::string::String>>
[INFO] [stdout]              at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/std/src/panic.rs:359:14
[INFO] [stdout]   14: test::run_test_in_process
[INFO] [stdout]              at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/test/src/lib.rs:747:27
[INFO] [stdout]   15: test::run_test::{closure#0}
[INFO] [stdout]              at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/test/src/lib.rs:668:43
[INFO] [stdout]   16: test::run_test::{closure#1}
[INFO] [stdout]              at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/test/src/lib.rs:698:41
[INFO] [stdout]   17: std::sys::backtrace::__rust_begin_short_backtrace::<test::run_test::{closure#1}, ()>
[INFO] [stdout]              at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/std/src/sys/backtrace.rs:166:18
[INFO] [stdout]   18: std::thread::lifecycle::spawn_unchecked::<test::run_test::{closure#1}, ()>::{closure#1}::{closure#0}
[INFO] [stdout]              at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/std/src/thread/lifecycle.rs:70:13
[INFO] [stdout]   19: <core::panic::unwind_safe::AssertUnwindSafe<std::thread::lifecycle::spawn_unchecked<test::run_test::{closure#1}, ()>::{closure#1}::{closure#0}> as core::ops::function::FnOnce<()>>::call_once
[INFO] [stdout]              at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/core/src/panic/unwind_safe.rs:275:9
[INFO] [stdout]   20: std::panicking::catch_unwind::do_call::<core::panic::unwind_safe::AssertUnwindSafe<std::thread::lifecycle::spawn_unchecked<test::run_test::{closure#1}, ()>::{closure#1}::{closure#0}>, ()>
[INFO] [stdout]              at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/std/src/panicking.rs:576:43
[INFO] [stdout]   21: std::panicking::catch_unwind::<(), core::panic::unwind_safe::AssertUnwindSafe<std::thread::lifecycle::spawn_unchecked<test::run_test::{closure#1}, ()>::{closure#1}::{closure#0}>>
[INFO] [stdout]              at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/std/src/panicking.rs:544:19
[INFO] [stdout]   22: std::panic::catch_unwind::<core::panic::unwind_safe::AssertUnwindSafe<std::thread::lifecycle::spawn_unchecked<test::run_test::{closure#1}, ()>::{closure#1}::{closure#0}>, ()>
[INFO] [stdout]              at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/std/src/panic.rs:359:14
[INFO] [stdout]   23: std::thread::lifecycle::spawn_unchecked::<test::run_test::{closure#1}, ()>::{closure#1}
[INFO] [stdout]              at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/std/src/thread/lifecycle.rs:68:26
[INFO] [stdout]   24: <std::thread::lifecycle::spawn_unchecked<test::run_test::{closure#1}, ()>::{closure#1} as core::ops::function::FnOnce<()>>::call_once::{shim:vtable#0}
[INFO] [stdout]              at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/core/src/ops/function.rs:250:5
[INFO] [stdout]   25: <alloc::boxed::Box<dyn core::ops::function::FnOnce<(), Output = ()> + core::marker::Send> as core::ops::function::FnOnce<()>>::call_once
[INFO] [stdout]              at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/alloc/src/boxed.rs:2319:9
[INFO] [stdout]   26: <std::sys::thread::unix::Thread>::new::thread_start
[INFO] [stdout]              at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/std/src/sys/thread/unix.rs:123:17
[INFO] [stdout]   27: <unknown>
[INFO] [stdout]   28: clone
[INFO] [stdout] stack backtrace:
[INFO] [stdout]    0:     0x5d3f63f2ace1 - std[73adb7dc35730857]::backtrace_rs::backtrace::libunwind::trace
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/std/src/../../backtrace/src/backtrace/libunwind.rs:117:9
[INFO] [stdout]    1:     0x5d3f63f2ace1 - std[73adb7dc35730857]::backtrace_rs::backtrace::trace_unsynchronized::<std[73adb7dc35730857]::sys::backtrace::_print_fmt::{closure#1}>
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/std/src/../../backtrace/src/backtrace/mod.rs:66:14
[INFO] [stdout]    2:     0x5d3f63f2ace1 - std[73adb7dc35730857]::sys::backtrace::_print_fmt
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/std/src/sys/backtrace.rs:74:9
[INFO] [stdout]    3:     0x5d3f63f2ace1 - <<std[73adb7dc35730857]::sys::backtrace::BacktraceLock>::print::DisplayBacktrace as core[6883ba1bc0fe4ed1]::fmt::Display>::fmt
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/std/src/sys/backtrace.rs:44:26
[INFO] [stdout]    4:     0x5d3f63f3f7ba - <core[6883ba1bc0fe4ed1]::fmt::rt::Argument>::fmt
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/core/src/fmt/rt.rs:152:76
[INFO] [stdout]    5:     0x5d3f63f3f7ba - core[6883ba1bc0fe4ed1]::fmt::write
[INFO] [stdout]    6:     0x5d3f63f2f6bc - std[73adb7dc35730857]::io::default_write_fmt::<alloc[55a36b64bcbf2c0d]::vec::Vec<u8>>
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/std/src/io/mod.rs:626:11
[INFO] [stdout]    7:     0x5d3f63f2f6bc - <alloc[55a36b64bcbf2c0d]::vec::Vec<u8> as std[73adb7dc35730857]::io::Write>::write_fmt
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/std/src/io/mod.rs:1730:13
[INFO] [stdout]    8:     0x5d3f63f07ed6 - <std[73adb7dc35730857]::sys::backtrace::BacktraceLock>::print
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/std/src/sys/backtrace.rs:47:9
[INFO] [stdout]    9:     0x5d3f63f07ed6 - std[73adb7dc35730857]::panicking::default_hook::{closure#0}
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/std/src/panicking.rs:292:27
[INFO] [stdout]   10:     0x5d3f63f22e29 - std[73adb7dc35730857]::panicking::default_hook
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/std/src/panicking.rs:316:9
[INFO] [stdout]   11:     0x5d3f63e6fa20 - <alloc[55a36b64bcbf2c0d]::boxed::Box<dyn for<'a, 'b> core[6883ba1bc0fe4ed1]::ops::function::Fn<(&'a std[73adb7dc35730857]::panic::PanicHookInfo<'b>,), Output = ()> + core[6883ba1bc0fe4ed1]::marker::Send + core[6883ba1bc0fe4ed1]::marker::Sync> as core[6883ba1bc0fe4ed1]::ops::function::Fn<(&std[73adb7dc35730857]::panic::PanicHookInfo,)>>::call
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/alloc/src/boxed.rs:2333:9
[INFO] [stdout]   12:     0x5d3f63e6fa20 - test[980ffaebd391d06d]::test_main_inner::<test[980ffaebd391d06d]::test_main_static::{closure#0}>::{closure#0}
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/test/src/lib.rs:155:21
[INFO] [stdout]   13:     0x5d3f63f22fe2 - <alloc[55a36b64bcbf2c0d]::boxed::Box<dyn for<'a, 'b> core[6883ba1bc0fe4ed1]::ops::function::Fn<(&'a std[73adb7dc35730857]::panic::PanicHookInfo<'b>,), Output = ()> + core[6883ba1bc0fe4ed1]::marker::Send + core[6883ba1bc0fe4ed1]::marker::Sync> as core[6883ba1bc0fe4ed1]::ops::function::Fn<(&std[73adb7dc35730857]::panic::PanicHookInfo,)>>::call
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/alloc/src/boxed.rs:2333:9
[INFO] [stdout]   14:     0x5d3f63f22fe2 - std[73adb7dc35730857]::panicking::panic_with_hook
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/std/src/panicking.rs:823:13
[INFO] [stdout]   15:     0x5d3f63f07f82 - std[73adb7dc35730857]::panicking::panic_handler::{closure#0}
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/std/src/panicking.rs:688:13
[INFO] [stdout]   16:     0x5d3f63eff229 - std[73adb7dc35730857]::sys::backtrace::__rust_end_short_backtrace::<std[73adb7dc35730857]::panicking::panic_handler::{closure#0}, !>
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/std/src/sys/backtrace.rs:182:18
[INFO] [stdout]   17:     0x5d3f63f08c8d - __rustc[a7b7b02e776dd976]::rust_begin_unwind
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/std/src/panicking.rs:679:5
[INFO] [stdout]   18:     0x5d3f63f3ff8c - core[6883ba1bc0fe4ed1]::panicking::panic_fmt
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/core/src/panicking.rs:80:14
[INFO] [stdout]   19:     0x5d3f63f3fd02 - core[6883ba1bc0fe4ed1]::result::unwrap_failed
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/core/src/result.rs:1870:5
[INFO] [stdout]   20:     0x5d3f63e621a3 - <core[6883ba1bc0fe4ed1]::result::Result<std[73adb7dc35730857]::path::PathBuf, anyhow[3b9a07a6948cd063]::Error>>::expect
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/core/src/result.rs:1183:23
[INFO] [stdout]   21:     0x5d3f63e60b69 - e2e[f0516541adf4547c]::llama_cli
[INFO] [stdout]                                at /opt/rustwide/workdir/tests/e2e.rs:26:10
[INFO] [stdout]   22:     0x5d3f63e60b91 - e2e[f0516541adf4547c]::test_e2e_cold_inference_succeeds
[INFO] [stdout]                                at /opt/rustwide/workdir/tests/e2e.rs:101:15
[INFO] [stdout]   23:     0x5d3f63e5fec7 - e2e[f0516541adf4547c]::test_e2e_cold_inference_succeeds::{closure#0}
[INFO] [stdout]                                at /opt/rustwide/workdir/tests/e2e.rs:100:38
[INFO] [stdout]   24:     0x5d3f63e62bb6 - <e2e[f0516541adf4547c]::test_e2e_cold_inference_succeeds::{closure#0} as core[6883ba1bc0fe4ed1]::ops::function::FnOnce<()>>::call_once
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/core/src/ops/function.rs:250:5
[INFO] [stdout]   25:     0x5d3f63e62d5b - <fn() -> core[6883ba1bc0fe4ed1]::result::Result<(), alloc[55a36b64bcbf2c0d]::string::String> as core[6883ba1bc0fe4ed1]::ops::function::FnOnce<()>>::call_once
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/core/src/ops/function.rs:250:5
[INFO] [stdout]   26:     0x5d3f63e62d5b - test[980ffaebd391d06d]::__rust_begin_short_backtrace::<core[6883ba1bc0fe4ed1]::result::Result<(), alloc[55a36b64bcbf2c0d]::string::String>, fn() -> core[6883ba1bc0fe4ed1]::result::Result<(), alloc[55a36b64bcbf2c0d]::string::String>>
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/test/src/lib.rs:724:18
[INFO] [stdout]   27:     0x5d3f63e703a5 - test[980ffaebd391d06d]::run_test_in_process::{closure#0}
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/test/src/lib.rs:747:74
[INFO] [stdout]   28:     0x5d3f63e703a5 - <core[6883ba1bc0fe4ed1]::panic::unwind_safe::AssertUnwindSafe<test[980ffaebd391d06d]::run_test_in_process::{closure#0}> as core[6883ba1bc0fe4ed1]::ops::function::FnOnce<()>>::call_once
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/core/src/panic/unwind_safe.rs:275:9
[INFO] [stdout]   29:     0x5d3f63e703a5 - std[73adb7dc35730857]::panicking::catch_unwind::do_call::<core[6883ba1bc0fe4ed1]::panic::unwind_safe::AssertUnwindSafe<test[980ffaebd391d06d]::run_test_in_process::{closure#0}>, core[6883ba1bc0fe4ed1]::result::Result<(), alloc[55a36b64bcbf2c0d]::string::String>>
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/std/src/panicking.rs:576:43
[INFO] [stdout]   30:     0x5d3f63e703a5 - std[73adb7dc35730857]::panicking::catch_unwind::<core[6883ba1bc0fe4ed1]::result::Result<(), alloc[55a36b64bcbf2c0d]::string::String>, core[6883ba1bc0fe4ed1]::panic::unwind_safe::AssertUnwindSafe<test[980ffaebd391d06d]::run_test_in_process::{closure#0}>>
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/std/src/panicking.rs:544:19
[INFO] [stdout]   31:     0x5d3f63e703a5 - std[73adb7dc35730857]::panic::catch_unwind::<core[6883ba1bc0fe4ed1]::panic::unwind_safe::AssertUnwindSafe<test[980ffaebd391d06d]::run_test_in_process::{closure#0}>, core[6883ba1bc0fe4ed1]::result::Result<(), alloc[55a36b64bcbf2c0d]::string::String>>
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/std/src/panic.rs:359:14
[INFO] [stdout]   32:     0x5d3f63e703a5 - test[980ffaebd391d06d]::run_test_in_process
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/test/src/lib.rs:747:27
[INFO] [stdout]   33:     0x5d3f63e703a5 - test[980ffaebd391d06d]::run_test::{closure#0}
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/test/src/lib.rs:668:43
[INFO] [stdout]   34:     0x5d3f63e6ae54 - test[980ffaebd391d06d]::run_test::{closure#1}
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/test/src/lib.rs:698:41
[INFO] [stdout]   35:     0x5d3f63e6ae54 - std[73adb7dc35730857]::sys::backtrace::__rust_begin_short_backtrace::<test[980ffaebd391d06d]::run_test::{closure#1}, ()>
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/std/src/sys/backtrace.rs:166:18
[INFO] [stdout]   36:     0x5d3f63e734f2 - std[73adb7dc35730857]::thread::lifecycle::spawn_unchecked::<test[980ffaebd391d06d]::run_test::{closure#1}, ()>::{closure#1}::{closure#0}
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/std/src/thread/lifecycle.rs:70:13
[INFO] [stdout]   37:     0x5d3f63e734f2 - <core[6883ba1bc0fe4ed1]::panic::unwind_safe::AssertUnwindSafe<std[73adb7dc35730857]::thread::lifecycle::spawn_unchecked<test[980ffaebd391d06d]::run_test::{closure#1}, ()>::{closure#1}::{closure#0}> as core[6883ba1bc0fe4ed1]::ops::function::FnOnce<()>>::call_once
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/core/src/panic/unwind_safe.rs:275:9
[INFO] [stdout]   38:     0x5d3f63e734f2 - std[73adb7dc35730857]::panicking::catch_unwind::do_call::<core[6883ba1bc0fe4ed1]::panic::unwind_safe::AssertUnwindSafe<std[73adb7dc35730857]::thread::lifecycle::spawn_unchecked<test[980ffaebd391d06d]::run_test::{closure#1}, ()>::{closure#1}::{closure#0}>, ()>
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/std/src/panicking.rs:576:43
[INFO] [stdout]   39:     0x5d3f63e734f2 - std[73adb7dc35730857]::panicking::catch_unwind::<(), core[6883ba1bc0fe4ed1]::panic::unwind_safe::AssertUnwindSafe<std[73adb7dc35730857]::thread::lifecycle::spawn_unchecked<test[980ffaebd391d06d]::run_test::{closure#1}, ()>::{closure#1}::{closure#0}>>
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/std/src/panicking.rs:544:19
[INFO] [stdout]   40:     0x5d3f63e734f2 - std[73adb7dc35730857]::panic::catch_unwind::<core[6883ba1bc0fe4ed1]::panic::unwind_safe::AssertUnwindSafe<std[73adb7dc35730857]::thread::lifecycle::spawn_unchecked<test[980ffaebd391d06d]::run_test::{closure#1}, ()>::{closure#1}::{closure#0}>, ()>
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/std/src/panic.rs:359:14
[INFO] [stdout]   41:     0x5d3f63e734f2 - std[73adb7dc35730857]::thread::lifecycle::spawn_unchecked::<test[980ffaebd391d06d]::run_test::{closure#1}, ()>::{closure#1}
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/std/src/thread/lifecycle.rs:68:26
[INFO] [stdout]   42:     0x5d3f63e734f2 - <std[73adb7dc35730857]::thread::lifecycle::spawn_unchecked<test[980ffaebd391d06d]::run_test::{closure#1}, ()>::{closure#1} as core[6883ba1bc0fe4ed1]::ops::function::FnOnce<()>>::call_once::{shim:vtable#0}
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/core/src/ops/function.rs:250:5
[INFO] [stdout]   43:     0x5d3f63f2a56f - <alloc[55a36b64bcbf2c0d]::boxed::Box<dyn core[6883ba1bc0fe4ed1]::ops::function::FnOnce<(), Output = ()> + core[6883ba1bc0fe4ed1]::marker::Send> as core[6883ba1bc0fe4ed1]::ops::function::FnOnce<()>>::call_once
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/alloc/src/boxed.rs:2319:9
[INFO] [stdout]   44:     0x5d3f63f2a56f - <std[73adb7dc35730857]::sys::thread::unix::Thread>::new::thread_start
[INFO] [stdout]                                at /rustc/6b3fa26749ab40e159b2dd5cf577acaaf5902772/library/std/src/sys/thread/unix.rs:123:17
[INFO] [stdout]   45:     0x7fac29384aa4 - <unknown>
[INFO] [stdout]   46:     0x7fac29411a64 - clone
[INFO] [stdout]   47:                0x0 - <unknown>
[INFO] [stdout] 
[INFO] [stdout] 
[INFO] [stdout] failures:
[INFO] [stdout]     test_e2e_aggregated_metrics_are_sane
[INFO] [stdout]     test_e2e_cold_inference_succeeds
[INFO] [stdout]     test_e2e_different_seeds_both_complete
[INFO] [stdout]     test_e2e_same_seed_same_token_count
[INFO] [stdout]     test_e2e_tokens_per_sec_plausible
[INFO] [stdout] 
[INFO] [stdout] test result: FAILED. 0 passed; 5 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.21s
[INFO] [stdout] 
[INFO] running `Command { std: "docker" "inspect" "29405f2eb4212d6a3b5596b52740a9042f41c76d021c856f904ae87ccc2d4f66", kill_on_drop: false }`
[INFO] running `Command { std: "docker" "rm" "-f" "29405f2eb4212d6a3b5596b52740a9042f41c76d021c856f904ae87ccc2d4f66", kill_on_drop: false }`
[INFO] [stdout] 29405f2eb4212d6a3b5596b52740a9042f41c76d021c856f904ae87ccc2d4f66
