[INFO] cloning repository https://github.com/llm-d-incubation/hermes
[INFO] running `Command { std: "git" "-c" "credential.helper=" "-c" "credential.helper=/workspace/cargo-home/bin/git-credential-null" "clone" "--bare" "https://github.com/llm-d-incubation/hermes" "/workspace/cache/git-repos/https%3A%2F%2Fgithub.com%2Fllm-d-incubation%2Fhermes", kill_on_drop: false }`
[INFO] [stderr] Cloning into bare repository '/workspace/cache/git-repos/https%3A%2F%2Fgithub.com%2Fllm-d-incubation%2Fhermes'...
[INFO] running `Command { std: "git" "rev-parse" "HEAD", kill_on_drop: false }`
[INFO] [stdout] 9937cdee66cc904174409844eed37558be1f34ff
[INFO] testing llm-d-incubation/hermes against try#9f93af291970322f4f1c6315ccde4d7078201159 for pr-146098-6
[INFO] running `Command { std: "git" "clone" "/workspace/cache/git-repos/https%3A%2F%2Fgithub.com%2Fllm-d-incubation%2Fhermes" "/workspace/builds/worker-4-tc2/source", kill_on_drop: false }`
[INFO] [stderr] Cloning into '/workspace/builds/worker-4-tc2/source'...
[INFO] [stderr] done.
[INFO] started tweaking git repo https://github.com/llm-d-incubation/hermes
[INFO] finished tweaking git repo https://github.com/llm-d-incubation/hermes
[INFO] tweaked toml for git repo https://github.com/llm-d-incubation/hermes written to /workspace/builds/worker-4-tc2/source/Cargo.toml
[INFO] validating manifest of git repo https://github.com/llm-d-incubation/hermes on toolchain 9f93af291970322f4f1c6315ccde4d7078201159
[INFO] running `Command { std: CARGO_HOME="/workspace/cargo-home" RUSTUP_HOME="/workspace/rustup-home" "/workspace/cargo-home/bin/cargo" "+9f93af291970322f4f1c6315ccde4d7078201159" "metadata" "--manifest-path" "Cargo.toml" "--no-deps", kill_on_drop: false }`
[INFO] crate git repo https://github.com/llm-d-incubation/hermes already has a lockfile, it will not be regenerated
[INFO] running `Command { std: CARGO_HOME="/workspace/cargo-home" RUSTUP_HOME="/workspace/rustup-home" "/workspace/cargo-home/bin/cargo" "+9f93af291970322f4f1c6315ccde4d7078201159" "fetch" "--manifest-path" "Cargo.toml", kill_on_drop: false }`
[INFO] running `Command { std: "docker" "create" "-v" "/var/lib/crater-agent-workspace/builds/worker-4-tc2/target:/opt/rustwide/target:rw,Z" "-v" "/var/lib/crater-agent-workspace/builds/worker-4-tc2/source:/opt/rustwide/workdir:ro,Z" "-v" "/var/lib/crater-agent-workspace/cargo-home:/opt/rustwide/cargo-home:ro,Z" "-v" "/var/lib/crater-agent-workspace/rustup-home:/opt/rustwide/rustup-home:ro,Z" "-e" "SOURCE_DIR=/opt/rustwide/workdir" "-e" "CARGO_TARGET_DIR=/opt/rustwide/target" "-e" "CARGO_HOME=/opt/rustwide/cargo-home" "-e" "RUSTUP_HOME=/opt/rustwide/rustup-home" "-w" "/opt/rustwide/workdir" "-m" "1610612736" "--user" "0:0" "--network" "none" "ghcr.io/rust-lang/crates-build-env/linux@sha256:4848fb76d95f26979359cc7e45710b1dbc8f3acb7aeedee7c460d7702230f228" "/opt/rustwide/cargo-home/bin/cargo" "+9f93af291970322f4f1c6315ccde4d7078201159" "metadata" "--no-deps" "--format-version=1", kill_on_drop: false }`
[INFO] [stdout] 2f13f558c78767f1b543988146432f35f6aa6b6018d8f8a47d9a5635a4db2990
[INFO] running `Command { std: "docker" "start" "-a" "2f13f558c78767f1b543988146432f35f6aa6b6018d8f8a47d9a5635a4db2990", kill_on_drop: false }`
[INFO] running `Command { std: "docker" "inspect" "2f13f558c78767f1b543988146432f35f6aa6b6018d8f8a47d9a5635a4db2990", kill_on_drop: false }`
[INFO] running `Command { std: "docker" "rm" "-f" "2f13f558c78767f1b543988146432f35f6aa6b6018d8f8a47d9a5635a4db2990", kill_on_drop: false }`
[INFO] [stdout] 2f13f558c78767f1b543988146432f35f6aa6b6018d8f8a47d9a5635a4db2990
[INFO] running `Command { std: "docker" "create" "-v" "/var/lib/crater-agent-workspace/builds/worker-4-tc2/target:/opt/rustwide/target:rw,Z" "-v" "/var/lib/crater-agent-workspace/builds/worker-4-tc2/source:/opt/rustwide/workdir:ro,Z" "-v" "/var/lib/crater-agent-workspace/cargo-home:/opt/rustwide/cargo-home:ro,Z" "-v" "/var/lib/crater-agent-workspace/rustup-home:/opt/rustwide/rustup-home:ro,Z" "-e" "SOURCE_DIR=/opt/rustwide/workdir" "-e" "CARGO_TARGET_DIR=/opt/rustwide/target" "-e" "CARGO_INCREMENTAL=0" "-e" "RUST_BACKTRACE=full" "-e" "RUSTFLAGS=--cap-lints=forbid" "-e" "RUSTDOCFLAGS=--cap-lints=forbid" "-e" "CARGO_HOME=/opt/rustwide/cargo-home" "-e" "RUSTUP_HOME=/opt/rustwide/rustup-home" "-w" "/opt/rustwide/workdir" "-m" "1610612736" "--user" "0:0" "--network" "none" "ghcr.io/rust-lang/crates-build-env/linux@sha256:4848fb76d95f26979359cc7e45710b1dbc8f3acb7aeedee7c460d7702230f228" "/opt/rustwide/cargo-home/bin/cargo" "+9f93af291970322f4f1c6315ccde4d7078201159" "build" "--frozen" "--message-format=json", kill_on_drop: false }`
[INFO] [stdout] bfadac71482cb20da03265efff14bbbaf168b1bd12f7fa4dbf40a7c914485713
[INFO] running `Command { std: "docker" "start" "-a" "bfadac71482cb20da03265efff14bbbaf168b1bd12f7fa4dbf40a7c914485713", kill_on_drop: false }`
[INFO] [stderr]    Compiling proc-macro2 v1.0.101
[INFO] [stderr]    Compiling unicode-ident v1.0.19
[INFO] [stderr]    Compiling libc v0.2.176
[INFO] [stderr]    Compiling cfg-if v1.0.3
[INFO] [stderr]    Compiling autocfg v1.5.0
[INFO] [stderr]    Compiling serde_core v1.0.228
[INFO] [stderr]    Compiling parking_lot_core v0.9.11
[INFO] [stderr]    Compiling serde v1.0.228
[INFO] [stderr]    Compiling find-msvc-tools v0.1.2
[INFO] [stderr]    Compiling futures-channel v0.3.31
[INFO] [stderr]    Compiling futures-io v0.3.31
[INFO] [stderr]    Compiling zeroize v1.8.1
[INFO] [stderr]    Compiling serde_json v1.0.145
[INFO] [stderr]    Compiling ucd-trie v0.1.7
[INFO] [stderr]    Compiling version_check v0.9.5
[INFO] [stderr]    Compiling untrusted v0.9.0
[INFO] [stderr]    Compiling typenum v1.18.0
[INFO] [stderr]    Compiling rustls v0.23.32
[INFO] [stderr]    Compiling cc v1.2.39
[INFO] [stderr]    Compiling rustls-pki-types v1.12.0
[INFO] [stderr]    Compiling k8s-openapi v0.26.0
[INFO] [stderr]    Compiling ref-cast v1.0.25
[INFO] [stderr]    Compiling regex-syntax v0.8.6
[INFO] [stderr]    Compiling ident_case v1.0.1
[INFO] [stderr]    Compiling iana-time-zone v0.1.64
[INFO] [stderr]    Compiling subtle v2.6.1
[INFO] [stderr]    Compiling generic-array v0.14.7
[INFO] [stderr]    Compiling indexmap v2.11.4
[INFO] [stderr]    Compiling dyn-clone v1.0.20
[INFO] [stderr]    Compiling rustls-native-certs v0.8.1
[INFO] [stderr]    Compiling bitflags v2.9.4
[INFO] [stderr]    Compiling lock_api v0.4.13
[INFO] [stderr]    Compiling num-traits v0.2.19
[INFO] [stderr]    Compiling getrandom v0.3.3
[INFO] [stderr]    Compiling unsafe-libyaml v0.2.11
[INFO] [stderr]    Compiling anstyle v1.0.12
[INFO] [stderr]    Compiling is_terminal_polyfill v1.70.1
[INFO] [stderr]    Compiling sync_wrapper v1.0.2
[INFO] [stderr]    Compiling secrecy v0.10.3
[INFO] [stderr]    Compiling pem v3.0.5
[INFO] [stderr]    Compiling csv-core v0.1.12
[INFO] [stderr]    Compiling clap_lex v0.7.5
[INFO] [stderr]    Compiling option-ext v0.2.0
[INFO] [stderr]    Compiling anstream v0.6.20
[INFO] [stderr]    Compiling quote v1.0.41
[INFO] [stderr]    Compiling anyhow v1.0.100
[INFO] [stderr]    Compiling home v0.5.11
[INFO] [stderr]    Compiling either v1.15.0
[INFO] [stderr]    Compiling syn v2.0.106
[INFO] [stderr]    Compiling mio v1.0.4
[INFO] [stderr]    Compiling socket2 v0.6.0
[INFO] [stderr]    Compiling signal-hook-registry v1.4.6
[INFO] [stderr]    Compiling getrandom v0.2.16
[INFO] [stderr]    Compiling ordered-float v2.10.1
[INFO] [stderr]    Compiling dirs-sys-next v0.1.2
[INFO] [stderr]    Compiling block-buffer v0.10.4
[INFO] [stderr]    Compiling parking_lot v0.12.4
[INFO] [stderr]    Compiling crypto-common v0.1.6
[INFO] [stderr]    Compiling dirs-next v2.0.0
[INFO] [stderr]    Compiling owo-colors v4.2.3
[INFO] [stderr]    Compiling clap_builder v4.5.48
[INFO] [stderr]    Compiling term v0.7.0
[INFO] [stderr]    Compiling digest v0.10.7
[INFO] [stderr]    Compiling is-terminal v0.4.16
[INFO] [stderr]    Compiling dirs-sys v0.4.1
[INFO] [stderr]    Compiling thread_local v1.1.9
[INFO] [stderr]    Compiling hermes v0.2.5 (/opt/rustwide/workdir)
[INFO] [stderr]    Compiling cpufeatures v0.2.17
[INFO] [stderr]    Compiling nu-ansi-term v0.50.1
[INFO] [stderr]    Compiling unicode-width v0.1.14
[INFO] [stderr]    Compiling encode_unicode v1.0.0
[INFO] [stderr]    Compiling tracing-subscriber v0.3.20
[INFO] [stderr]    Compiling sha2 v0.10.9
[INFO] [stderr]    Compiling dirs v5.0.1
[INFO] [stderr]    Compiling uuid v1.18.1
[INFO] [stderr]    Compiling ring v0.17.14
[INFO] [stderr]    Compiling regex-automata v0.4.11
[INFO] [stderr]    Compiling serde_derive_internals v0.29.1
[INFO] [stderr]    Compiling darling_core v0.21.3
[INFO] [stderr]    Compiling serde_derive v1.0.228
[INFO] [stderr]    Compiling tokio-macros v2.5.0
[INFO] [stderr]    Compiling thiserror-impl v2.0.17
[INFO] [stderr]    Compiling futures-macro v0.3.31
[INFO] [stderr]    Compiling tracing-attributes v0.1.30
[INFO] [stderr]    Compiling ref-cast-impl v1.0.25
[INFO] [stderr]    Compiling derive_more-impl v2.0.1
[INFO] [stderr]    Compiling schemars_derive v1.0.4
[INFO] [stderr]    Compiling clap_derive v4.5.47
[INFO] [stderr]    Compiling regex v1.11.3
[INFO] [stderr]    Compiling tokio v1.47.1
[INFO] [stderr]    Compiling futures-util v0.3.31
[INFO] [stderr]    Compiling derive_more v2.0.1
[INFO] [stderr]    Compiling thiserror v2.0.17
[INFO] [stderr]    Compiling tracing v0.1.41
[INFO] [stderr]    Compiling pest v2.8.2
[INFO] [stderr]    Compiling tower-http v0.6.6
[INFO] [stderr]    Compiling rustls-webpki v0.103.6
[INFO] [stderr]    Compiling clap v4.5.48
[INFO] [stderr]    Compiling darling_macro v0.21.3
[INFO] [stderr]    Compiling pest_meta v2.8.2
[INFO] [stderr]    Compiling darling v0.21.3
[INFO] [stderr]    Compiling futures-executor v0.3.31
[INFO] [stderr]    Compiling futures v0.3.31
[INFO] [stderr]    Compiling pest_generator v2.8.2
[INFO] [stderr]    Compiling kube-derive v2.0.1
[INFO] [stderr]    Compiling chrono v0.4.42
[INFO] [stderr]    Compiling schemars v1.0.4
[INFO] [stderr]    Compiling serde-value v0.7.0
[INFO] [stderr]    Compiling serde_yaml v0.9.34+deprecated
[INFO] [stderr]    Compiling csv v1.3.1
[INFO] [stderr]    Compiling minijinja v2.12.0
[INFO] [stderr]    Compiling pest_derive v2.8.2
[INFO] [stderr]    Compiling prettytable-rs v0.10.0
[INFO] [stderr]    Compiling jsonpath-rust v0.7.5
[INFO] [stderr]    Compiling hyper v1.7.0
[INFO] [stderr]    Compiling tokio-util v0.7.16
[INFO] [stderr]    Compiling tokio-stream v0.1.17
[INFO] [stderr]    Compiling tower v0.5.2
[INFO] [stderr]    Compiling hyper-util v0.1.17
[INFO] [stderr]    Compiling tokio-rustls v0.26.4
[INFO] [stderr]    Compiling hyper-rustls v0.27.7
[INFO] [stderr]    Compiling hyper-timeout v0.5.2
[INFO] [stderr]    Compiling kube-core v2.0.1
[INFO] [stderr]    Compiling kube-client v2.0.1
[INFO] [stderr]    Compiling kube v2.0.1
[INFO] [stderr]     Finished `dev` profile [unoptimized + debuginfo] target(s) in 1m 26s
[INFO] running `Command { std: "docker" "inspect" "bfadac71482cb20da03265efff14bbbaf168b1bd12f7fa4dbf40a7c914485713", kill_on_drop: false }`
[INFO] running `Command { std: "docker" "rm" "-f" "bfadac71482cb20da03265efff14bbbaf168b1bd12f7fa4dbf40a7c914485713", kill_on_drop: false }`
[INFO] [stdout] bfadac71482cb20da03265efff14bbbaf168b1bd12f7fa4dbf40a7c914485713
[INFO] running `Command { std: "docker" "create" "-v" "/var/lib/crater-agent-workspace/builds/worker-4-tc2/target:/opt/rustwide/target:rw,Z" "-v" "/var/lib/crater-agent-workspace/builds/worker-4-tc2/source:/opt/rustwide/workdir:ro,Z" "-v" "/var/lib/crater-agent-workspace/cargo-home:/opt/rustwide/cargo-home:ro,Z" "-v" "/var/lib/crater-agent-workspace/rustup-home:/opt/rustwide/rustup-home:ro,Z" "-e" "SOURCE_DIR=/opt/rustwide/workdir" "-e" "CARGO_TARGET_DIR=/opt/rustwide/target" "-e" "CARGO_INCREMENTAL=0" "-e" "RUST_BACKTRACE=full" "-e" "RUSTFLAGS=--cap-lints=forbid" "-e" "RUSTDOCFLAGS=--cap-lints=forbid" "-e" "CARGO_HOME=/opt/rustwide/cargo-home" "-e" "RUSTUP_HOME=/opt/rustwide/rustup-home" "-w" "/opt/rustwide/workdir" "-m" "1610612736" "--user" "0:0" "--network" "none" "ghcr.io/rust-lang/crates-build-env/linux@sha256:4848fb76d95f26979359cc7e45710b1dbc8f3acb7aeedee7c460d7702230f228" "/opt/rustwide/cargo-home/bin/cargo" "+9f93af291970322f4f1c6315ccde4d7078201159" "test" "--frozen" "--no-run" "--message-format=json", kill_on_drop: false }`
[INFO] [stdout] 2fc2b1f584635789e5568b9ff51c3080d1077fa2176bdcf81b7e49634c4db520
[INFO] running `Command { std: "docker" "start" "-a" "2fc2b1f584635789e5568b9ff51c3080d1077fa2176bdcf81b7e49634c4db520", kill_on_drop: false }`
[INFO] [stderr]    Compiling console v0.15.11
[INFO] [stderr]    Compiling similar v2.7.0
[INFO] [stderr]    Compiling insta v1.43.2
[INFO] [stderr]    Compiling hermes v0.2.5 (/opt/rustwide/workdir)
[INFO] [stderr]     Finished `test` profile [unoptimized + debuginfo] target(s) in 9.67s
[INFO] running `Command { std: "docker" "inspect" "2fc2b1f584635789e5568b9ff51c3080d1077fa2176bdcf81b7e49634c4db520", kill_on_drop: false }`
[INFO] running `Command { std: "docker" "rm" "-f" "2fc2b1f584635789e5568b9ff51c3080d1077fa2176bdcf81b7e49634c4db520", kill_on_drop: false }`
[INFO] [stdout] 2fc2b1f584635789e5568b9ff51c3080d1077fa2176bdcf81b7e49634c4db520
[INFO] running `Command { std: "docker" "create" "-v" "/var/lib/crater-agent-workspace/builds/worker-4-tc2/target:/opt/rustwide/target:rw,Z" "-v" "/var/lib/crater-agent-workspace/builds/worker-4-tc2/source:/opt/rustwide/workdir:ro,Z" "-v" "/var/lib/crater-agent-workspace/cargo-home:/opt/rustwide/cargo-home:ro,Z" "-v" "/var/lib/crater-agent-workspace/rustup-home:/opt/rustwide/rustup-home:ro,Z" "-e" "SOURCE_DIR=/opt/rustwide/workdir" "-e" "CARGO_TARGET_DIR=/opt/rustwide/target" "-e" "CARGO_INCREMENTAL=0" "-e" "RUST_BACKTRACE=full" "-e" "RUSTFLAGS=--cap-lints=forbid" "-e" "RUSTDOCFLAGS=--cap-lints=forbid" "-e" "CARGO_HOME=/opt/rustwide/cargo-home" "-e" "RUSTUP_HOME=/opt/rustwide/rustup-home" "-w" "/opt/rustwide/workdir" "-m" "1610612736" "--user" "0:0" "--network" "none" "ghcr.io/rust-lang/crates-build-env/linux@sha256:4848fb76d95f26979359cc7e45710b1dbc8f3acb7aeedee7c460d7702230f228" "/opt/rustwide/cargo-home/bin/cargo" "+9f93af291970322f4f1c6315ccde4d7078201159" "test" "--frozen", kill_on_drop: false }`
[INFO] [stdout] 58dc7266a790258cccfae05e9c9a374b04a09c6a22faa146248a77c65949c2bc
[INFO] running `Command { std: "docker" "start" "-a" "58dc7266a790258cccfae05e9c9a374b04a09c6a22faa146248a77c65949c2bc", kill_on_drop: false }`
[INFO] [stderr]     Finished `test` profile [unoptimized + debuginfo] target(s) in 0.25s
[INFO] [stderr]      Running unittests src/lib.rs (/opt/rustwide/target/debug/deps/hermes-3ed0216f65cac55f)
[INFO] [stdout] 
[INFO] [stdout] running 5 tests
[INFO] [stdout] test cache::tests::test_humanize_duration ... ok
[INFO] [stdout] test topology_selector::tests::test_coreweave_selector_same_leafgroup ... FAILED
[INFO] [stdout] test topology_selector::tests::test_gke_selector_different_topologies ... FAILED
[INFO] [stdout] test analyzer::tests::test_determine_topology_strategy_gke_with_fabric ... FAILED
[INFO] [stdout] test analyzer::tests::test_analyze_node_gke_with_rdma ... FAILED
[INFO] [stdout] 
[INFO] [stdout] failures:
[INFO] [stdout] 
[INFO] [stdout] ---- topology_selector::tests::test_coreweave_selector_same_leafgroup stdout ----
[INFO] [stdout] ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ Snapshot Summary ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
[INFO] [stdout] Snapshot file: src/snapshots/hermes__topology_selector__tests__coreweave_selector_same_leafgroup.snap
[INFO] [stdout] Snapshot: coreweave_selector_same_leafgroup
[INFO] [stdout] Source: src/topology_selector.rs:283
[INFO] [stdout] ────────────────────────────────────────────────────────────────────────────────
[INFO] [stdout] Expression: reason
[INFO] [stdout] ────────────────────────────────────────────────────────────────────────────────
[INFO] [stdout] +new results
[INFO] [stdout] ────────────┬───────────────────────────────────────────────────────────────────
[INFO] [stdout]           1 │+Optimal CoreWeave same-topology: RoCE RDMA within leafgroup 'lg1'
[INFO] [stdout] ────────────┴───────────────────────────────────────────────────────────────────
[INFO] [stdout] 
[INFO] [stdout] thread 'topology_selector::tests::test_coreweave_selector_same_leafgroup' (28) panicked at src/topology_selector.rs:283:9:
[INFO] [stdout] called `Result::unwrap()` on an `Err` value: Os { code: 30, kind: ReadOnlyFilesystem, message: "Read-only file system" }
[INFO] [stdout] stack backtrace:
[INFO] [stdout]    0:     0x58370179e432 - std::backtrace_rs::backtrace::libunwind::trace::h16affffe904e891e
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/std/src/../../backtrace/src/backtrace/libunwind.rs:117:9
[INFO] [stdout]    1:     0x58370179e432 - std::backtrace_rs::backtrace::trace_unsynchronized::h5c14b13373ed4150
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/std/src/../../backtrace/src/backtrace/mod.rs:66:14
[INFO] [stdout]    2:     0x58370179e432 - std::sys::backtrace::_print_fmt::hcbb507f162c816cc
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/std/src/sys/backtrace.rs:66:9
[INFO] [stdout]    3:     0x58370179e432 - <std::sys::backtrace::BacktraceLock::print::DisplayBacktrace as core::fmt::Display>::fmt::h8be9aa933f14675f
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/std/src/sys/backtrace.rs:39:26
[INFO] [stdout]    4:     0x5837017b0c1f - core::fmt::rt::Argument::fmt::h30ed739d33467c3a
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/core/src/fmt/rt.rs:173:76
[INFO] [stdout]    5:     0x5837017b0c1f - core::fmt::write::hfd0efbb002ac7eea
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/core/src/fmt/mod.rs:1469:25
[INFO] [stdout]    6:     0x583701769203 - std::io::default_write_fmt::hd6d24501f2d7f8d3
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/std/src/io/mod.rs:639:11
[INFO] [stdout]    7:     0x583701769203 - std::io::Write::write_fmt::h79eca2f72fc24111
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/std/src/io/mod.rs:1954:13
[INFO] [stdout]    8:     0x583701775902 - std::sys::backtrace::BacktraceLock::print::hf2554f6030d393f7
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/std/src/sys/backtrace.rs:42:9
[INFO] [stdout]    9:     0x58370177b82f - std::panicking::default_hook::{{closure}}::h8873121c56335b01
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/std/src/panicking.rs:301:27
[INFO] [stdout]   10:     0x58370177b6c1 - std::panicking::default_hook::hbafefc2d196267a2
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/std/src/panicking.rs:325:9
[INFO] [stdout]   11:     0x58370153babe - <alloc::boxed::Box<F,A> as core::ops::function::Fn<Args>>::call::ha834d5846f91b30b
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/alloc/src/boxed.rs:2099:9
[INFO] [stdout]   12:     0x58370153babe - test::test_main_with_exit_callback::{{closure}}::h63c167737eecb025
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/test/src/lib.rs:145:21
[INFO] [stdout]   13:     0x58370177be3f - <alloc::boxed::Box<F,A> as core::ops::function::Fn<Args>>::call::hbf9b0f7a281291fd
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/alloc/src/boxed.rs:2099:9
[INFO] [stdout]   14:     0x58370177be3f - std::panicking::panic_with_hook::h9f5b09d5adc1a745
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/std/src/panicking.rs:842:13
[INFO] [stdout]   15:     0x58370177bc9a - std::panicking::panic_handler::{{closure}}::h08111e483bdf6a89
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/std/src/panicking.rs:707:13
[INFO] [stdout]   16:     0x583701775a39 - std::sys::backtrace::__rust_end_short_backtrace::h1b86e3414ecbbe8d
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/std/src/sys/backtrace.rs:174:18
[INFO] [stdout]   17:     0x58370175c79d - __rustc[b292c645e8102103]::rust_begin_unwind
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/std/src/panicking.rs:698:5
[INFO] [stdout]   18:     0x5837017ba580 - core::panicking::panic_fmt::h31cc490ecc8cc1fa
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/core/src/panicking.rs:80:14
[INFO] [stdout]   19:     0x5837017b97d6 - core::result::unwrap_failed::he05762ef9e6bdf9c
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/core/src/result.rs:1862:5
[INFO] [stdout]   20:     0x583701519fdb - core::result::Result<T,E>::unwrap::h2298545f13ad2571
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/core/src/result.rs:1233:23
[INFO] [stdout]   21:     0x583701519fdb - hermes::topology_selector::tests::test_coreweave_selector_same_leafgroup::hd47dc9773fcd25fd
[INFO] [stdout]                                at /opt/rustwide/workdir/src/topology_selector.rs:283:9
[INFO] [stdout]   22:     0x58370151a117 - hermes::topology_selector::tests::test_coreweave_selector_same_leafgroup::{{closure}}::h3e24afebddfd4e30
[INFO] [stdout]                                at /opt/rustwide/workdir/src/topology_selector.rs:266:48
[INFO] [stdout]   23:     0x583701515836 - core::ops::function::FnOnce::call_once::hf53ed088d3cc8326
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/core/src/ops/function.rs:250:5
[INFO] [stdout]   24:     0x58370153b8bb - core::ops::function::FnOnce::call_once::h2b2de5fdd23aab3e
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/core/src/ops/function.rs:250:5
[INFO] [stdout]   25:     0x58370153b8bb - test::__rust_begin_short_backtrace::he551dd004770be01
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/test/src/lib.rs:663:18
[INFO] [stdout]   26:     0x58370154f91d - test::run_test_in_process::{{closure}}::h5f0b44080a35ed87
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/test/src/lib.rs:686:74
[INFO] [stdout]   27:     0x58370154f91d - <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once::h4dbf65d14893ecf5
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/core/src/panic/unwind_safe.rs:274:9
[INFO] [stdout]   28:     0x58370154f91d - std::panicking::catch_unwind::do_call::hc37c563b8a006285
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/std/src/panicking.rs:590:40
[INFO] [stdout]   29:     0x58370154f91d - std::panicking::catch_unwind::h616b6e2e7a27f612
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/std/src/panicking.rs:553:19
[INFO] [stdout]   30:     0x58370154f91d - std::panic::catch_unwind::h1e788dd57758e6d8
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/std/src/panic.rs:359:14
[INFO] [stdout]   31:     0x58370154f91d - test::run_test_in_process::hf073c2764f29f8ad
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/test/src/lib.rs:686:27
[INFO] [stdout]   32:     0x58370154f91d - test::run_test::{{closure}}::h3dc46b7a0c340fa6
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/test/src/lib.rs:607:43
[INFO] [stdout]   33:     0x583701528e64 - test::run_test::{{closure}}::h6a4da3c57ef4505f
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/test/src/lib.rs:637:41
[INFO] [stdout]   34:     0x583701528e64 - std::sys::backtrace::__rust_begin_short_backtrace::hfd8e44bc311a5d57
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/std/src/sys/backtrace.rs:158:18
[INFO] [stdout]   35:     0x58370152c6ba - std::thread::Builder::spawn_unchecked_::{{closure}}::{{closure}}::h57c4ddec344fe24c
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/std/src/thread/mod.rs:562:17
[INFO] [stdout]   36:     0x58370152c6ba - <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once::hd7cbe09591f06dfb
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/core/src/panic/unwind_safe.rs:274:9
[INFO] [stdout]   37:     0x58370152c6ba - std::panicking::catch_unwind::do_call::hdcd076e8e993dfbc
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/std/src/panicking.rs:590:40
[INFO] [stdout]   38:     0x58370152c6ba - std::panicking::catch_unwind::h8f9f675f3756eab1
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/std/src/panicking.rs:553:19
[INFO] [stdout]   39:     0x58370152c6ba - std::panic::catch_unwind::he8f74a93abeceb9b
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/std/src/panic.rs:359:14
[INFO] [stdout]   40:     0x58370152c6ba - std::thread::Builder::spawn_unchecked_::{{closure}}::he43db13a2caa41d5
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/std/src/thread/mod.rs:560:30
[INFO] [stdout]   41:     0x58370152c6ba - core::ops::function::FnOnce::call_once{{vtable.shim}}::h8f3531a7e0d83514
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/core/src/ops/function.rs:250:5
[INFO] [stdout]   42:     0x583701770c4f - <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once::h6ff05134d80ef20e
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/alloc/src/boxed.rs:2085:9
[INFO] [stdout]   43:     0x583701770c4f - std::sys::thread::unix::Thread::new::thread_start::h9a4a41a076a486e0
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/std/src/sys/thread/unix.rs:124:17
[INFO] [stdout]   44:     0x7ce4ce7fbaa4 - <unknown>
[INFO] [stdout]   45:     0x7ce4ce888a64 - clone
[INFO] [stdout]   46:                0x0 - <unknown>
[INFO] [stdout] 
[INFO] [stdout] ---- topology_selector::tests::test_gke_selector_different_topologies stdout ----
[INFO] [stdout] ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ Snapshot Summary ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
[INFO] [stdout] Snapshot file: src/snapshots/hermes__topology_selector__tests__gke_selector_different_topologies.snap
[INFO] [stdout] Snapshot: gke_selector_different_topologies
[INFO] [stdout] Source: src/topology_selector.rs:301
[INFO] [stdout] ────────────────────────────────────────────────────────────────────────────────
[INFO] [stdout] Expression: reason
[INFO] [stdout] ────────────────────────────────────────────────────────────────────────────────
[INFO] [stdout] +new results
[INFO] [stdout] ────────────┬───────────────────────────────────────────────────────────────────
[INFO] [stdout]           1 │+GKE fallback: GKE RDMA RDMA nodes (topology may differ)
[INFO] [stdout] ────────────┴───────────────────────────────────────────────────────────────────
[INFO] [stdout] 
[INFO] [stdout] thread 'topology_selector::tests::test_gke_selector_different_topologies' (29) panicked at src/topology_selector.rs:301:9:
[INFO] [stdout] called `Result::unwrap()` on an `Err` value: Os { code: 30, kind: ReadOnlyFilesystem, message: "Read-only file system" }
[INFO] [stdout] stack backtrace:
[INFO] [stdout]    0:     0x58370179e432 - std::backtrace_rs::backtrace::libunwind::trace::h16affffe904e891e
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/std/src/../../backtrace/src/backtrace/libunwind.rs:117:9
[INFO] [stdout]    1:     0x58370179e432 - std::backtrace_rs::backtrace::trace_unsynchronized::h5c14b13373ed4150
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/std/src/../../backtrace/src/backtrace/mod.rs:66:14
[INFO] [stdout]    2:     0x58370179e432 - std::sys::backtrace::_print_fmt::hcbb507f162c816cc
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/std/src/sys/backtrace.rs:66:9
[INFO] [stdout]    3:     0x58370179e432 - <std::sys::backtrace::BacktraceLock::print::DisplayBacktrace as core::fmt::Display>::fmt::h8be9aa933f14675f
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/std/src/sys/backtrace.rs:39:26
[INFO] [stdout]    4:     0x5837017b0c1f - core::fmt::rt::Argument::fmt::h30ed739d33467c3a
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/core/src/fmt/rt.rs:173:76
[INFO] [stdout]    5:     0x5837017b0c1f - core::fmt::write::hfd0efbb002ac7eea
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/core/src/fmt/mod.rs:1469:25
[INFO] [stdout]    6:     0x583701769203 - std::io::default_write_fmt::hd6d24501f2d7f8d3
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/std/src/io/mod.rs:639:11
[INFO] [stdout]    7:     0x583701769203 - std::io::Write::write_fmt::h79eca2f72fc24111
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/std/src/io/mod.rs:1954:13
[INFO] [stdout]    8:     0x583701775902 - std::sys::backtrace::BacktraceLock::print::hf2554f6030d393f7
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/std/src/sys/backtrace.rs:42:9
[INFO] [stdout]    9:     0x58370177b82f - std::panicking::default_hook::{{closure}}::h8873121c56335b01
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/std/src/panicking.rs:301:27
[INFO] [stdout]   10:     0x58370177b6c1 - std::panicking::default_hook::hbafefc2d196267a2
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/std/src/panicking.rs:325:9
[INFO] [stdout]   11:     0x58370153babe - <alloc::boxed::Box<F,A> as core::ops::function::Fn<Args>>::call::ha834d5846f91b30b
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/alloc/src/boxed.rs:2099:9
[INFO] [stdout]   12:     0x58370153babe - test::test_main_with_exit_callback::{{closure}}::h63c167737eecb025
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/test/src/lib.rs:145:21
[INFO] [stdout]   13:     0x58370177be3f - <alloc::boxed::Box<F,A> as core::ops::function::Fn<Args>>::call::hbf9b0f7a281291fd
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/alloc/src/boxed.rs:2099:9
[INFO] [stdout]   14:     0x58370177be3f - std::panicking::panic_with_hook::h9f5b09d5adc1a745
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/std/src/panicking.rs:842:13
[INFO] [stdout]   15:     0x58370177bc9a - std::panicking::panic_handler::{{closure}}::h08111e483bdf6a89
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/std/src/panicking.rs:707:13
[INFO] [stdout]   16:     0x583701775a39 - std::sys::backtrace::__rust_end_short_backtrace::h1b86e3414ecbbe8d
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/std/src/sys/backtrace.rs:174:18
[INFO] [stdout]   17:     0x58370175c79d - __rustc[b292c645e8102103]::rust_begin_unwind
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/std/src/panicking.rs:698:5
[INFO] [stdout]   18:     0x5837017ba580 - core::panicking::panic_fmt::h31cc490ecc8cc1fa
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/core/src/panicking.rs:80:14
[INFO] [stdout]   19:     0x5837017b97d6 - core::result::unwrap_failed::he05762ef9e6bdf9c
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/core/src/result.rs:1862:5
[INFO] [stdout]   20:     0x58370151aac3 - core::result::Result<T,E>::unwrap::h2298545f13ad2571
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/core/src/result.rs:1233:23
[INFO] [stdout]   21:     0x58370151aac3 - hermes::topology_selector::tests::test_gke_selector_different_topologies::hea751036da8dfae5
[INFO] [stdout]                                at /opt/rustwide/workdir/src/topology_selector.rs:301:9
[INFO] [stdout]   22:     0x58370151acb7 - hermes::topology_selector::tests::test_gke_selector_different_topologies::{{closure}}::hb32e6833b1f416a8
[INFO] [stdout]                                at /opt/rustwide/workdir/src/topology_selector.rs:287:48
[INFO] [stdout]   23:     0x583701515636 - core::ops::function::FnOnce::call_once::h54bcc003afe68c20
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/core/src/ops/function.rs:250:5
[INFO] [stdout]   24:     0x58370153b8bb - core::ops::function::FnOnce::call_once::h2b2de5fdd23aab3e
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/core/src/ops/function.rs:250:5
[INFO] [stdout]   25:     0x58370153b8bb - test::__rust_begin_short_backtrace::he551dd004770be01
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/test/src/lib.rs:663:18
[INFO] [stdout]   26:     0x58370154f91d - test::run_test_in_process::{{closure}}::h5f0b44080a35ed87
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/test/src/lib.rs:686:74
[INFO] [stdout]   27:     0x58370154f91d - <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once::h4dbf65d14893ecf5
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/core/src/panic/unwind_safe.rs:274:9
[INFO] [stdout]   28:     0x58370154f91d - std::panicking::catch_unwind::do_call::hc37c563b8a006285
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/std/src/panicking.rs:590:40
[INFO] [stdout]   29:     0x58370154f91d - std::panicking::catch_unwind::h616b6e2e7a27f612
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/std/src/panicking.rs:553:19
[INFO] [stdout]   30:     0x58370154f91d - std::panic::catch_unwind::h1e788dd57758e6d8
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/std/src/panic.rs:359:14
[INFO] [stdout]   31:     0x58370154f91d - test::run_test_in_process::hf073c2764f29f8ad
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/test/src/lib.rs:686:27
[INFO] [stdout]   32:     0x58370154f91d - test::run_test::{{closure}}::h3dc46b7a0c340fa6
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/test/src/lib.rs:607:43
[INFO] [stdout]   33:     0x583701528e64 - test::run_test::{{closure}}::h6a4da3c57ef4505f
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/test/src/lib.rs:637:41
[INFO] [stdout]   34:     0x583701528e64 - std::sys::backtrace::__rust_begin_short_backtrace::hfd8e44bc311a5d57
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/std/src/sys/backtrace.rs:158:18
[INFO] [stdout]   35:     0x58370152c6ba - std::thread::Builder::spawn_unchecked_::{{closure}}::{{closure}}::h57c4ddec344fe24c
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/std/src/thread/mod.rs:562:17
[INFO] [stdout]   36:     0x58370152c6ba - <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once::hd7cbe09591f06dfb
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/core/src/panic/unwind_safe.rs:274:9
[INFO] [stdout]   37:     0x58370152c6ba - std::panicking::catch_unwind::do_call::hdcd076e8e993dfbc
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/std/src/panicking.rs:590:40
[INFO] [stdout]   38:     0x58370152c6ba - std::panicking::catch_unwind::h8f9f675f3756eab1
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/std/src/panicking.rs:553:19
[INFO] [stdout]   39:     0x58370152c6ba - std::panic::catch_unwind::he8f74a93abeceb9b
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/std/src/panic.rs:359:14
[INFO] [stdout]   40:     0x58370152c6ba - std::thread::Builder::spawn_unchecked_::{{closure}}::he43db13a2caa41d5
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/std/src/thread/mod.rs:560:30
[INFO] [stdout]   41:     0x58370152c6ba - core::ops::function::FnOnce::call_once{{vtable.shim}}::h8f3531a7e0d83514
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/core/src/ops/function.rs:250:5
[INFO] [stdout]   42:     0x583701770c4f - <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once::h6ff05134d80ef20e
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/alloc/src/boxed.rs:2085:9
[INFO] [stdout]   43:     0x583701770c4f - std::sys::thread::unix::Thread::new::thread_start::h9a4a41a076a486e0
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/std/src/sys/thread/unix.rs:124:17
[INFO] [stdout]   44:     0x7ce4ce7fbaa4 - <unknown>
[INFO] [stdout]   45:     0x7ce4ce888a64 - clone
[INFO] [stdout]   46:                0x0 - <unknown>
[INFO] [stdout] 
[INFO] [stdout] ---- analyzer::tests::test_determine_topology_strategy_gke_with_fabric stdout ----
[INFO] [stdout] ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ Snapshot Summary ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
[INFO] [stdout] Snapshot file: src/snapshots/hermes__analyzer__tests__determine_topology_strategy_gke_with_fabric.snap
[INFO] [stdout] Snapshot: determine_topology_strategy_gke_with_fabric
[INFO] [stdout] Source: src/analyzer.rs:575
[INFO] [stdout] ────────────────────────────────────────────────────────────────────────────────
[INFO] [stdout] Expression: result
[INFO] [stdout] ────────────────────────────────────────────────────────────────────────────────
[INFO] [stdout] +new results
[INFO] [stdout] ────────────┬───────────────────────────────────────────────────────────────────
[INFO] [stdout]           1 │+topology_type: Hardware
[INFO] [stdout]           2 │+detection_method: GKE RDMA fabric domain analysis
[INFO] [stdout]           3 │+confidence: High
[INFO] [stdout] ────────────┴───────────────────────────────────────────────────────────────────
[INFO] [stdout] 
[INFO] [stdout] thread 'analyzer::tests::test_determine_topology_strategy_gke_with_fabric' (26) panicked at src/analyzer.rs:575:9:
[INFO] [stdout] called `Result::unwrap()` on an `Err` value: Os { code: 30, kind: ReadOnlyFilesystem, message: "Read-only file system" }
[INFO] [stdout] stack backtrace:
[INFO] [stdout]    0:     0x58370179e432 - std::backtrace_rs::backtrace::libunwind::trace::h16affffe904e891e
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/std/src/../../backtrace/src/backtrace/libunwind.rs:117:9
[INFO] [stdout]    1:     0x58370179e432 - std::backtrace_rs::backtrace::trace_unsynchronized::h5c14b13373ed4150
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/std/src/../../backtrace/src/backtrace/mod.rs:66:14
[INFO] [stdout]    2:     0x58370179e432 - std::sys::backtrace::_print_fmt::hcbb507f162c816cc
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/std/src/sys/backtrace.rs:66:9
[INFO] [stdout]    3:     0x58370179e432 - <std::sys::backtrace::BacktraceLock::print::DisplayBacktrace as core::fmt::Display>::fmt::h8be9aa933f14675f
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/std/src/sys/backtrace.rs:39:26
[INFO] [stdout]    4:     0x5837017b0c1f - core::fmt::rt::Argument::fmt::h30ed739d33467c3a
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/core/src/fmt/rt.rs:173:76
[INFO] [stdout]    5:     0x5837017b0c1f - core::fmt::write::hfd0efbb002ac7eea
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/core/src/fmt/mod.rs:1469:25
[INFO] [stdout]    6:     0x583701769203 - std::io::default_write_fmt::hd6d24501f2d7f8d3
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/std/src/io/mod.rs:639:11
[INFO] [stdout]    7:     0x583701769203 - std::io::Write::write_fmt::h79eca2f72fc24111
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/std/src/io/mod.rs:1954:13
[INFO] [stdout]    8:     0x583701775902 - std::sys::backtrace::BacktraceLock::print::hf2554f6030d393f7
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/std/src/sys/backtrace.rs:42:9
[INFO] [stdout]    9:     0x58370177b82f - std::panicking::default_hook::{{closure}}::h8873121c56335b01
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/std/src/panicking.rs:301:27
[INFO] [stdout]   10:     0x58370177b6c1 - std::panicking::default_hook::hbafefc2d196267a2
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/std/src/panicking.rs:325:9
[INFO] [stdout]   11:     0x58370153babe - <alloc::boxed::Box<F,A> as core::ops::function::Fn<Args>>::call::ha834d5846f91b30b
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/alloc/src/boxed.rs:2099:9
[INFO] [stdout]   12:     0x58370153babe - test::test_main_with_exit_callback::{{closure}}::h63c167737eecb025
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/test/src/lib.rs:145:21
[INFO] [stdout]   13:     0x58370177be3f - <alloc::boxed::Box<F,A> as core::ops::function::Fn<Args>>::call::hbf9b0f7a281291fd
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/alloc/src/boxed.rs:2099:9
[INFO] [stdout]   14:     0x58370177be3f - std::panicking::panic_with_hook::h9f5b09d5adc1a745
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/std/src/panicking.rs:842:13
[INFO] [stdout]   15:     0x58370177bc9a - std::panicking::panic_handler::{{closure}}::h08111e483bdf6a89
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/std/src/panicking.rs:707:13
[INFO] [stdout]   16:     0x583701775a39 - std::sys::backtrace::__rust_end_short_backtrace::h1b86e3414ecbbe8d
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/std/src/sys/backtrace.rs:174:18
[INFO] [stdout]   17:     0x58370175c79d - __rustc[b292c645e8102103]::rust_begin_unwind
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/std/src/panicking.rs:698:5
[INFO] [stdout]   18:     0x5837017ba580 - core::panicking::panic_fmt::h31cc490ecc8cc1fa
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/core/src/panicking.rs:80:14
[INFO] [stdout]   19:     0x5837017b97d6 - core::result::unwrap_failed::he05762ef9e6bdf9c
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/core/src/result.rs:1862:5
[INFO] [stdout]   20:     0x58370150da12 - core::result::Result<T,E>::unwrap::h2298545f13ad2571
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/core/src/result.rs:1233:23
[INFO] [stdout]   21:     0x58370150da12 - hermes::analyzer::tests::test_determine_topology_strategy_gke_with_fabric::h831ccac724613f70
[INFO] [stdout]                                at /opt/rustwide/workdir/src/analyzer.rs:575:9
[INFO] [stdout]   22:     0x58370150db57 - hermes::analyzer::tests::test_determine_topology_strategy_gke_with_fabric::{{closure}}::h884e7c111e59e327
[INFO] [stdout]                                at /opt/rustwide/workdir/src/analyzer.rs:570:58
[INFO] [stdout]   23:     0x5837015157f6 - core::ops::function::FnOnce::call_once::heb80f66df1bf6f44
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/core/src/ops/function.rs:250:5
[INFO] [stdout]   24:     0x58370153b8bb - core::ops::function::FnOnce::call_once::h2b2de5fdd23aab3e
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/core/src/ops/function.rs:250:5
[INFO] [stdout]   25:     0x58370153b8bb - test::__rust_begin_short_backtrace::he551dd004770be01
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/test/src/lib.rs:663:18
[INFO] [stdout]   26:     0x58370154f91d - test::run_test_in_process::{{closure}}::h5f0b44080a35ed87
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/test/src/lib.rs:686:74
[INFO] [stdout]   27:     0x58370154f91d - <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once::h4dbf65d14893ecf5
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/core/src/panic/unwind_safe.rs:274:9
[INFO] [stdout]   28:     0x58370154f91d - std::panicking::catch_unwind::do_call::hc37c563b8a006285
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/std/src/panicking.rs:590:40
[INFO] [stdout]   29:     0x58370154f91d - std::panicking::catch_unwind::h616b6e2e7a27f612
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/std/src/panicking.rs:553:19
[INFO] [stdout]   30:     0x58370154f91d - std::panic::catch_unwind::h1e788dd57758e6d8
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/std/src/panic.rs:359:14
[INFO] [stdout]   31:     0x58370154f91d - test::run_test_in_process::hf073c2764f29f8ad
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/test/src/lib.rs:686:27
[INFO] [stdout]   32:     0x58370154f91d - test::run_test::{{closure}}::h3dc46b7a0c340fa6
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/test/src/lib.rs:607:43
[INFO] [stdout]   33:     0x583701528e64 - test::run_test::{{closure}}::h6a4da3c57ef4505f
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/test/src/lib.rs:637:41
[INFO] [stdout]   34:     0x583701528e64 - std::sys::backtrace::__rust_begin_short_backtrace::hfd8e44bc311a5d57
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/std/src/sys/backtrace.rs:158:18
[INFO] [stdout]   35:     0x58370152c6ba - std::thread::Builder::spawn_unchecked_::{{closure}}::{{closure}}::h57c4ddec344fe24c
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/std/src/thread/mod.rs:562:17
[INFO] [stdout]   36:     0x58370152c6ba - <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once::hd7cbe09591f06dfb
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/core/src/panic/unwind_safe.rs:274:9
[INFO] [stdout]   37:     0x58370152c6ba - std::panicking::catch_unwind::do_call::hdcd076e8e993dfbc
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/std/src/panicking.rs:590:40
[INFO] [stdout]   38:     0x58370152c6ba - std::panicking::catch_unwind::h8f9f675f3756eab1
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/std/src/panicking.rs:553:19
[INFO] [stdout]   39:     0x58370152c6ba - std::panic::catch_unwind::he8f74a93abeceb9b
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/std/src/panic.rs:359:14
[INFO] [stdout]   40:     0x58370152c6ba - std::thread::Builder::spawn_unchecked_::{{closure}}::he43db13a2caa41d5
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/std/src/thread/mod.rs:560:30
[INFO] [stdout]   41:     0x58370152c6ba - core::ops::function::FnOnce::call_once{{vtable.shim}}::h8f3531a7e0d83514
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/core/src/ops/function.rs:250:5
[INFO] [stdout]   42:     0x583701770c4f - <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once::h6ff05134d80ef20e
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/alloc/src/boxed.rs:2085:9
[INFO] [stdout]   43:     0x583701770c4f - std::sys::thread::unix::Thread::new::thread_start::h9a4a41a076a486e0
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/std/src/sys/thread/unix.rs:124:17
[INFO] [stdout]   44:     0x7ce4ce7fbaa4 - <unknown>
[INFO] [stdout]   45:     0x7ce4ce888a64 - clone
[INFO] [stdout]   46:                0x0 - <unknown>
[INFO] [stdout] 
[INFO] [stdout] ---- analyzer::tests::test_analyze_node_gke_with_rdma stdout ----
[INFO] [stdout] ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ Snapshot Summary ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
[INFO] [stdout] Snapshot file: src/snapshots/hermes__analyzer__tests__analyze_node_gke_with_rdma.snap
[INFO] [stdout] Snapshot: analyze_node_gke_with_rdma
[INFO] [stdout] Source: src/analyzer.rs:564
[INFO] [stdout] ────────────────────────────────────────────────────────────────────────────────
[INFO] [stdout] Expression: result
[INFO] [stdout] ────────────────────────────────────────────────────────────────────────────────
[INFO] [stdout] +new results
[INFO] [stdout] ────────────┬───────────────────────────────────────────────────────────────────
[INFO] [stdout]           1 │+name: test-node-1
[INFO] [stdout]           2 │+rdma_capability: Capable
[INFO] [stdout]           3 │+rdma_type: GKE RDMA
[INFO] [stdout]           4 │+rdma_resource: 1 RDMA interfaces
[INFO] [stdout]           5 │+platform_type: GKE
[INFO] [stdout]           6 │+topology_block: us-central1-a-gpu-pool
[INFO] [stdout]           7 │+topology_detection:
[INFO] [stdout]           8 │+  topology_type: Zone
[INFO] [stdout]           9 │+  detection_method: GKE zone+nodepool topology
[INFO] [stdout]          10 │+  confidence: Medium
[INFO] [stdout]          11 │+ib_speed: ~
[INFO] [stdout]          12 │+ib_fabric: ~
[INFO] [stdout]          13 │+ib_ports: ~
[INFO] [stdout]          14 │+leafgroup: ~
[INFO] [stdout]          15 │+superpod: ~
[INFO] [stdout]          16 │+neighbors: []
[INFO] [stdout]          17 │+mellanox_nics: []
[INFO] [stdout]          18 │+node_labels:
[INFO] [stdout]          19 │+  cloud.google.com/gke-nodepool: gpu-pool
[INFO] [stdout]          20 │+  topology.gke.io/zone: us-central1-a
[INFO] [stdout]          21 │+gpu_count: 8
[INFO] [stdout]          22 │+gpu_type: NVIDIA GPU
[INFO] [stdout]          23 │+gpu_allocatable: 8
[INFO] [stdout]          24 │+gpu_allocated: ~
[INFO] [stdout]          25 │+cpu_allocatable: ~
[INFO] [stdout]          26 │+cpu_allocated: ~
[INFO] [stdout]          27 │+memory_allocatable: ~
[INFO] [stdout]          28 │+memory_allocated: ~
[INFO] [stdout]          29 │+platform_data:
[INFO] [stdout]          30 │+  Gke:
[INFO] [stdout]          31 │+    nodepool: gpu-pool
[INFO] [stdout]          32 │+    machine_family: ~
[INFO] [stdout]          33 │+    zone: us-central1-a
[INFO] [stdout]          34 │+    rdma_interfaces: []
[INFO] [stdout]          35 │+    pci_topology: ~
[INFO] [stdout]          36 │+    fabric_domain: ~
[INFO] [stdout]          37 │+    topology_block: ~
[INFO] [stdout]          38 │+    topology_subblock: ~
[INFO] [stdout]          39 │+    topology_host: ~
[INFO] [stdout]          40 │+image_cache_status: Unknown
[INFO] [stdout]          41 │+image_cache_checked_at: ~
[INFO] [stdout] ────────────┴───────────────────────────────────────────────────────────────────
[INFO] [stdout] 
[INFO] [stdout] thread 'analyzer::tests::test_analyze_node_gke_with_rdma' (25) panicked at src/analyzer.rs:564:9:
[INFO] [stdout] called `Result::unwrap()` on an `Err` value: Os { code: 30, kind: ReadOnlyFilesystem, message: "Read-only file system" }
[INFO] [stdout] stack backtrace:
[INFO] [stdout]    0:     0x58370179e432 - std::backtrace_rs::backtrace::libunwind::trace::h16affffe904e891e
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/std/src/../../backtrace/src/backtrace/libunwind.rs:117:9
[INFO] [stdout]    1:     0x58370179e432 - std::backtrace_rs::backtrace::trace_unsynchronized::h5c14b13373ed4150
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/std/src/../../backtrace/src/backtrace/mod.rs:66:14
[INFO] [stdout]    2:     0x58370179e432 - std::sys::backtrace::_print_fmt::hcbb507f162c816cc
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/std/src/sys/backtrace.rs:66:9
[INFO] [stdout]    3:     0x58370179e432 - <std::sys::backtrace::BacktraceLock::print::DisplayBacktrace as core::fmt::Display>::fmt::h8be9aa933f14675f
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/std/src/sys/backtrace.rs:39:26
[INFO] [stdout]    4:     0x5837017b0c1f - core::fmt::rt::Argument::fmt::h30ed739d33467c3a
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/core/src/fmt/rt.rs:173:76
[INFO] [stdout]    5:     0x5837017b0c1f - core::fmt::write::hfd0efbb002ac7eea
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/core/src/fmt/mod.rs:1469:25
[INFO] [stdout]    6:     0x583701769203 - std::io::default_write_fmt::hd6d24501f2d7f8d3
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/std/src/io/mod.rs:639:11
[INFO] [stdout]    7:     0x583701769203 - std::io::Write::write_fmt::h79eca2f72fc24111
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/std/src/io/mod.rs:1954:13
[INFO] [stdout]    8:     0x583701775902 - std::sys::backtrace::BacktraceLock::print::hf2554f6030d393f7
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/std/src/sys/backtrace.rs:42:9
[INFO] [stdout]    9:     0x58370177b82f - std::panicking::default_hook::{{closure}}::h8873121c56335b01
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/std/src/panicking.rs:301:27
[INFO] [stdout]   10:     0x58370177b6c1 - std::panicking::default_hook::hbafefc2d196267a2
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/std/src/panicking.rs:325:9
[INFO] [stdout]   11:     0x58370153babe - <alloc::boxed::Box<F,A> as core::ops::function::Fn<Args>>::call::ha834d5846f91b30b
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/alloc/src/boxed.rs:2099:9
[INFO] [stdout]   12:     0x58370153babe - test::test_main_with_exit_callback::{{closure}}::h63c167737eecb025
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/test/src/lib.rs:145:21
[INFO] [stdout]   13:     0x58370177be3f - <alloc::boxed::Box<F,A> as core::ops::function::Fn<Args>>::call::hbf9b0f7a281291fd
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/alloc/src/boxed.rs:2099:9
[INFO] [stdout]   14:     0x58370177be3f - std::panicking::panic_with_hook::h9f5b09d5adc1a745
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/std/src/panicking.rs:842:13
[INFO] [stdout]   15:     0x58370177bc9a - std::panicking::panic_handler::{{closure}}::h08111e483bdf6a89
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/std/src/panicking.rs:707:13
[INFO] [stdout]   16:     0x583701775a39 - std::sys::backtrace::__rust_end_short_backtrace::h1b86e3414ecbbe8d
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/std/src/sys/backtrace.rs:174:18
[INFO] [stdout]   17:     0x58370175c79d - __rustc[b292c645e8102103]::rust_begin_unwind
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/std/src/panicking.rs:698:5
[INFO] [stdout]   18:     0x5837017ba580 - core::panicking::panic_fmt::h31cc490ecc8cc1fa
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/core/src/panicking.rs:80:14
[INFO] [stdout]   19:     0x5837017b97d6 - core::result::unwrap_failed::he05762ef9e6bdf9c
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/core/src/result.rs:1862:5
[INFO] [stdout]   20:     0x58370150cd8b - core::result::Result<T,E>::unwrap::h2298545f13ad2571
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/core/src/result.rs:1233:23
[INFO] [stdout]   21:     0x58370150cd8b - hermes::analyzer::tests::test_analyze_node_gke_with_rdma::h202a63fef7b8cc03
[INFO] [stdout]                                at /opt/rustwide/workdir/src/analyzer.rs:564:9
[INFO] [stdout]   22:     0x58370150d1d7 - hermes::analyzer::tests::test_analyze_node_gke_with_rdma::{{closure}}::hd9c8f273cba02090
[INFO] [stdout]                                at /opt/rustwide/workdir/src/analyzer.rs:559:41
[INFO] [stdout]   23:     0x5837015155a6 - core::ops::function::FnOnce::call_once::h0345d4db9b396c7a
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/core/src/ops/function.rs:250:5
[INFO] [stdout]   24:     0x58370153b8bb - core::ops::function::FnOnce::call_once::h2b2de5fdd23aab3e
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/core/src/ops/function.rs:250:5
[INFO] [stdout]   25:     0x58370153b8bb - test::__rust_begin_short_backtrace::he551dd004770be01
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/test/src/lib.rs:663:18
[INFO] [stdout]   26:     0x58370154f91d - test::run_test_in_process::{{closure}}::h5f0b44080a35ed87
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/test/src/lib.rs:686:74
[INFO] [stdout]   27:     0x58370154f91d - <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once::h4dbf65d14893ecf5
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/core/src/panic/unwind_safe.rs:274:9
[INFO] [stdout]   28:     0x58370154f91d - std::panicking::catch_unwind::do_call::hc37c563b8a006285
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/std/src/panicking.rs:590:40
[INFO] [stdout]   29:     0x58370154f91d - std::panicking::catch_unwind::h616b6e2e7a27f612
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/std/src/panicking.rs:553:19
[INFO] [stdout]   30:     0x58370154f91d - std::panic::catch_unwind::h1e788dd57758e6d8
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/std/src/panic.rs:359:14
[INFO] [stdout]   31:     0x58370154f91d - test::run_test_in_process::hf073c2764f29f8ad
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/test/src/lib.rs:686:27
[INFO] [stdout]   32:     0x58370154f91d - test::run_test::{{closure}}::h3dc46b7a0c340fa6
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/test/src/lib.rs:607:43
[INFO] [stdout]   33:     0x583701528e64 - test::run_test::{{closure}}::h6a4da3c57ef4505f
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/test/src/lib.rs:637:41
[INFO] [stdout]   34:     0x583701528e64 - std::sys::backtrace::__rust_begin_short_backtrace::hfd8e44bc311a5d57
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/std/src/sys/backtrace.rs:158:18
[INFO] [stdout]   35:     0x58370152c6ba - std::thread::Builder::spawn_unchecked_::{{closure}}::{{closure}}::h57c4ddec344fe24c
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/std/src/thread/mod.rs:562:17
[INFO] [stdout]   36:     0x58370152c6ba - <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once::hd7cbe09591f06dfb
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/core/src/panic/unwind_safe.rs:274:9
[INFO] [stdout]   37:     0x58370152c6ba - std::panicking::catch_unwind::do_call::hdcd076e8e993dfbc
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/std/src/panicking.rs:590:40
[INFO] [stdout]   38:     0x58370152c6ba - std::panicking::catch_unwind::h8f9f675f3756eab1
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/std/src/panicking.rs:553:19
[INFO] [stdout]   39:     0x58370152c6ba - std::panic::catch_unwind::he8f74a93abeceb9b
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/std/src/panic.rs:359:14
[INFO] [stdout]   40:     0x58370152c6ba - std::thread::Builder::spawn_unchecked_::{{closure}}::he43db13a2caa41d5
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/std/src/thread/mod.rs:560:30
[INFO] [stdout]   41:     0x58370152c6ba - core::ops::function::FnOnce::call_once{{vtable.shim}}::h8f3531a7e0d83514
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/core/src/ops/function.rs:250:5
[INFO] [stdout]   42:     0x583701770c4f - <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once::h6ff05134d80ef20e
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/alloc/src/boxed.rs:2085:9
[INFO] [stdout]   43:     0x583701770c4f - std::sys::thread::unix::Thread::new::thread_start::h9a4a41a076a486e0
[INFO] [stdout]                                at /rustc/9f93af291970322f4f1c6315ccde4d7078201159/library/std/src/sys/thread/unix.rs:124:17
[INFO] [stdout]   44:     0x7ce4ce7fbaa4 - <unknown>
[INFO] [stdout]   45:     0x7ce4ce888a64 - clone
[INFO] [stdout]   46:                0x0 - <unknown>
[INFO] [stdout] 
[INFO] [stdout] 
[INFO] [stdout] failures:
[INFO] [stdout]     analyzer::tests::test_analyze_node_gke_with_rdma
[INFO] [stdout]     analyzer::tests::test_determine_topology_strategy_gke_with_fabric
[INFO] [stdout]     topology_selector::tests::test_coreweave_selector_same_leafgroup
[INFO] [stdout]     topology_selector::tests::test_gke_selector_different_topologies
[INFO] [stdout] 
[INFO] [stdout] test result: FAILED. 1 passed; 4 failed; 0 ignored; 0 measured; 0 filtered out; finished in 1.00s
[INFO] [stdout] 
[INFO] [stderr] error: test failed, to rerun pass `--lib`
[INFO] running `Command { std: "docker" "inspect" "58dc7266a790258cccfae05e9c9a374b04a09c6a22faa146248a77c65949c2bc", kill_on_drop: false }`
[INFO] running `Command { std: "docker" "rm" "-f" "58dc7266a790258cccfae05e9c9a374b04a09c6a22faa146248a77c65949c2bc", kill_on_drop: false }`
[INFO] [stdout] 58dc7266a790258cccfae05e9c9a374b04a09c6a22faa146248a77c65949c2bc
