[INFO] cloning repository https://github.com/llm-d-incubation/hermes
[INFO] running `Command { std: "git" "-c" "credential.helper=" "-c" "credential.helper=/workspace/cargo-home/bin/git-credential-null" "clone" "--bare" "https://github.com/llm-d-incubation/hermes" "/workspace/cache/git-repos/https%3A%2F%2Fgithub.com%2Fllm-d-incubation%2Fhermes", kill_on_drop: false }`
[INFO] [stderr] Cloning into bare repository '/workspace/cache/git-repos/https%3A%2F%2Fgithub.com%2Fllm-d-incubation%2Fhermes'...
[INFO] running `Command { std: "git" "rev-parse" "HEAD", kill_on_drop: false }`
[INFO] [stdout] 9937cdee66cc904174409844eed37558be1f34ff
[INFO] testing llm-d-incubation/hermes against master#1ef7943ee607160a564655b6596f83670ef95df5 for pr-146098-6
[INFO] running `Command { std: "git" "clone" "/workspace/cache/git-repos/https%3A%2F%2Fgithub.com%2Fllm-d-incubation%2Fhermes" "/workspace/builds/worker-4-tc1/source", kill_on_drop: false }`
[INFO] [stderr] Cloning into '/workspace/builds/worker-4-tc1/source'...
[INFO] [stderr] done.
[INFO] started tweaking git repo https://github.com/llm-d-incubation/hermes
[INFO] finished tweaking git repo https://github.com/llm-d-incubation/hermes
[INFO] tweaked toml for git repo https://github.com/llm-d-incubation/hermes written to /workspace/builds/worker-4-tc1/source/Cargo.toml
[INFO] validating manifest of git repo https://github.com/llm-d-incubation/hermes on toolchain 1ef7943ee607160a564655b6596f83670ef95df5
[INFO] running `Command { std: CARGO_HOME="/workspace/cargo-home" RUSTUP_HOME="/workspace/rustup-home" "/workspace/cargo-home/bin/cargo" "+1ef7943ee607160a564655b6596f83670ef95df5" "metadata" "--manifest-path" "Cargo.toml" "--no-deps", kill_on_drop: false }`
[INFO] crate git repo https://github.com/llm-d-incubation/hermes already has a lockfile, it will not be regenerated
[INFO] running `Command { std: CARGO_HOME="/workspace/cargo-home" RUSTUP_HOME="/workspace/rustup-home" "/workspace/cargo-home/bin/cargo" "+1ef7943ee607160a564655b6596f83670ef95df5" "fetch" "--manifest-path" "Cargo.toml", kill_on_drop: false }`
[INFO] [stderr]     Updating crates.io index
[INFO] [stderr]  Downloading crates ...
[INFO] [stderr]   Downloaded windows-link v0.2.0
[INFO] [stderr]   Downloaded wasi v0.14.7+wasi-0.2.4
[INFO] [stderr]   Downloaded prettytable-rs v0.10.0
[INFO] [stderr]   Downloaded bitflags v2.9.4
[INFO] [stderr]   Downloaded schemars v1.0.4
[INFO] [stderr]   Downloaded clap v4.5.48
[INFO] [stderr]   Downloaded windows-targets v0.53.4
[INFO] [stderr]   Downloaded anstyle v1.0.12
[INFO] [stderr]   Downloaded pest_derive v2.8.2
[INFO] [stderr]   Downloaded kube-client v2.0.1
[INFO] [stderr]   Downloaded rustls v0.23.32
[INFO] [stderr]   Downloaded rustls-webpki v0.103.6
[INFO] [stderr]   Downloaded cc v1.2.39
[INFO] [stderr]   Downloaded kube-core v2.0.1
[INFO] [stderr]   Downloaded insta v1.43.2
[INFO] [stderr]   Downloaded security-framework v3.5.1
[INFO] [stderr]   Downloaded clap_builder v4.5.48
[INFO] [stderr]   Downloaded minijinja v2.12.0
[INFO] [stderr]   Downloaded pest v2.8.2
[INFO] [stderr]   Downloaded derive_more-impl v2.0.1
[INFO] [stderr]   Downloaded derive_more v2.0.1
[INFO] [stderr]   Downloaded regex-syntax v0.8.6
[INFO] [stderr]   Downloaded schemars_derive v1.0.4
[INFO] [stderr]   Downloaded jsonpath-rust v0.7.5
[INFO] [stderr]   Downloaded kube v2.0.1
[INFO] [stderr]   Downloaded clap_derive v4.5.47
[INFO] [stderr]   Downloaded pest_meta v2.8.2
[INFO] [stderr]   Downloaded find-msvc-tools v0.1.2
[INFO] [stderr]   Downloaded windows-interface v0.59.2
[INFO] [stderr]   Downloaded windows-strings v0.5.0
[INFO] [stderr]   Downloaded windows-core v0.62.1
[INFO] [stderr]   Downloaded kube-derive v2.0.1
[INFO] [stderr]   Downloaded owo-colors v4.2.3
[INFO] [stderr]   Downloaded csv-core v0.1.12
[INFO] [stderr]   Downloaded libredox v0.1.10
[INFO] [stderr]   Downloaded anstream v0.6.20
[INFO] [stderr]   Downloaded console v0.15.11
[INFO] [stderr]   Downloaded hyper-timeout v0.5.2
[INFO] [stderr]   Downloaded pest_generator v2.8.2
[INFO] [stderr]   Downloaded secrecy v0.10.3
[INFO] [stderr]   Downloaded windows-implement v0.60.1
[INFO] [stderr]   Downloaded windows-result v0.4.0
[INFO] [stderr]   Downloaded csv v1.3.1
[INFO] [stderr]   Downloaded ordered-float v2.10.1
[INFO] [stderr]   Downloaded libc v0.2.176
[INFO] [stderr]   Downloaded core-foundation v0.10.1
[INFO] [stderr]   Downloaded rustls-native-certs v0.8.1
[INFO] [stderr]   Downloaded io-uring v0.7.10
[INFO] [stderr]   Downloaded indexmap v2.11.4
[INFO] [stderr]   Downloaded unicode-ident v1.0.19
[INFO] [stderr]   Downloaded nu-ansi-term v0.50.1
[INFO] [stderr]   Downloaded k8s-openapi v0.26.0
[INFO] [stderr]   Downloaded windows-sys v0.61.1
[INFO] [stderr]   Downloaded regex v1.11.3
[INFO] [stderr]   Downloaded regex-automata v0.4.11
[INFO] running `Command { std: "docker" "create" "-v" "/var/lib/crater-agent-workspace/builds/worker-4-tc1/target:/opt/rustwide/target:rw,Z" "-v" "/var/lib/crater-agent-workspace/builds/worker-4-tc1/source:/opt/rustwide/workdir:ro,Z" "-v" "/var/lib/crater-agent-workspace/cargo-home:/opt/rustwide/cargo-home:ro,Z" "-v" "/var/lib/crater-agent-workspace/rustup-home:/opt/rustwide/rustup-home:ro,Z" "-e" "SOURCE_DIR=/opt/rustwide/workdir" "-e" "CARGO_TARGET_DIR=/opt/rustwide/target" "-e" "CARGO_HOME=/opt/rustwide/cargo-home" "-e" "RUSTUP_HOME=/opt/rustwide/rustup-home" "-w" "/opt/rustwide/workdir" "-m" "1610612736" "--user" "0:0" "--network" "none" "ghcr.io/rust-lang/crates-build-env/linux@sha256:4848fb76d95f26979359cc7e45710b1dbc8f3acb7aeedee7c460d7702230f228" "/opt/rustwide/cargo-home/bin/cargo" "+1ef7943ee607160a564655b6596f83670ef95df5" "metadata" "--no-deps" "--format-version=1", kill_on_drop: false }`
[INFO] [stdout] f9d5a41290ee44992cab191dbf70f33d914f0b59e65b62757f72ffa0b43f1064
[INFO] running `Command { std: "docker" "start" "-a" "f9d5a41290ee44992cab191dbf70f33d914f0b59e65b62757f72ffa0b43f1064", kill_on_drop: false }`
[INFO] running `Command { std: "docker" "inspect" "f9d5a41290ee44992cab191dbf70f33d914f0b59e65b62757f72ffa0b43f1064", kill_on_drop: false }`
[INFO] running `Command { std: "docker" "rm" "-f" "f9d5a41290ee44992cab191dbf70f33d914f0b59e65b62757f72ffa0b43f1064", kill_on_drop: false }`
[INFO] [stdout] f9d5a41290ee44992cab191dbf70f33d914f0b59e65b62757f72ffa0b43f1064
[INFO] running `Command { std: "docker" "create" "-v" "/var/lib/crater-agent-workspace/builds/worker-4-tc1/target:/opt/rustwide/target:rw,Z" "-v" "/var/lib/crater-agent-workspace/builds/worker-4-tc1/source:/opt/rustwide/workdir:ro,Z" "-v" "/var/lib/crater-agent-workspace/cargo-home:/opt/rustwide/cargo-home:ro,Z" "-v" "/var/lib/crater-agent-workspace/rustup-home:/opt/rustwide/rustup-home:ro,Z" "-e" "SOURCE_DIR=/opt/rustwide/workdir" "-e" "CARGO_TARGET_DIR=/opt/rustwide/target" "-e" "CARGO_INCREMENTAL=0" "-e" "RUST_BACKTRACE=full" "-e" "RUSTFLAGS=--cap-lints=forbid" "-e" "RUSTDOCFLAGS=--cap-lints=forbid" "-e" "CARGO_HOME=/opt/rustwide/cargo-home" "-e" "RUSTUP_HOME=/opt/rustwide/rustup-home" "-w" "/opt/rustwide/workdir" "-m" "1610612736" "--user" "0:0" "--network" "none" "ghcr.io/rust-lang/crates-build-env/linux@sha256:4848fb76d95f26979359cc7e45710b1dbc8f3acb7aeedee7c460d7702230f228" "/opt/rustwide/cargo-home/bin/cargo" "+1ef7943ee607160a564655b6596f83670ef95df5" "build" "--frozen" "--message-format=json", kill_on_drop: false }`
[INFO] [stdout] e95d840647eeb0604c92a7e4ebf9780b02142f1d893007928c876b0f3795418f
[INFO] running `Command { std: "docker" "start" "-a" "e95d840647eeb0604c92a7e4ebf9780b02142f1d893007928c876b0f3795418f", kill_on_drop: false }`
[INFO] [stderr]    Compiling proc-macro2 v1.0.101
[INFO] [stderr]    Compiling unicode-ident v1.0.19
[INFO] [stderr]    Compiling libc v0.2.176
[INFO] [stderr]    Compiling cfg-if v1.0.3
[INFO] [stderr]    Compiling autocfg v1.5.0
[INFO] [stderr]    Compiling serde_core v1.0.228
[INFO] [stderr]    Compiling parking_lot_core v0.9.11
[INFO] [stderr]    Compiling serde v1.0.228
[INFO] [stderr]    Compiling find-msvc-tools v0.1.2
[INFO] [stderr]    Compiling futures-channel v0.3.31
[INFO] [stderr]    Compiling futures-io v0.3.31
[INFO] [stderr]    Compiling zeroize v1.8.1
[INFO] [stderr]    Compiling serde_json v1.0.145
[INFO] [stderr]    Compiling version_check v0.9.5
[INFO] [stderr]    Compiling ucd-trie v0.1.7
[INFO] [stderr]    Compiling typenum v1.18.0
[INFO] [stderr]    Compiling untrusted v0.9.0
[INFO] [stderr]    Compiling rustls v0.23.32
[INFO] [stderr]    Compiling cc v1.2.39
[INFO] [stderr]    Compiling rustls-pki-types v1.12.0
[INFO] [stderr]    Compiling ref-cast v1.0.25
[INFO] [stderr]    Compiling k8s-openapi v0.26.0
[INFO] [stderr]    Compiling regex-syntax v0.8.6
[INFO] [stderr]    Compiling iana-time-zone v0.1.64
[INFO] [stderr]    Compiling ident_case v1.0.1
[INFO] [stderr]    Compiling subtle v2.6.1
[INFO] [stderr]    Compiling indexmap v2.11.4
[INFO] [stderr]    Compiling generic-array v0.14.7
[INFO] [stderr]    Compiling dyn-clone v1.0.20
[INFO] [stderr]    Compiling lock_api v0.4.13
[INFO] [stderr]    Compiling num-traits v0.2.19
[INFO] [stderr]    Compiling rustls-native-certs v0.8.1
[INFO] [stderr]    Compiling unsafe-libyaml v0.2.11
[INFO] [stderr]    Compiling is_terminal_polyfill v1.70.1
[INFO] [stderr]    Compiling sync_wrapper v1.0.2
[INFO] [stderr]    Compiling getrandom v0.3.3
[INFO] [stderr]    Compiling bitflags v2.9.4
[INFO] [stderr]    Compiling anstyle v1.0.12
[INFO] [stderr]    Compiling quote v1.0.41
[INFO] [stderr]    Compiling anstream v0.6.20
[INFO] [stderr]    Compiling secrecy v0.10.3
[INFO] [stderr]    Compiling pem v3.0.5
[INFO] [stderr]    Compiling csv-core v0.1.12
[INFO] [stderr]    Compiling syn v2.0.106
[INFO] [stderr]    Compiling option-ext v0.2.0
[INFO] [stderr]    Compiling clap_lex v0.7.5
[INFO] [stderr]    Compiling home v0.5.11
[INFO] [stderr]    Compiling either v1.15.0
[INFO] [stderr]    Compiling anyhow v1.0.100
[INFO] [stderr]    Compiling signal-hook-registry v1.4.6
[INFO] [stderr]    Compiling mio v1.0.4
[INFO] [stderr]    Compiling socket2 v0.6.0
[INFO] [stderr]    Compiling getrandom v0.2.16
[INFO] [stderr]    Compiling parking_lot v0.12.4
[INFO] [stderr]    Compiling ordered-float v2.10.1
[INFO] [stderr]    Compiling dirs-sys-next v0.1.2
[INFO] [stderr]    Compiling owo-colors v4.2.3
[INFO] [stderr]    Compiling dirs-next v2.0.0
[INFO] [stderr]    Compiling is-terminal v0.4.16
[INFO] [stderr]    Compiling block-buffer v0.10.4
[INFO] [stderr]    Compiling crypto-common v0.1.6
[INFO] [stderr]    Compiling dirs-sys v0.4.1
[INFO] [stderr]    Compiling digest v0.10.7
[INFO] [stderr]    Compiling clap_builder v4.5.48
[INFO] [stderr]    Compiling thread_local v1.1.9
[INFO] [stderr]    Compiling hermes v0.2.5 (/opt/rustwide/workdir)
[INFO] [stderr]    Compiling encode_unicode v1.0.0
[INFO] [stderr]    Compiling unicode-width v0.1.14
[INFO] [stderr]    Compiling regex-automata v0.4.11
[INFO] [stderr]    Compiling ring v0.17.14
[INFO] [stderr]    Compiling cpufeatures v0.2.17
[INFO] [stderr]    Compiling nu-ansi-term v0.50.1
[INFO] [stderr]    Compiling term v0.7.0
[INFO] [stderr]    Compiling sha2 v0.10.9
[INFO] [stderr]    Compiling dirs v5.0.1
[INFO] [stderr]    Compiling uuid v1.18.1
[INFO] [stderr]    Compiling tracing-subscriber v0.3.20
[INFO] [stderr]    Compiling regex v1.11.3
[INFO] [stderr]    Compiling serde_derive_internals v0.29.1
[INFO] [stderr]    Compiling darling_core v0.21.3
[INFO] [stderr]    Compiling serde_derive v1.0.228
[INFO] [stderr]    Compiling tokio-macros v2.5.0
[INFO] [stderr]    Compiling thiserror-impl v2.0.17
[INFO] [stderr]    Compiling futures-macro v0.3.31
[INFO] [stderr]    Compiling tracing-attributes v0.1.30
[INFO] [stderr]    Compiling ref-cast-impl v1.0.25
[INFO] [stderr]    Compiling derive_more-impl v2.0.1
[INFO] [stderr]    Compiling clap_derive v4.5.47
[INFO] [stderr]    Compiling tokio v1.47.1
[INFO] [stderr]    Compiling schemars_derive v1.0.4
[INFO] [stderr]    Compiling futures-util v0.3.31
[INFO] [stderr]    Compiling derive_more v2.0.1
[INFO] [stderr]    Compiling tracing v0.1.41
[INFO] [stderr]    Compiling thiserror v2.0.17
[INFO] [stderr]    Compiling pest v2.8.2
[INFO] [stderr]    Compiling tower-http v0.6.6
[INFO] [stderr]    Compiling darling_macro v0.21.3
[INFO] [stderr]    Compiling rustls-webpki v0.103.6
[INFO] [stderr]    Compiling darling v0.21.3
[INFO] [stderr]    Compiling clap v4.5.48
[INFO] [stderr]    Compiling pest_meta v2.8.2
[INFO] [stderr]    Compiling futures-executor v0.3.31
[INFO] [stderr]    Compiling pest_generator v2.8.2
[INFO] [stderr]    Compiling futures v0.3.31
[INFO] [stderr]    Compiling chrono v0.4.42
[INFO] [stderr]    Compiling schemars v1.0.4
[INFO] [stderr]    Compiling serde-value v0.7.0
[INFO] [stderr]    Compiling serde_yaml v0.9.34+deprecated
[INFO] [stderr]    Compiling csv v1.3.1
[INFO] [stderr]    Compiling minijinja v2.12.0
[INFO] [stderr]    Compiling kube-derive v2.0.1
[INFO] [stderr]    Compiling prettytable-rs v0.10.0
[INFO] [stderr]    Compiling pest_derive v2.8.2
[INFO] [stderr]    Compiling jsonpath-rust v0.7.5
[INFO] [stderr]    Compiling hyper v1.7.0
[INFO] [stderr]    Compiling tokio-util v0.7.16
[INFO] [stderr]    Compiling tokio-stream v0.1.17
[INFO] [stderr]    Compiling tokio-rustls v0.26.4
[INFO] [stderr]    Compiling tower v0.5.2
[INFO] [stderr]    Compiling hyper-util v0.1.17
[INFO] [stderr]    Compiling hyper-rustls v0.27.7
[INFO] [stderr]    Compiling hyper-timeout v0.5.2
[INFO] [stderr]    Compiling kube-core v2.0.1
[INFO] [stderr]    Compiling kube-client v2.0.1
[INFO] [stderr]    Compiling kube v2.0.1
[INFO] [stderr]     Finished `dev` profile [unoptimized + debuginfo] target(s) in 1m 12s
[INFO] running `Command { std: "docker" "inspect" "e95d840647eeb0604c92a7e4ebf9780b02142f1d893007928c876b0f3795418f", kill_on_drop: false }`
[INFO] running `Command { std: "docker" "rm" "-f" "e95d840647eeb0604c92a7e4ebf9780b02142f1d893007928c876b0f3795418f", kill_on_drop: false }`
[INFO] [stdout] e95d840647eeb0604c92a7e4ebf9780b02142f1d893007928c876b0f3795418f
[INFO] running `Command { std: "docker" "create" "-v" "/var/lib/crater-agent-workspace/builds/worker-4-tc1/target:/opt/rustwide/target:rw,Z" "-v" "/var/lib/crater-agent-workspace/builds/worker-4-tc1/source:/opt/rustwide/workdir:ro,Z" "-v" "/var/lib/crater-agent-workspace/cargo-home:/opt/rustwide/cargo-home:ro,Z" "-v" "/var/lib/crater-agent-workspace/rustup-home:/opt/rustwide/rustup-home:ro,Z" "-e" "SOURCE_DIR=/opt/rustwide/workdir" "-e" "CARGO_TARGET_DIR=/opt/rustwide/target" "-e" "CARGO_INCREMENTAL=0" "-e" "RUST_BACKTRACE=full" "-e" "RUSTFLAGS=--cap-lints=forbid" "-e" "RUSTDOCFLAGS=--cap-lints=forbid" "-e" "CARGO_HOME=/opt/rustwide/cargo-home" "-e" "RUSTUP_HOME=/opt/rustwide/rustup-home" "-w" "/opt/rustwide/workdir" "-m" "1610612736" "--user" "0:0" "--network" "none" "ghcr.io/rust-lang/crates-build-env/linux@sha256:4848fb76d95f26979359cc7e45710b1dbc8f3acb7aeedee7c460d7702230f228" "/opt/rustwide/cargo-home/bin/cargo" "+1ef7943ee607160a564655b6596f83670ef95df5" "test" "--frozen" "--no-run" "--message-format=json", kill_on_drop: false }`
[INFO] [stdout] 5c6d90371de0f0789bdf3bf6f3bf121e9c9869ba86ac3d0cddffab8364c4a9c7
[INFO] running `Command { std: "docker" "start" "-a" "5c6d90371de0f0789bdf3bf6f3bf121e9c9869ba86ac3d0cddffab8364c4a9c7", kill_on_drop: false }`
[INFO] [stderr]    Compiling console v0.15.11
[INFO] [stderr]    Compiling similar v2.7.0
[INFO] [stderr]    Compiling insta v1.43.2
[INFO] [stderr]    Compiling hermes v0.2.5 (/opt/rustwide/workdir)
[INFO] [stderr]     Finished `test` profile [unoptimized + debuginfo] target(s) in 9.44s
[INFO] running `Command { std: "docker" "inspect" "5c6d90371de0f0789bdf3bf6f3bf121e9c9869ba86ac3d0cddffab8364c4a9c7", kill_on_drop: false }`
[INFO] running `Command { std: "docker" "rm" "-f" "5c6d90371de0f0789bdf3bf6f3bf121e9c9869ba86ac3d0cddffab8364c4a9c7", kill_on_drop: false }`
[INFO] [stdout] 5c6d90371de0f0789bdf3bf6f3bf121e9c9869ba86ac3d0cddffab8364c4a9c7
[INFO] running `Command { std: "docker" "create" "-v" "/var/lib/crater-agent-workspace/builds/worker-4-tc1/target:/opt/rustwide/target:rw,Z" "-v" "/var/lib/crater-agent-workspace/builds/worker-4-tc1/source:/opt/rustwide/workdir:ro,Z" "-v" "/var/lib/crater-agent-workspace/cargo-home:/opt/rustwide/cargo-home:ro,Z" "-v" "/var/lib/crater-agent-workspace/rustup-home:/opt/rustwide/rustup-home:ro,Z" "-e" "SOURCE_DIR=/opt/rustwide/workdir" "-e" "CARGO_TARGET_DIR=/opt/rustwide/target" "-e" "CARGO_INCREMENTAL=0" "-e" "RUST_BACKTRACE=full" "-e" "RUSTFLAGS=--cap-lints=forbid" "-e" "RUSTDOCFLAGS=--cap-lints=forbid" "-e" "CARGO_HOME=/opt/rustwide/cargo-home" "-e" "RUSTUP_HOME=/opt/rustwide/rustup-home" "-w" "/opt/rustwide/workdir" "-m" "1610612736" "--user" "0:0" "--network" "none" "ghcr.io/rust-lang/crates-build-env/linux@sha256:4848fb76d95f26979359cc7e45710b1dbc8f3acb7aeedee7c460d7702230f228" "/opt/rustwide/cargo-home/bin/cargo" "+1ef7943ee607160a564655b6596f83670ef95df5" "test" "--frozen", kill_on_drop: false }`
[INFO] [stdout] b04886e2dc2b5b0d4bf7210831a32998e3dab9ed347f3ac19f0db47583d61c10
[INFO] running `Command { std: "docker" "start" "-a" "b04886e2dc2b5b0d4bf7210831a32998e3dab9ed347f3ac19f0db47583d61c10", kill_on_drop: false }`
[INFO] [stderr]     Finished `test` profile [unoptimized + debuginfo] target(s) in 0.25s
[INFO] [stderr]      Running unittests src/lib.rs (/opt/rustwide/target/debug/deps/hermes-3ed0216f65cac55f)
[INFO] [stdout] 
[INFO] [stdout] running 5 tests
[INFO] [stdout] test cache::tests::test_humanize_duration ... ok
[INFO] [stdout] test analyzer::tests::test_analyze_node_gke_with_rdma ... FAILED
[INFO] [stdout] test topology_selector::tests::test_gke_selector_different_topologies ... FAILED
[INFO] [stdout] test analyzer::tests::test_determine_topology_strategy_gke_with_fabric ... FAILED
[INFO] [stdout] test topology_selector::tests::test_coreweave_selector_same_leafgroup ... FAILED
[INFO] [stdout] 
[INFO] [stdout] failures:
[INFO] [stdout] 
[INFO] [stdout] ---- analyzer::tests::test_analyze_node_gke_with_rdma stdout ----
[INFO] [stdout] ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ Snapshot Summary ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
[INFO] [stdout] Snapshot file: src/snapshots/hermes__analyzer__tests__analyze_node_gke_with_rdma.snap
[INFO] [stdout] Snapshot: analyze_node_gke_with_rdma
[INFO] [stdout] Source: src/analyzer.rs:564
[INFO] [stdout] ────────────────────────────────────────────────────────────────────────────────
[INFO] [stdout] Expression: result
[INFO] [stdout] ────────────────────────────────────────────────────────────────────────────────
[INFO] [stdout] +new results
[INFO] [stdout] ────────────┬───────────────────────────────────────────────────────────────────
[INFO] [stdout]           1 │+name: test-node-1
[INFO] [stdout]           2 │+rdma_capability: Capable
[INFO] [stdout]           3 │+rdma_type: GKE RDMA
[INFO] [stdout]           4 │+rdma_resource: 1 RDMA interfaces
[INFO] [stdout]           5 │+platform_type: GKE
[INFO] [stdout]           6 │+topology_block: us-central1-a-gpu-pool
[INFO] [stdout]           7 │+topology_detection:
[INFO] [stdout]           8 │+  topology_type: Zone
[INFO] [stdout]           9 │+  detection_method: GKE zone+nodepool topology
[INFO] [stdout]          10 │+  confidence: Medium
[INFO] [stdout]          11 │+ib_speed: ~
[INFO] [stdout]          12 │+ib_fabric: ~
[INFO] [stdout]          13 │+ib_ports: ~
[INFO] [stdout]          14 │+leafgroup: ~
[INFO] [stdout]          15 │+superpod: ~
[INFO] [stdout]          16 │+neighbors: []
[INFO] [stdout]          17 │+mellanox_nics: []
[INFO] [stdout]          18 │+node_labels:
[INFO] [stdout]          19 │+  cloud.google.com/gke-nodepool: gpu-pool
[INFO] [stdout]          20 │+  topology.gke.io/zone: us-central1-a
[INFO] [stdout]          21 │+gpu_count: 8
[INFO] [stdout]          22 │+gpu_type: NVIDIA GPU
[INFO] [stdout]          23 │+gpu_allocatable: 8
[INFO] [stdout]          24 │+gpu_allocated: ~
[INFO] [stdout]          25 │+cpu_allocatable: ~
[INFO] [stdout]          26 │+cpu_allocated: ~
[INFO] [stdout]          27 │+memory_allocatable: ~
[INFO] [stdout]          28 │+memory_allocated: ~
[INFO] [stdout]          29 │+platform_data:
[INFO] [stdout]          30 │+  Gke:
[INFO] [stdout]          31 │+    nodepool: gpu-pool
[INFO] [stdout]          32 │+    machine_family: ~
[INFO] [stdout]          33 │+    zone: us-central1-a
[INFO] [stdout]          34 │+    rdma_interfaces: []
[INFO] [stdout]          35 │+    pci_topology: ~
[INFO] [stdout]          36 │+    fabric_domain: ~
[INFO] [stdout]          37 │+    topology_block: ~
[INFO] [stdout]          38 │+    topology_subblock: ~
[INFO] [stdout]          39 │+    topology_host: ~
[INFO] [stdout]          40 │+image_cache_status: Unknown
[INFO] [stdout]          41 │+image_cache_checked_at: ~
[INFO] [stdout] ────────────┴───────────────────────────────────────────────────────────────────
[INFO] [stdout] 
[INFO] [stdout] thread 'analyzer::tests::test_analyze_node_gke_with_rdma' (25) panicked at src/analyzer.rs:564:9:
[INFO] [stdout] called `Result::unwrap()` on an `Err` value: Os { code: 30, kind: ReadOnlyFilesystem, message: "Read-only file system" }
[INFO] [stdout] stack backtrace:
[INFO] [stdout]    0:     0x64676dfb0b22 - std::backtrace_rs::backtrace::libunwind::trace::h786de35fecf3582f
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/std/src/../../backtrace/src/backtrace/libunwind.rs:117:9
[INFO] [stdout]    1:     0x64676dfb0b22 - std::backtrace_rs::backtrace::trace_unsynchronized::h4a7da1a2a64387f1
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/std/src/../../backtrace/src/backtrace/mod.rs:66:14
[INFO] [stdout]    2:     0x64676dfb0b22 - std::sys::backtrace::_print_fmt::h6bd7d500070c788c
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/std/src/sys/backtrace.rs:66:9
[INFO] [stdout]    3:     0x64676dfb0b22 - <std::sys::backtrace::BacktraceLock::print::DisplayBacktrace as core::fmt::Display>::fmt::h6d82c1afff976903
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/std/src/sys/backtrace.rs:39:26
[INFO] [stdout]    4:     0x64676dfc330f - core::fmt::rt::Argument::fmt::hc4ce6d643d397690
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/core/src/fmt/rt.rs:173:76
[INFO] [stdout]    5:     0x64676dfc330f - core::fmt::write::hb1e7ca88b6a3936e
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/core/src/fmt/mod.rs:1469:25
[INFO] [stdout]    6:     0x64676df7b7c3 - std::io::default_write_fmt::haffd49d96f1984a8
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/std/src/io/mod.rs:639:11
[INFO] [stdout]    7:     0x64676df7b7c3 - std::io::Write::write_fmt::h027871c57cf57c01
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/std/src/io/mod.rs:1954:13
[INFO] [stdout]    8:     0x64676df87ff2 - std::sys::backtrace::BacktraceLock::print::ha2430613ee79d059
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/std/src/sys/backtrace.rs:42:9
[INFO] [stdout]    9:     0x64676df8df1f - std::panicking::default_hook::{{closure}}::hdbd2db9e5c303cf6
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/std/src/panicking.rs:301:27
[INFO] [stdout]   10:     0x64676df8ddb1 - std::panicking::default_hook::hed93c70cba5fdcf0
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/std/src/panicking.rs:325:9
[INFO] [stdout]   11:     0x64676dd4d60e - <alloc::boxed::Box<F,A> as core::ops::function::Fn<Args>>::call::hd0ee8b569efc6a07
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/alloc/src/boxed.rs:2099:9
[INFO] [stdout]   12:     0x64676dd4d60e - test::test_main_with_exit_callback::{{closure}}::hf10864b576ecd15d
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/test/src/lib.rs:145:21
[INFO] [stdout]   13:     0x64676df8e52f - <alloc::boxed::Box<F,A> as core::ops::function::Fn<Args>>::call::h3a55ca34534c0d00
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/alloc/src/boxed.rs:2099:9
[INFO] [stdout]   14:     0x64676df8e52f - std::panicking::panic_with_hook::h3862d766c2cec19b
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/std/src/panicking.rs:842:13
[INFO] [stdout]   15:     0x64676df8e38a - std::panicking::panic_handler::{{closure}}::hb95eb402b5e28ee1
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/std/src/panicking.rs:707:13
[INFO] [stdout]   16:     0x64676df88129 - std::sys::backtrace::__rust_end_short_backtrace::hf73a26dc1835d85a
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/std/src/sys/backtrace.rs:174:18
[INFO] [stdout]   17:     0x64676df6ee8d - __rustc[6ed5915ee467787]::rust_begin_unwind
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/std/src/panicking.rs:698:5
[INFO] [stdout]   18:     0x64676dfccc70 - core::panicking::panic_fmt::h3454303eb8e6f7cd
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/core/src/panicking.rs:80:14
[INFO] [stdout]   19:     0x64676dfcbec6 - core::result::unwrap_failed::h2380020e6daeb464
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/core/src/result.rs:1862:5
[INFO] [stdout]   20:     0x64676dd189cb - core::result::Result<T,E>::unwrap::he149a572cf4c6167
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/core/src/result.rs:1233:23
[INFO] [stdout]   21:     0x64676dd189cb - hermes::analyzer::tests::test_analyze_node_gke_with_rdma::h10037e92cbf4c51b
[INFO] [stdout]                                at /opt/rustwide/workdir/src/analyzer.rs:564:9
[INFO] [stdout]   22:     0x64676dd18ad7 - hermes::analyzer::tests::test_analyze_node_gke_with_rdma::{{closure}}::h6653e6d38ca2ac7e
[INFO] [stdout]                                at /opt/rustwide/workdir/src/analyzer.rs:559:41
[INFO] [stdout]   23:     0x64676dd1e6e6 - core::ops::function::FnOnce::call_once::hbc5a2f676cde6104
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/core/src/ops/function.rs:250:5
[INFO] [stdout]   24:     0x64676dd4d3eb - core::ops::function::FnOnce::call_once::ha729ee35d2fab541
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/core/src/ops/function.rs:250:5
[INFO] [stdout]   25:     0x64676dd4d3eb - test::__rust_begin_short_backtrace::ha40f4db8207e1111
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/test/src/lib.rs:663:18
[INFO] [stdout]   26:     0x64676dd6146d - test::run_test_in_process::{{closure}}::hc1b77cda5d44f0f3
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/test/src/lib.rs:686:74
[INFO] [stdout]   27:     0x64676dd6146d - <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once::h1dbea1de64785521
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/core/src/panic/unwind_safe.rs:274:9
[INFO] [stdout]   28:     0x64676dd6146d - std::panicking::catch_unwind::do_call::hd5febe9affd5a1b3
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/std/src/panicking.rs:590:40
[INFO] [stdout]   29:     0x64676dd6146d - std::panicking::catch_unwind::h51ea89627559b6f4
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/std/src/panicking.rs:553:19
[INFO] [stdout]   30:     0x64676dd6146d - std::panic::catch_unwind::hb8b8c2367cae3d66
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/std/src/panic.rs:359:14
[INFO] [stdout]   31:     0x64676dd6146d - test::run_test_in_process::ha5b55801407ea100
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/test/src/lib.rs:686:27
[INFO] [stdout]   32:     0x64676dd6146d - test::run_test::{{closure}}::h0b9d4072b527abf5
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/test/src/lib.rs:607:43
[INFO] [stdout]   33:     0x64676dd3a984 - test::run_test::{{closure}}::hb1056b5731205822
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/test/src/lib.rs:637:41
[INFO] [stdout]   34:     0x64676dd3a984 - std::sys::backtrace::__rust_begin_short_backtrace::hb61ff5a34023c7ef
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/std/src/sys/backtrace.rs:158:18
[INFO] [stdout]   35:     0x64676dd3e2ba - std::thread::Builder::spawn_unchecked_::{{closure}}::{{closure}}::hb0a83c0e8b353cff
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/std/src/thread/mod.rs:562:17
[INFO] [stdout]   36:     0x64676dd3e2ba - <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once::hb4f32cb008535298
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/core/src/panic/unwind_safe.rs:274:9
[INFO] [stdout]   37:     0x64676dd3e2ba - std::panicking::catch_unwind::do_call::h16d110e4fc35789a
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/std/src/panicking.rs:590:40
[INFO] [stdout]   38:     0x64676dd3e2ba - std::panicking::catch_unwind::hea9118f355699c4b
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/std/src/panicking.rs:553:19
[INFO] [stdout]   39:     0x64676dd3e2ba - std::panic::catch_unwind::hf4b3c2a06d3f42f5
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/std/src/panic.rs:359:14
[INFO] [stdout]   40:     0x64676dd3e2ba - std::thread::Builder::spawn_unchecked_::{{closure}}::h3f4cb733a52d53d5
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/std/src/thread/mod.rs:560:30
[INFO] [stdout]   41:     0x64676dd3e2ba - core::ops::function::FnOnce::call_once{{vtable.shim}}::h4052f0967b37caeb
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/core/src/ops/function.rs:250:5
[INFO] [stdout]   42:     0x64676df8333f - <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once::h7ec433abd3f148b4
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/alloc/src/boxed.rs:2085:9
[INFO] [stdout]   43:     0x64676df8333f - std::sys::thread::unix::Thread::new::thread_start::he514622d3d7ba65c
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/std/src/sys/thread/unix.rs:124:17
[INFO] [stdout]   44:     0x7d3d12e31aa4 - <unknown>
[INFO] [stdout]   45:     0x7d3d12ebea64 - clone
[INFO] [stdout]   46:                0x0 - <unknown>
[INFO] [stdout] 
[INFO] [stdout] ---- topology_selector::tests::test_gke_selector_different_topologies stdout ----
[INFO] [stdout] ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ Snapshot Summary ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
[INFO] [stdout] Snapshot file: src/snapshots/hermes__topology_selector__tests__gke_selector_different_topologies.snap
[INFO] [stdout] Snapshot: gke_selector_different_topologies
[INFO] [stdout] Source: src/topology_selector.rs:301
[INFO] [stdout] ────────────────────────────────────────────────────────────────────────────────
[INFO] [stdout] Expression: reason
[INFO] [stdout] ────────────────────────────────────────────────────────────────────────────────
[INFO] [stdout] +new results
[INFO] [stdout] ────────────┬───────────────────────────────────────────────────────────────────
[INFO] [stdout]           1 │+GKE fallback: GKE RDMA RDMA nodes (topology may differ)
[INFO] [stdout] ────────────┴───────────────────────────────────────────────────────────────────
[INFO] [stdout] 
[INFO] [stdout] thread 'topology_selector::tests::test_gke_selector_different_topologies' (29) panicked at src/topology_selector.rs:301:9:
[INFO] [stdout] called `Result::unwrap()` on an `Err` value: Os { code: 30, kind: ReadOnlyFilesystem, message: "Read-only file system" }
[INFO] [stdout] stack backtrace:
[INFO] [stdout]    0:     0x64676dfb0b22 - std::backtrace_rs::backtrace::libunwind::trace::h786de35fecf3582f
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/std/src/../../backtrace/src/backtrace/libunwind.rs:117:9
[INFO] [stdout]    1:     0x64676dfb0b22 - std::backtrace_rs::backtrace::trace_unsynchronized::h4a7da1a2a64387f1
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/std/src/../../backtrace/src/backtrace/mod.rs:66:14
[INFO] [stdout]    2:     0x64676dfb0b22 - std::sys::backtrace::_print_fmt::h6bd7d500070c788c
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/std/src/sys/backtrace.rs:66:9
[INFO] [stdout]    3:     0x64676dfb0b22 - <std::sys::backtrace::BacktraceLock::print::DisplayBacktrace as core::fmt::Display>::fmt::h6d82c1afff976903
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/std/src/sys/backtrace.rs:39:26
[INFO] [stdout]    4:     0x64676dfc330f - core::fmt::rt::Argument::fmt::hc4ce6d643d397690
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/core/src/fmt/rt.rs:173:76
[INFO] [stdout]    5:     0x64676dfc330f - core::fmt::write::hb1e7ca88b6a3936e
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/core/src/fmt/mod.rs:1469:25
[INFO] [stdout]    6:     0x64676df7b7c3 - std::io::default_write_fmt::haffd49d96f1984a8
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/std/src/io/mod.rs:639:11
[INFO] [stdout]    7:     0x64676df7b7c3 - std::io::Write::write_fmt::h027871c57cf57c01
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/std/src/io/mod.rs:1954:13
[INFO] [stdout]    8:     0x64676df87ff2 - std::sys::backtrace::BacktraceLock::print::ha2430613ee79d059
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/std/src/sys/backtrace.rs:42:9
[INFO] [stdout]    9:     0x64676df8df1f - std::panicking::default_hook::{{closure}}::hdbd2db9e5c303cf6
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/std/src/panicking.rs:301:27
[INFO] [stdout]   10:     0x64676df8ddb1 - std::panicking::default_hook::hed93c70cba5fdcf0
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/std/src/panicking.rs:325:9
[INFO] [stdout]   11:     0x64676dd4d60e - <alloc::boxed::Box<F,A> as core::ops::function::Fn<Args>>::call::hd0ee8b569efc6a07
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/alloc/src/boxed.rs:2099:9
[INFO] [stdout]   12:     0x64676dd4d60e - test::test_main_with_exit_callback::{{closure}}::hf10864b576ecd15d
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/test/src/lib.rs:145:21
[INFO] [stdout]   13:     0x64676df8e52f - <alloc::boxed::Box<F,A> as core::ops::function::Fn<Args>>::call::h3a55ca34534c0d00
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/alloc/src/boxed.rs:2099:9
[INFO] [stdout]   14:     0x64676df8e52f - std::panicking::panic_with_hook::h3862d766c2cec19b
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/std/src/panicking.rs:842:13
[INFO] [stdout]   15:     0x64676df8e38a - std::panicking::panic_handler::{{closure}}::hb95eb402b5e28ee1
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/std/src/panicking.rs:707:13
[INFO] [stdout]   16:     0x64676df88129 - std::sys::backtrace::__rust_end_short_backtrace::hf73a26dc1835d85a
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/std/src/sys/backtrace.rs:174:18
[INFO] [stdout]   17:     0x64676df6ee8d - __rustc[6ed5915ee467787]::rust_begin_unwind
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/std/src/panicking.rs:698:5
[INFO] [stdout]   18:     0x64676dfccc70 - core::panicking::panic_fmt::h3454303eb8e6f7cd
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/core/src/panicking.rs:80:14
[INFO] [stdout]   19:     0x64676dfcbec6 - core::result::unwrap_failed::h2380020e6daeb464
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/core/src/result.rs:1862:5
[INFO] [stdout]   20:     0x64676dd25f03 - core::result::Result<T,E>::unwrap::he149a572cf4c6167
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/core/src/result.rs:1233:23
[INFO] [stdout]   21:     0x64676dd25f03 - hermes::topology_selector::tests::test_gke_selector_different_topologies::he2ee12b6af3e6a84
[INFO] [stdout]                                at /opt/rustwide/workdir/src/topology_selector.rs:301:9
[INFO] [stdout]   22:     0x64676dd26047 - hermes::topology_selector::tests::test_gke_selector_different_topologies::{{closure}}::h2f02e7114a69fc14
[INFO] [stdout]                                at /opt/rustwide/workdir/src/topology_selector.rs:287:48
[INFO] [stdout]   23:     0x64676dd1e766 - core::ops::function::FnOnce::call_once::hf5c864a7f95f44f3
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/core/src/ops/function.rs:250:5
[INFO] [stdout]   24:     0x64676dd4d3eb - core::ops::function::FnOnce::call_once::ha729ee35d2fab541
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/core/src/ops/function.rs:250:5
[INFO] [stdout]   25:     0x64676dd4d3eb - test::__rust_begin_short_backtrace::ha40f4db8207e1111
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/test/src/lib.rs:663:18
[INFO] [stdout]   26:     0x64676dd6146d - test::run_test_in_process::{{closure}}::hc1b77cda5d44f0f3
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/test/src/lib.rs:686:74
[INFO] [stdout]   27:     0x64676dd6146d - <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once::h1dbea1de64785521
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/core/src/panic/unwind_safe.rs:274:9
[INFO] [stdout]   28:     0x64676dd6146d - std::panicking::catch_unwind::do_call::hd5febe9affd5a1b3
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/std/src/panicking.rs:590:40
[INFO] [stdout]   29:     0x64676dd6146d - std::panicking::catch_unwind::h51ea89627559b6f4
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/std/src/panicking.rs:553:19
[INFO] [stdout]   30:     0x64676dd6146d - std::panic::catch_unwind::hb8b8c2367cae3d66
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/std/src/panic.rs:359:14
[INFO] [stdout]   31:     0x64676dd6146d - test::run_test_in_process::ha5b55801407ea100
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/test/src/lib.rs:686:27
[INFO] [stdout]   32:     0x64676dd6146d - test::run_test::{{closure}}::h0b9d4072b527abf5
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/test/src/lib.rs:607:43
[INFO] [stdout]   33:     0x64676dd3a984 - test::run_test::{{closure}}::hb1056b5731205822
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/test/src/lib.rs:637:41
[INFO] [stdout]   34:     0x64676dd3a984 - std::sys::backtrace::__rust_begin_short_backtrace::hb61ff5a34023c7ef
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/std/src/sys/backtrace.rs:158:18
[INFO] [stdout]   35:     0x64676dd3e2ba - std::thread::Builder::spawn_unchecked_::{{closure}}::{{closure}}::hb0a83c0e8b353cff
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/std/src/thread/mod.rs:562:17
[INFO] [stdout]   36:     0x64676dd3e2ba - <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once::hb4f32cb008535298
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/core/src/panic/unwind_safe.rs:274:9
[INFO] [stdout]   37:     0x64676dd3e2ba - std::panicking::catch_unwind::do_call::h16d110e4fc35789a
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/std/src/panicking.rs:590:40
[INFO] [stdout]   38:     0x64676dd3e2ba - std::panicking::catch_unwind::hea9118f355699c4b
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/std/src/panicking.rs:553:19
[INFO] [stdout]   39:     0x64676dd3e2ba - std::panic::catch_unwind::hf4b3c2a06d3f42f5
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/std/src/panic.rs:359:14
[INFO] [stdout]   40:     0x64676dd3e2ba - std::thread::Builder::spawn_unchecked_::{{closure}}::h3f4cb733a52d53d5
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/std/src/thread/mod.rs:560:30
[INFO] [stdout]   41:     0x64676dd3e2ba - core::ops::function::FnOnce::call_once{{vtable.shim}}::h4052f0967b37caeb
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/core/src/ops/function.rs:250:5
[INFO] [stdout]   42:     0x64676df8333f - <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once::h7ec433abd3f148b4
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/alloc/src/boxed.rs:2085:9
[INFO] [stdout]   43:     0x64676df8333f - std::sys::thread::unix::Thread::new::thread_start::he514622d3d7ba65c
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/std/src/sys/thread/unix.rs:124:17
[INFO] [stdout]   44:     0x7d3d12e31aa4 - <unknown>
[INFO] [stdout]   45:     0x7d3d12ebea64 - clone
[INFO] [stdout]   46:                0x0 - <unknown>
[INFO] [stdout] 
[INFO] [stdout] ---- analyzer::tests::test_determine_topology_strategy_gke_with_fabric stdout ----
[INFO] [stdout] ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ Snapshot Summary ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
[INFO] [stdout] Snapshot file: src/snapshots/hermes__analyzer__tests__determine_topology_strategy_gke_with_fabric.snap
[INFO] [stdout] Snapshot: determine_topology_strategy_gke_with_fabric
[INFO] [stdout] Source: src/analyzer.rs:575
[INFO] [stdout] ────────────────────────────────────────────────────────────────────────────────
[INFO] [stdout] Expression: result
[INFO] [stdout] ────────────────────────────────────────────────────────────────────────────────
[INFO] [stdout] +new results
[INFO] [stdout] ────────────┬───────────────────────────────────────────────────────────────────
[INFO] [stdout]           1 │+topology_type: Hardware
[INFO] [stdout]           2 │+detection_method: GKE RDMA fabric domain analysis
[INFO] [stdout]           3 │+confidence: High
[INFO] [stdout] ────────────┴───────────────────────────────────────────────────────────────────
[INFO] [stdout] 
[INFO] [stdout] thread 'analyzer::tests::test_determine_topology_strategy_gke_with_fabric' (26) panicked at src/analyzer.rs:575:9:
[INFO] [stdout] called `Result::unwrap()` on an `Err` value: Os { code: 30, kind: ReadOnlyFilesystem, message: "Read-only file system" }
[INFO] [stdout] stack backtrace:
[INFO] [stdout]    0:     0x64676dfb0b22 - std::backtrace_rs::backtrace::libunwind::trace::h786de35fecf3582f
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/std/src/../../backtrace/src/backtrace/libunwind.rs:117:9
[INFO] [stdout]    1:     0x64676dfb0b22 - std::backtrace_rs::backtrace::trace_unsynchronized::h4a7da1a2a64387f1
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/std/src/../../backtrace/src/backtrace/mod.rs:66:14
[INFO] [stdout]    2:     0x64676dfb0b22 - std::sys::backtrace::_print_fmt::h6bd7d500070c788c
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/std/src/sys/backtrace.rs:66:9
[INFO] [stdout]    3:     0x64676dfb0b22 - <std::sys::backtrace::BacktraceLock::print::DisplayBacktrace as core::fmt::Display>::fmt::h6d82c1afff976903
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/std/src/sys/backtrace.rs:39:26
[INFO] [stdout]    4:     0x64676dfc330f - core::fmt::rt::Argument::fmt::hc4ce6d643d397690
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/core/src/fmt/rt.rs:173:76
[INFO] [stdout]    5:     0x64676dfc330f - core::fmt::write::hb1e7ca88b6a3936e
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/core/src/fmt/mod.rs:1469:25
[INFO] [stdout]    6:     0x64676df7b7c3 - std::io::default_write_fmt::haffd49d96f1984a8
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/std/src/io/mod.rs:639:11
[INFO] [stdout]    7:     0x64676df7b7c3 - std::io::Write::write_fmt::h027871c57cf57c01
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/std/src/io/mod.rs:1954:13
[INFO] [stdout]    8:     0x64676df87ff2 - std::sys::backtrace::BacktraceLock::print::ha2430613ee79d059
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/std/src/sys/backtrace.rs:42:9
[INFO] [stdout]    9:     0x64676df8df1f - std::panicking::default_hook::{{closure}}::hdbd2db9e5c303cf6
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/std/src/panicking.rs:301:27
[INFO] [stdout]   10:     0x64676df8ddb1 - std::panicking::default_hook::hed93c70cba5fdcf0
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/std/src/panicking.rs:325:9
[INFO] [stdout]   11:     0x64676dd4d60e - <alloc::boxed::Box<F,A> as core::ops::function::Fn<Args>>::call::hd0ee8b569efc6a07
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/alloc/src/boxed.rs:2099:9
[INFO] [stdout]   12:     0x64676dd4d60e - test::test_main_with_exit_callback::{{closure}}::hf10864b576ecd15d
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/test/src/lib.rs:145:21
[INFO] [stdout]   13:     0x64676df8e52f - <alloc::boxed::Box<F,A> as core::ops::function::Fn<Args>>::call::h3a55ca34534c0d00
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/alloc/src/boxed.rs:2099:9
[INFO] [stdout]   14:     0x64676df8e52f - std::panicking::panic_with_hook::h3862d766c2cec19b
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/std/src/panicking.rs:842:13
[INFO] [stdout]   15:     0x64676df8e38a - std::panicking::panic_handler::{{closure}}::hb95eb402b5e28ee1
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/std/src/panicking.rs:707:13
[INFO] [stdout]   16:     0x64676df88129 - std::sys::backtrace::__rust_end_short_backtrace::hf73a26dc1835d85a
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/std/src/sys/backtrace.rs:174:18
[INFO] [stdout]   17:     0x64676df6ee8d - __rustc[6ed5915ee467787]::rust_begin_unwind
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/std/src/panicking.rs:698:5
[INFO] [stdout]   18:     0x64676dfccc70 - core::panicking::panic_fmt::h3454303eb8e6f7cd
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/core/src/panicking.rs:80:14
[INFO] [stdout]   19:     0x64676dfcbec6 - core::result::unwrap_failed::h2380020e6daeb464
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/core/src/result.rs:1862:5
[INFO] [stdout]   20:     0x64676dd19652 - core::result::Result<T,E>::unwrap::he149a572cf4c6167
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/core/src/result.rs:1233:23
[INFO] [stdout]   21:     0x64676dd19652 - hermes::analyzer::tests::test_determine_topology_strategy_gke_with_fabric::h0fe40ac5c97690b3
[INFO] [stdout]                                at /opt/rustwide/workdir/src/analyzer.rs:575:9
[INFO] [stdout]   22:     0x64676dd19797 - hermes::analyzer::tests::test_determine_topology_strategy_gke_with_fabric::{{closure}}::h6a54d7c9240846e8
[INFO] [stdout]                                at /opt/rustwide/workdir/src/analyzer.rs:570:58
[INFO] [stdout]   23:     0x64676dd1e666 - core::ops::function::FnOnce::call_once::h622306830a071e89
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/core/src/ops/function.rs:250:5
[INFO] [stdout]   24:     0x64676dd4d3eb - core::ops::function::FnOnce::call_once::ha729ee35d2fab541
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/core/src/ops/function.rs:250:5
[INFO] [stdout]   25:     0x64676dd4d3eb - test::__rust_begin_short_backtrace::ha40f4db8207e1111
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/test/src/lib.rs:663:18
[INFO] [stdout]   26:     0x64676dd6146d - test::run_test_in_process::{{closure}}::hc1b77cda5d44f0f3
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/test/src/lib.rs:686:74
[INFO] [stdout]   27:     0x64676dd6146d - <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once::h1dbea1de64785521
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/core/src/panic/unwind_safe.rs:274:9
[INFO] [stdout]   28:     0x64676dd6146d - std::panicking::catch_unwind::do_call::hd5febe9affd5a1b3
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/std/src/panicking.rs:590:40
[INFO] [stdout]   29:     0x64676dd6146d - std::panicking::catch_unwind::h51ea89627559b6f4
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/std/src/panicking.rs:553:19
[INFO] [stdout]   30:     0x64676dd6146d - std::panic::catch_unwind::hb8b8c2367cae3d66
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/std/src/panic.rs:359:14
[INFO] [stdout]   31:     0x64676dd6146d - test::run_test_in_process::ha5b55801407ea100
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/test/src/lib.rs:686:27
[INFO] [stdout]   32:     0x64676dd6146d - test::run_test::{{closure}}::h0b9d4072b527abf5
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/test/src/lib.rs:607:43
[INFO] [stdout]   33:     0x64676dd3a984 - test::run_test::{{closure}}::hb1056b5731205822
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/test/src/lib.rs:637:41
[INFO] [stdout]   34:     0x64676dd3a984 - std::sys::backtrace::__rust_begin_short_backtrace::hb61ff5a34023c7ef
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/std/src/sys/backtrace.rs:158:18
[INFO] [stdout]   35:     0x64676dd3e2ba - std::thread::Builder::spawn_unchecked_::{{closure}}::{{closure}}::hb0a83c0e8b353cff
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/std/src/thread/mod.rs:562:17
[INFO] [stdout]   36:     0x64676dd3e2ba - <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once::hb4f32cb008535298
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/core/src/panic/unwind_safe.rs:274:9
[INFO] [stdout]   37:     0x64676dd3e2ba - std::panicking::catch_unwind::do_call::h16d110e4fc35789a
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/std/src/panicking.rs:590:40
[INFO] [stdout]   38:     0x64676dd3e2ba - std::panicking::catch_unwind::hea9118f355699c4b
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/std/src/panicking.rs:553:19
[INFO] [stdout]   39:     0x64676dd3e2ba - std::panic::catch_unwind::hf4b3c2a06d3f42f5
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/std/src/panic.rs:359:14
[INFO] [stdout]   40:     0x64676dd3e2ba - std::thread::Builder::spawn_unchecked_::{{closure}}::h3f4cb733a52d53d5
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/std/src/thread/mod.rs:560:30
[INFO] [stdout]   41:     0x64676dd3e2ba - core::ops::function::FnOnce::call_once{{vtable.shim}}::h4052f0967b37caeb
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/core/src/ops/function.rs:250:5
[INFO] [stdout]   42:     0x64676df8333f - <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once::h7ec433abd3f148b4
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/alloc/src/boxed.rs:2085:9
[INFO] [stdout]   43:     0x64676df8333f - std::sys::thread::unix::Thread::new::thread_start::he514622d3d7ba65c
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/std/src/sys/thread/unix.rs:124:17
[INFO] [stdout]   44:     0x7d3d12e31aa4 - <unknown>
[INFO] [stdout]   45:     0x7d3d12ebea64 - clone
[INFO] [stdout]   46:                0x0 - <unknown>
[INFO] [stdout] 
[INFO] [stdout] ---- topology_selector::tests::test_coreweave_selector_same_leafgroup stdout ----
[INFO] [stdout] ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ Snapshot Summary ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
[INFO] [stdout] Snapshot file: src/snapshots/hermes__topology_selector__tests__coreweave_selector_same_leafgroup.snap
[INFO] [stdout] Snapshot: coreweave_selector_same_leafgroup
[INFO] [stdout] Source: src/topology_selector.rs:283
[INFO] [stdout] ────────────────────────────────────────────────────────────────────────────────
[INFO] [stdout] Expression: reason
[INFO] [stdout] ────────────────────────────────────────────────────────────────────────────────
[INFO] [stdout] +new results
[INFO] [stdout] ────────────┬───────────────────────────────────────────────────────────────────
[INFO] [stdout]           1 │+Optimal CoreWeave same-topology: RoCE RDMA within leafgroup 'lg1'
[INFO] [stdout] ────────────┴───────────────────────────────────────────────────────────────────
[INFO] [stdout] 
[INFO] [stdout] thread 'topology_selector::tests::test_coreweave_selector_same_leafgroup' (28) panicked at src/topology_selector.rs:283:9:
[INFO] [stdout] called `Result::unwrap()` on an `Err` value: Os { code: 30, kind: ReadOnlyFilesystem, message: "Read-only file system" }
[INFO] [stdout] stack backtrace:
[INFO] [stdout]    0:     0x64676dfb0b22 - std::backtrace_rs::backtrace::libunwind::trace::h786de35fecf3582f
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/std/src/../../backtrace/src/backtrace/libunwind.rs:117:9
[INFO] [stdout]    1:     0x64676dfb0b22 - std::backtrace_rs::backtrace::trace_unsynchronized::h4a7da1a2a64387f1
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/std/src/../../backtrace/src/backtrace/mod.rs:66:14
[INFO] [stdout]    2:     0x64676dfb0b22 - std::sys::backtrace::_print_fmt::h6bd7d500070c788c
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/std/src/sys/backtrace.rs:66:9
[INFO] [stdout]    3:     0x64676dfb0b22 - <std::sys::backtrace::BacktraceLock::print::DisplayBacktrace as core::fmt::Display>::fmt::h6d82c1afff976903
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/std/src/sys/backtrace.rs:39:26
[INFO] [stdout]    4:     0x64676dfc330f - core::fmt::rt::Argument::fmt::hc4ce6d643d397690
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/core/src/fmt/rt.rs:173:76
[INFO] [stdout]    5:     0x64676dfc330f - core::fmt::write::hb1e7ca88b6a3936e
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/core/src/fmt/mod.rs:1469:25
[INFO] [stdout]    6:     0x64676df7b7c3 - std::io::default_write_fmt::haffd49d96f1984a8
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/std/src/io/mod.rs:639:11
[INFO] [stdout]    7:     0x64676df7b7c3 - std::io::Write::write_fmt::h027871c57cf57c01
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/std/src/io/mod.rs:1954:13
[INFO] [stdout]    8:     0x64676df87ff2 - std::sys::backtrace::BacktraceLock::print::ha2430613ee79d059
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/std/src/sys/backtrace.rs:42:9
[INFO] [stdout]    9:     0x64676df8df1f - std::panicking::default_hook::{{closure}}::hdbd2db9e5c303cf6
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/std/src/panicking.rs:301:27
[INFO] [stdout]   10:     0x64676df8ddb1 - std::panicking::default_hook::hed93c70cba5fdcf0
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/std/src/panicking.rs:325:9
[INFO] [stdout]   11:     0x64676dd4d60e - <alloc::boxed::Box<F,A> as core::ops::function::Fn<Args>>::call::hd0ee8b569efc6a07
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/alloc/src/boxed.rs:2099:9
[INFO] [stdout]   12:     0x64676dd4d60e - test::test_main_with_exit_callback::{{closure}}::hf10864b576ecd15d
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/test/src/lib.rs:145:21
[INFO] [stdout]   13:     0x64676df8e52f - <alloc::boxed::Box<F,A> as core::ops::function::Fn<Args>>::call::h3a55ca34534c0d00
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/alloc/src/boxed.rs:2099:9
[INFO] [stdout]   14:     0x64676df8e52f - std::panicking::panic_with_hook::h3862d766c2cec19b
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/std/src/panicking.rs:842:13
[INFO] [stdout]   15:     0x64676df8e38a - std::panicking::panic_handler::{{closure}}::hb95eb402b5e28ee1
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/std/src/panicking.rs:707:13
[INFO] [stdout]   16:     0x64676df88129 - std::sys::backtrace::__rust_end_short_backtrace::hf73a26dc1835d85a
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/std/src/sys/backtrace.rs:174:18
[INFO] [stdout]   17:     0x64676df6ee8d - __rustc[6ed5915ee467787]::rust_begin_unwind
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/std/src/panicking.rs:698:5
[INFO] [stdout]   18:     0x64676dfccc70 - core::panicking::panic_fmt::h3454303eb8e6f7cd
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/core/src/panicking.rs:80:14
[INFO] [stdout]   19:     0x64676dfcbec6 - core::result::unwrap_failed::h2380020e6daeb464
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/core/src/result.rs:1862:5
[INFO] [stdout]   20:     0x64676dd2541b - core::result::Result<T,E>::unwrap::he149a572cf4c6167
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/core/src/result.rs:1233:23
[INFO] [stdout]   21:     0x64676dd2541b - hermes::topology_selector::tests::test_coreweave_selector_same_leafgroup::h7c42bd491c7c3288
[INFO] [stdout]                                at /opt/rustwide/workdir/src/topology_selector.rs:283:9
[INFO] [stdout]   22:     0x64676dd25607 - hermes::topology_selector::tests::test_coreweave_selector_same_leafgroup::{{closure}}::hbef22a46086120dd
[INFO] [stdout]                                at /opt/rustwide/workdir/src/topology_selector.rs:266:48
[INFO] [stdout]   23:     0x64676dd1e586 - core::ops::function::FnOnce::call_once::h360d0e0167eaa53e
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/core/src/ops/function.rs:250:5
[INFO] [stdout]   24:     0x64676dd4d3eb - core::ops::function::FnOnce::call_once::ha729ee35d2fab541
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/core/src/ops/function.rs:250:5
[INFO] [stdout]   25:     0x64676dd4d3eb - test::__rust_begin_short_backtrace::ha40f4db8207e1111
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/test/src/lib.rs:663:18
[INFO] [stderr] error: test failed, to rerun pass `--lib`
[INFO] [stdout]   26:     0x64676dd6146d - test::run_test_in_process::{{closure}}::hc1b77cda5d44f0f3
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/test/src/lib.rs:686:74
[INFO] [stdout]   27:     0x64676dd6146d - <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once::h1dbea1de64785521
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/core/src/panic/unwind_safe.rs:274:9
[INFO] [stdout]   28:     0x64676dd6146d - std::panicking::catch_unwind::do_call::hd5febe9affd5a1b3
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/std/src/panicking.rs:590:40
[INFO] [stdout]   29:     0x64676dd6146d - std::panicking::catch_unwind::h51ea89627559b6f4
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/std/src/panicking.rs:553:19
[INFO] [stdout]   30:     0x64676dd6146d - std::panic::catch_unwind::hb8b8c2367cae3d66
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/std/src/panic.rs:359:14
[INFO] [stdout]   31:     0x64676dd6146d - test::run_test_in_process::ha5b55801407ea100
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/test/src/lib.rs:686:27
[INFO] [stdout]   32:     0x64676dd6146d - test::run_test::{{closure}}::h0b9d4072b527abf5
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/test/src/lib.rs:607:43
[INFO] [stdout]   33:     0x64676dd3a984 - test::run_test::{{closure}}::hb1056b5731205822
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/test/src/lib.rs:637:41
[INFO] [stdout]   34:     0x64676dd3a984 - std::sys::backtrace::__rust_begin_short_backtrace::hb61ff5a34023c7ef
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/std/src/sys/backtrace.rs:158:18
[INFO] [stdout]   35:     0x64676dd3e2ba - std::thread::Builder::spawn_unchecked_::{{closure}}::{{closure}}::hb0a83c0e8b353cff
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/std/src/thread/mod.rs:562:17
[INFO] [stdout]   36:     0x64676dd3e2ba - <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once::hb4f32cb008535298
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/core/src/panic/unwind_safe.rs:274:9
[INFO] [stdout]   37:     0x64676dd3e2ba - std::panicking::catch_unwind::do_call::h16d110e4fc35789a
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/std/src/panicking.rs:590:40
[INFO] [stdout]   38:     0x64676dd3e2ba - std::panicking::catch_unwind::hea9118f355699c4b
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/std/src/panicking.rs:553:19
[INFO] [stdout]   39:     0x64676dd3e2ba - std::panic::catch_unwind::hf4b3c2a06d3f42f5
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/std/src/panic.rs:359:14
[INFO] [stdout]   40:     0x64676dd3e2ba - std::thread::Builder::spawn_unchecked_::{{closure}}::h3f4cb733a52d53d5
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/std/src/thread/mod.rs:560:30
[INFO] [stdout]   41:     0x64676dd3e2ba - core::ops::function::FnOnce::call_once{{vtable.shim}}::h4052f0967b37caeb
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/core/src/ops/function.rs:250:5
[INFO] [stdout]   42:     0x64676df8333f - <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once::h7ec433abd3f148b4
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/alloc/src/boxed.rs:2085:9
[INFO] [stdout]   43:     0x64676df8333f - std::sys::thread::unix::Thread::new::thread_start::he514622d3d7ba65c
[INFO] [stdout]                                at /rustc/1ef7943ee607160a564655b6596f83670ef95df5/library/std/src/sys/thread/unix.rs:124:17
[INFO] [stdout]   44:     0x7d3d12e31aa4 - <unknown>
[INFO] [stdout]   45:     0x7d3d12ebea64 - clone
[INFO] [stdout]   46:                0x0 - <unknown>
[INFO] [stdout] 
[INFO] [stdout] 
[INFO] [stdout] failures:
[INFO] [stdout]     analyzer::tests::test_analyze_node_gke_with_rdma
[INFO] [stdout]     analyzer::tests::test_determine_topology_strategy_gke_with_fabric
[INFO] [stdout]     topology_selector::tests::test_coreweave_selector_same_leafgroup
[INFO] [stdout]     topology_selector::tests::test_gke_selector_different_topologies
[INFO] [stdout] 
[INFO] [stdout] test result: FAILED. 1 passed; 4 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.81s
[INFO] [stdout] 
[INFO] running `Command { std: "docker" "inspect" "b04886e2dc2b5b0d4bf7210831a32998e3dab9ed347f3ac19f0db47583d61c10", kill_on_drop: false }`
[INFO] running `Command { std: "docker" "rm" "-f" "b04886e2dc2b5b0d4bf7210831a32998e3dab9ed347f3ac19f0db47583d61c10", kill_on_drop: false }`
[INFO] [stdout] b04886e2dc2b5b0d4bf7210831a32998e3dab9ed347f3ac19f0db47583d61c10
