[INFO] cloning repository https://github.com/ilyalabun/raft_vibe
[INFO] running `Command { std: "git" "-c" "credential.helper=" "-c" "credential.helper=/workspace/cargo-home/bin/git-credential-null" "clone" "--bare" "https://github.com/ilyalabun/raft_vibe" "/workspace/cache/git-repos/https%3A%2F%2Fgithub.com%2Filyalabun%2Fraft_vibe", kill_on_drop: false }`
[INFO] [stderr] Cloning into bare repository '/workspace/cache/git-repos/https%3A%2F%2Fgithub.com%2Filyalabun%2Fraft_vibe'...
[INFO] running `Command { std: "git" "rev-parse" "HEAD", kill_on_drop: false }`
[INFO] [stdout] 5bc1857f6fc1da0bb02004c76ceba93c19faa99c
[INFO] testing ilyalabun/raft_vibe against 1.94.0 for beta-1.95-1
[INFO] running `Command { std: "git" "clone" "/workspace/cache/git-repos/https%3A%2F%2Fgithub.com%2Filyalabun%2Fraft_vibe" "/workspace/builds/worker-7-tc1/source", kill_on_drop: false }`
[INFO] [stderr] Cloning into '/workspace/builds/worker-7-tc1/source'...
[INFO] [stderr] done.
[INFO] started tweaking git repo https://github.com/ilyalabun/raft_vibe
[INFO] finished tweaking git repo https://github.com/ilyalabun/raft_vibe
[INFO] tweaked toml for git repo https://github.com/ilyalabun/raft_vibe written to /workspace/builds/worker-7-tc1/source/Cargo.toml
[INFO] validating manifest of git repo https://github.com/ilyalabun/raft_vibe on toolchain 1.94.0
[INFO] running `Command { std: CARGO_HOME="/workspace/cargo-home" RUSTUP_HOME="/workspace/rustup-home" "/workspace/cargo-home/bin/cargo" "+1.94.0" "metadata" "--manifest-path" "Cargo.toml" "--no-deps", kill_on_drop: false }`
[INFO] crate git repo https://github.com/ilyalabun/raft_vibe already has a lockfile, it will not be regenerated
[INFO] running `Command { std: CARGO_HOME="/workspace/cargo-home" RUSTUP_HOME="/workspace/rustup-home" "/workspace/cargo-home/bin/cargo" "+1.94.0" "fetch" "--manifest-path" "Cargo.toml", kill_on_drop: false }`
[INFO] [stderr]     Blocking waiting for file lock on package cache
[INFO] running `Command { std: "docker" "create" "-v" "/var/lib/crater-agent-workspace/builds/worker-7-tc1/target:/opt/rustwide/target:rw,Z" "-v" "/var/lib/crater-agent-workspace/builds/worker-7-tc1/source:/opt/rustwide/workdir:ro,Z" "-v" "/var/lib/crater-agent-workspace/cargo-home:/opt/rustwide/cargo-home:ro,Z" "-v" "/var/lib/crater-agent-workspace/rustup-home:/opt/rustwide/rustup-home:ro,Z" "-e" "SOURCE_DIR=/opt/rustwide/workdir" "-e" "CARGO_TARGET_DIR=/opt/rustwide/target" "-e" "CARGO_HOME=/opt/rustwide/cargo-home" "-e" "RUSTUP_HOME=/opt/rustwide/rustup-home" "-w" "/opt/rustwide/workdir" "-m" "1610612736" "--user" "0:0" "--network" "none" "ghcr.io/rust-lang/crates-build-env/linux@sha256:d429b63d4308055ea97f60fb1d3dfca48854a00942f1bd2ad806beaf015945ec" "/opt/rustwide/cargo-home/bin/cargo" "+1.94.0" "metadata" "--no-deps" "--format-version=1", kill_on_drop: false }`
[INFO] [stdout] 2f6fd87281c546b695419837e9f131681385c188cc9b174949bb507b89324204
[INFO] running `Command { std: "docker" "start" "-a" "2f6fd87281c546b695419837e9f131681385c188cc9b174949bb507b89324204", kill_on_drop: false }`
[INFO] running `Command { std: "docker" "inspect" "2f6fd87281c546b695419837e9f131681385c188cc9b174949bb507b89324204", kill_on_drop: false }`
[INFO] running `Command { std: "docker" "rm" "-f" "2f6fd87281c546b695419837e9f131681385c188cc9b174949bb507b89324204", kill_on_drop: false }`
[INFO] [stdout] 2f6fd87281c546b695419837e9f131681385c188cc9b174949bb507b89324204
[INFO] running `Command { std: "docker" "create" "-v" "/var/lib/crater-agent-workspace/builds/worker-7-tc1/target:/opt/rustwide/target:rw,Z" "-v" "/var/lib/crater-agent-workspace/builds/worker-7-tc1/source:/opt/rustwide/workdir:ro,Z" "-v" "/var/lib/crater-agent-workspace/cargo-home:/opt/rustwide/cargo-home:ro,Z" "-v" "/var/lib/crater-agent-workspace/rustup-home:/opt/rustwide/rustup-home:ro,Z" "-e" "SOURCE_DIR=/opt/rustwide/workdir" "-e" "CARGO_TARGET_DIR=/opt/rustwide/target" "-e" "CARGO_INCREMENTAL=0" "-e" "RUST_BACKTRACE=full" "-e" "RUSTFLAGS=--cap-lints=warn" "-e" "RUSTDOCFLAGS=--cap-lints=warn" "-e" "CARGO_HOME=/opt/rustwide/cargo-home" "-e" "RUSTUP_HOME=/opt/rustwide/rustup-home" "-w" "/opt/rustwide/workdir" "-m" "1610612736" "--user" "0:0" "--network" "none" "ghcr.io/rust-lang/crates-build-env/linux@sha256:d429b63d4308055ea97f60fb1d3dfca48854a00942f1bd2ad806beaf015945ec" "/opt/rustwide/cargo-home/bin/cargo" "+1.94.0" "build" "--frozen" "--message-format=json", kill_on_drop: false }`
[INFO] [stdout] 6033f84da3aabf3f5af78e5ec7e035ae03e1ca24d87c0452e2bf4b57d9a7f591
[INFO] running `Command { std: "docker" "start" "-a" "6033f84da3aabf3f5af78e5ec7e035ae03e1ca24d87c0452e2bf4b57d9a7f591", kill_on_drop: false }`
[INFO] [stderr]    Compiling bytes v1.11.0
[INFO] [stderr]    Compiling futures-sink v0.3.31
[INFO] [stderr]    Compiling futures-io v0.3.31
[INFO] [stderr]    Compiling find-msvc-tools v0.1.8
[INFO] [stderr]    Compiling smallvec v1.15.1
[INFO] [stderr]    Compiling log v0.4.29
[INFO] [stderr]    Compiling serde_core v1.0.228
[INFO] [stderr]    Compiling syn v2.0.111
[INFO] [stderr]    Compiling getrandom v0.2.17
[INFO] [stderr]    Compiling sync_wrapper v1.0.2
[INFO] [stderr]    Compiling zerocopy v0.8.31
[INFO] [stderr]    Compiling getrandom v0.3.4
[INFO] [stderr]    Compiling rustls v0.23.36
[INFO] [stderr]    Compiling zmij v1.0.16
[INFO] [stderr]    Compiling ipnet v2.11.0
[INFO] [stderr]    Compiling ryu v1.0.22
[INFO] [stderr]    Compiling futures-channel v0.3.31
[INFO] [stderr]    Compiling tracing v0.1.44
[INFO] [stderr]    Compiling webpki-roots v1.0.5
[INFO] [stderr]    Compiling iri-string v0.7.10
[INFO] [stderr]    Compiling bitflags v2.10.0
[INFO] [stderr]    Compiling cc v1.2.54
[INFO] [stderr]    Compiling http v1.4.0
[INFO] [stderr]    Compiling rand_core v0.9.3
[INFO] [stderr]    Compiling http-body v1.0.1
[INFO] [stderr]    Compiling http-body-util v0.1.3
[INFO] [stderr]    Compiling ring v0.17.14
[INFO] [stderr]    Compiling ppv-lite86 v0.2.21
[INFO] [stderr]    Compiling rand_chacha v0.9.0
[INFO] [stderr]    Compiling rand v0.9.2
[INFO] [stderr]    Compiling serde_json v1.0.149
[INFO] [stderr]    Compiling serde_path_to_error v0.1.20
[INFO] [stderr]    Compiling synstructure v0.13.2
[INFO] [stderr]    Compiling zerofrom-derive v0.1.6
[INFO] [stderr]    Compiling yoke-derive v0.8.1
[INFO] [stderr]    Compiling zerovec-derive v0.11.2
[INFO] [stderr]    Compiling displaydoc v0.2.5
[INFO] [stderr]    Compiling futures-macro v0.3.31
[INFO] [stderr]    Compiling tokio-macros v2.6.0
[INFO] [stderr]    Compiling serde_derive v1.0.228
[INFO] [stderr]    Compiling async-trait v0.1.89
[INFO] [stderr]    Compiling rustls-webpki v0.103.9
[INFO] [stderr]    Compiling tokio v1.48.0
[INFO] [stderr]    Compiling zerofrom v0.1.6
[INFO] [stderr]    Compiling futures-util v0.3.31
[INFO] [stderr]    Compiling yoke v0.8.1
[INFO] [stderr]    Compiling zerotrie v0.2.3
[INFO] [stderr]    Compiling zerovec v0.11.5
[INFO] [stderr]    Compiling tinystr v0.8.2
[INFO] [stderr]    Compiling potential_utf v0.1.4
[INFO] [stderr]    Compiling icu_collections v2.1.1
[INFO] [stderr]    Compiling icu_locale_core v2.1.1
[INFO] [stderr]    Compiling serde v1.0.228
[INFO] [stderr]    Compiling serde_urlencoded v0.7.1
[INFO] [stderr]    Compiling futures-executor v0.3.31
[INFO] [stderr]    Compiling axum-core v0.4.5
[INFO] [stderr]    Compiling futures v0.3.31
[INFO] [stderr]    Compiling icu_provider v2.1.1
[INFO] [stderr]    Compiling icu_properties v2.1.2
[INFO] [stderr]    Compiling icu_normalizer v2.1.1
[INFO] [stderr]    Compiling idna_adapter v1.2.1
[INFO] [stderr]    Compiling idna v1.1.0
[INFO] [stderr]    Compiling url v2.5.8
[INFO] [stderr]    Compiling hyper v1.8.1
[INFO] [stderr]    Compiling tower v0.5.3
[INFO] [stderr]    Compiling tokio-rustls v0.26.4
[INFO] [stderr]    Compiling tower-http v0.6.8
[INFO] [stderr]    Compiling hyper-util v0.1.19
[INFO] [stderr]    Compiling hyper-rustls v0.27.7
[INFO] [stderr]    Compiling axum v0.7.9
[INFO] [stderr]    Compiling reqwest v0.12.28
[INFO] [stderr]    Compiling raft_vibe v0.1.0 (/opt/rustwide/workdir)
[INFO] [stderr]     Finished `dev` profile [unoptimized + debuginfo] target(s) in 51.50s
[INFO] running `Command { std: "docker" "inspect" "6033f84da3aabf3f5af78e5ec7e035ae03e1ca24d87c0452e2bf4b57d9a7f591", kill_on_drop: false }`
[INFO] running `Command { std: "docker" "rm" "-f" "6033f84da3aabf3f5af78e5ec7e035ae03e1ca24d87c0452e2bf4b57d9a7f591", kill_on_drop: false }`
[INFO] [stdout] 6033f84da3aabf3f5af78e5ec7e035ae03e1ca24d87c0452e2bf4b57d9a7f591
[INFO] running `Command { std: "docker" "create" "-v" "/var/lib/crater-agent-workspace/builds/worker-7-tc1/target:/opt/rustwide/target:rw,Z" "-v" "/var/lib/crater-agent-workspace/builds/worker-7-tc1/source:/opt/rustwide/workdir:ro,Z" "-v" "/var/lib/crater-agent-workspace/cargo-home:/opt/rustwide/cargo-home:ro,Z" "-v" "/var/lib/crater-agent-workspace/rustup-home:/opt/rustwide/rustup-home:ro,Z" "-e" "SOURCE_DIR=/opt/rustwide/workdir" "-e" "CARGO_TARGET_DIR=/opt/rustwide/target" "-e" "CARGO_INCREMENTAL=0" "-e" "RUST_BACKTRACE=full" "-e" "RUSTFLAGS=--cap-lints=warn" "-e" "RUSTDOCFLAGS=--cap-lints=warn" "-e" "CARGO_HOME=/opt/rustwide/cargo-home" "-e" "RUSTUP_HOME=/opt/rustwide/rustup-home" "-w" "/opt/rustwide/workdir" "-m" "1610612736" "--user" "0:0" "--network" "none" "ghcr.io/rust-lang/crates-build-env/linux@sha256:d429b63d4308055ea97f60fb1d3dfca48854a00942f1bd2ad806beaf015945ec" "/opt/rustwide/cargo-home/bin/cargo" "+1.94.0" "test" "--frozen" "--no-run" "--message-format=json", kill_on_drop: false }`
[INFO] [stdout] 11de4fb8b9ea5f929941829c83d162e2fa9d1928754bab034fb53c3efe774d6a
[INFO] running `Command { std: "docker" "start" "-a" "11de4fb8b9ea5f929941829c83d162e2fa9d1928754bab034fb53c3efe774d6a", kill_on_drop: false }`
[INFO] [stderr]    Compiling signal-hook-registry v1.4.7
[INFO] [stderr]    Compiling rustix v1.1.3
[INFO] [stderr]    Compiling linux-raw-sys v0.11.0
[INFO] [stderr]    Compiling parking_lot_core v0.9.12
[INFO] [stderr]    Compiling tokio v1.48.0
[INFO] [stderr]    Compiling parking_lot v0.12.5
[INFO] [stderr]    Compiling tempfile v3.24.0
[INFO] [stderr]    Compiling hyper v1.8.1
[INFO] [stderr]    Compiling tower v0.5.3
[INFO] [stderr]    Compiling tokio-rustls v0.26.4
[INFO] [stderr]    Compiling tower-http v0.6.8
[INFO] [stderr]    Compiling hyper-util v0.1.19
[INFO] [stderr]    Compiling hyper-rustls v0.27.7
[INFO] [stderr]    Compiling axum v0.7.9
[INFO] [stderr]    Compiling reqwest v0.12.28
[INFO] [stderr]    Compiling chaos-test v0.1.0 (/opt/rustwide/workdir/chaos-test)
[INFO] [stderr]    Compiling raft_vibe v0.1.0 (/opt/rustwide/workdir)
[INFO] [stderr]     Finished `test` profile [unoptimized + debuginfo] target(s) in 40.96s
[INFO] running `Command { std: "docker" "inspect" "11de4fb8b9ea5f929941829c83d162e2fa9d1928754bab034fb53c3efe774d6a", kill_on_drop: false }`
[INFO] running `Command { std: "docker" "rm" "-f" "11de4fb8b9ea5f929941829c83d162e2fa9d1928754bab034fb53c3efe774d6a", kill_on_drop: false }`
[INFO] [stdout] 11de4fb8b9ea5f929941829c83d162e2fa9d1928754bab034fb53c3efe774d6a
[INFO] running `Command { std: "docker" "create" "-v" "/var/lib/crater-agent-workspace/builds/worker-7-tc1/target:/opt/rustwide/target:rw,Z" "-v" "/var/lib/crater-agent-workspace/builds/worker-7-tc1/source:/opt/rustwide/workdir:ro,Z" "-v" "/var/lib/crater-agent-workspace/cargo-home:/opt/rustwide/cargo-home:ro,Z" "-v" "/var/lib/crater-agent-workspace/rustup-home:/opt/rustwide/rustup-home:ro,Z" "-e" "SOURCE_DIR=/opt/rustwide/workdir" "-e" "CARGO_TARGET_DIR=/opt/rustwide/target" "-e" "CARGO_INCREMENTAL=0" "-e" "RUST_BACKTRACE=full" "-e" "RUSTFLAGS=--cap-lints=warn" "-e" "RUSTDOCFLAGS=--cap-lints=warn" "-e" "CARGO_HOME=/opt/rustwide/cargo-home" "-e" "RUSTUP_HOME=/opt/rustwide/rustup-home" "-w" "/opt/rustwide/workdir" "-m" "1610612736" "--user" "0:0" "--network" "none" "ghcr.io/rust-lang/crates-build-env/linux@sha256:d429b63d4308055ea97f60fb1d3dfca48854a00942f1bd2ad806beaf015945ec" "/opt/rustwide/cargo-home/bin/cargo" "+1.94.0" "test" "--frozen", kill_on_drop: false }`
[INFO] [stdout] c017efbab1fe5117a8f95fcff13217933b1986f16ec55bdadfa07ac32a32290d
[INFO] running `Command { std: "docker" "start" "-a" "c017efbab1fe5117a8f95fcff13217933b1986f16ec55bdadfa07ac32a32290d", kill_on_drop: false }`
[INFO] [stderr]     Finished `test` profile [unoptimized + debuginfo] target(s) in 0.27s
[INFO] [stderr]      Running unittests src/lib.rs (/opt/rustwide/target/debug/deps/raft_vibe-5134618d223841cd)
[INFO] [stdout] 
[INFO] [stdout] running 192 tests
[INFO] [stdout] test api::client_http::tests::test_leader_endpoint_leader ... ok
[INFO] [stdout] test api::client_http::tests::test_status_endpoint ... ok
[INFO] [stdout] test api::client_http::tests::test_submit_as_leader_full ... ok
[INFO] [stdout] test api::client_http::tests::test_submit_not_leader_full ... ok
[INFO] [stdout] test core::raft_core::tests::test_append_entries_idempotent ... ok
[INFO] [stdout] test api::client_http::tests::test_leader_endpoint_follower ... ok
[INFO] [stdout] test core::raft_core::tests::test_append_entries_fails_prev_log_index_too_high ... ok
[INFO] [stdout] test core::raft_core::tests::test_append_entries_truncates_conflicting_entries ... ok
[INFO] [stdout] test core::raft_core::tests::test_append_entries_stale_term_no_reset ... ok
[INFO] [stdout] test core::raft_core::tests::test_append_entries_with_entries_already_present ... ok
[INFO] [stdout] test core::raft_core::tests::test_apply_committed_entries_does_nothing_when_up_to_date ... ok
[INFO] [stdout] test core::raft_core::tests::test_append_entries_fails_prev_log_term_mismatch ... ok
[INFO] [stdout] test core::raft_core::tests::test_automatic_snapshot_on_follower ... ok
[INFO] [stdout] test core::raft_core::tests::test_automatic_snapshot_triggering ... ok
[INFO] [stdout] test core::raft_core::tests::test_become_leader_initializes_next_index ... ok
[INFO] [stdout] test core::raft_core::tests::test_candidate_steps_down_on_append_entries_from_new_leader ... ok
[INFO] [stdout] test core::raft_core::tests::test_candidate_resets_votes_on_new_election ... ok
[INFO] [stdout] test core::raft_core::tests::test_automatic_snapshot_disabled_when_threshold_is_zero ... ok
[INFO] [stdout] test core::raft_core::tests::test_conflict_hint_empty_log ... ok
[INFO] [stdout] test core::raft_core::tests::test_apply_committed_entries_updates_last_applied ... ok
[INFO] [stdout] test core::raft_core::tests::test_commit_index_advances_on_append_entries ... ok
[INFO] [stdout] test core::raft_core::tests::test_conflict_hint_gap_log ... ok
[INFO] [stdout] test core::raft_core::tests::test_candidate_steps_down_on_higher_term_request_vote ... ok
[INFO] [stdout] test core::raft_core::tests::test_apply_entries_after_snapshot_restoration ... ok
[INFO] [stdout] test core::raft_core::tests::test_conflict_hint_log_too_short ... ok
[INFO] [stdout] test core::raft_core::tests::test_conflict_hint_term_mismatch ... ok
[INFO] [stdout] test core::raft_core::tests::test_conflict_hint_term_mismatch_leader_has_term ... ok
[INFO] [stdout] test core::raft_core::tests::test_duplicate_vote_from_same_peer_ignored ... ok
[INFO] [stdout] test core::raft_core::tests::test_election_needs_majority_in_5_node_cluster ... ok
[INFO] [stdout] test core::raft_core::tests::test_election ... ok
[INFO] [stdout] test core::raft_core::tests::test_empty_append_entries_still_updates_commit_index ... ok
[INFO] [stdout] test core::raft_core::tests::test_entries_applied_to_state_machine ... ok
[INFO] [stdout] test core::raft_core::tests::test_append_entries ... ok
[INFO] [stdout] test core::raft_core::tests::test_commit_index_limited_by_log_length ... ok
[INFO] [stdout] test core::raft_core::tests::test_commit_multiple_entries_at_once ... ok
[INFO] [stdout] test core::raft_core::tests::test_entry_not_committed_without_majority ... ok
[INFO] [stdout] test core::raft_core::tests::test_follower_catches_up_incrementally ... ok
[INFO] [stdout] test core::raft_core::tests::test_follower_updates_term_on_higher_term_append_entries ... ok
[INFO] [stdout] test core::raft_core::tests::test_follower_catches_up_multiple_entries ... ok
[INFO] [stdout] test core::raft_core::tests::test_follower_updates_term_on_higher_term_request_vote ... ok
[INFO] [stdout] test core::raft_core::tests::test_four_node_cluster_majority ... ok
[INFO] [stdout] test core::raft_core::tests::test_get_log_entry_with_snapshot ... ok
[INFO] [stdout] test core::raft_core::tests::test_handle_install_snapshot_stale_term ... ok
[INFO] [stdout] test core::raft_core::tests::test_follower_with_gap_rejects_append ... ok
[INFO] [stdout] test core::raft_core::tests::test_handle_install_snapshot_success ... ok
[INFO] [stdout] test core::raft_core::tests::test_heartbeat_resets_election_timeout ... ok
[INFO] [stdout] test core::raft_core::tests::test_ignore_stale_vote_response_from_old_term ... ok
[INFO] [stdout] test core::raft_core::tests::test_last_log_index_with_snapshot ... ok
[INFO] [stdout] test core::raft_core::tests::test_leader_cannot_commit_previous_term_entries_directly ... ok
[INFO] [stdout] test core::raft_core::tests::test_leader_appends_multiple_entries_sequentially ... ok
[INFO] [stdout] test core::raft_core::tests::test_handle_install_snapshot_updates_commit_and_applied ... ok
[INFO] [stdout] test core::raft_core::tests::test_full_catch_up_after_wipe ... ok
[INFO] [stdout] test core::raft_core::tests::test_follower_with_extra_uncommitted_entries_gets_truncated ... ok
[INFO] [stdout] test core::raft_core::tests::test_leader_steps_down_on_higher_term_in_append_response ... ok
[INFO] [stdout] test core::raft_core::tests::test_leader_entry_has_current_term ... ok
[INFO] [stdout] test core::raft_core::tests::test_match_index_does_not_decrease ... ok
[INFO] [stdout] test core::raft_core::tests::test_leader_steps_down_on_higher_term_in_vote_response ... ok
[INFO] [stdout] test core::raft_core::tests::test_multiple_elections_term_increases ... ok
[INFO] [stdout] test core::raft_core::tests::test_match_index_updates_on_successful_append ... ok
[INFO] [stdout] test core::raft_core::tests::test_handle_install_snapshot_discards_old_log_entries ... ok
[INFO] [stdout] test core::raft_core::tests::test_multiple_snapshots_work ... ok
[INFO] [stdout] test core::raft_core::tests::test_next_index_decrements_on_failed_append ... ok
[INFO] [stdout] test core::raft_core::tests::test_entry_committed_with_majority ... ok
[INFO] [stdout] test core::raft_core::tests::test_node_restarts_with_saved_log ... ok
[INFO] [stdout] test core::raft_core::tests::test_next_index_does_not_go_below_1 ... ok
[INFO] [stdout] test core::raft_core::tests::test_node_restarts_with_full_state ... ok
[INFO] [stdout] test core::raft_core::tests::test_node_restarts_with_saved_term ... ok
[INFO] [stdout] test core::raft_core::tests::test_node_restarts_with_saved_voted_for ... ok
[INFO] [stdout] test core::raft_core::tests::test_previous_term_entries_committed_indirectly ... ok
[INFO] [stdout] test core::raft_core::tests::test_restarted_node_accepts_append_entries ... ok
[INFO] [stdout] test core::raft_core::tests::test_restarted_node_can_continue_election ... ok
[INFO] [stdout] test core::raft_core::tests::test_single_node_cluster_immediate_leader ... ok
[INFO] [stdout] test core::raft_core::tests::test_replication_after_compaction ... ok
[INFO] [stdout] test core::raft_core::tests::test_stale_append_response_does_not_affect_commit ... ok
[INFO] [stdout] test core::raft_core::tests::test_non_leader_cannot_append_entries ... ok
[INFO] [stdout] test core::raft_core::tests::test_state_persists_across_operations ... ok
[INFO] [stdout] test core::raft_core::tests::test_request_vote ... ok
[INFO] [stdout] test core::raft_core::tests::test_snapshot_restores_state_machine_on_startup ... ok
[INFO] [stdout] test core::raft_core::tests::test_take_snapshot_already_snapshotted ... ok
[INFO] [stdout] test core::raft_core::tests::test_restarted_leader_must_reestablish ... ok
[INFO] [stdout] test core::raft_core::tests::test_handle_install_snapshot_old_snapshot ... ok
[INFO] [stdout] test core::raft_core::tests::test_leader_loses_leadership_on_higher_term_response ... ok
[INFO] [stdout] test core::raft_core::tests::test_new_node ... ok
[INFO] [stdout] test core::raft_core::tests::test_take_snapshot_no_entries_applied ... ok
[INFO] [stdout] test core::raft_node::tests::test_election ... ok
[INFO] [stdout] test core::raft_core::tests::test_two_node_cluster_becomes_leader_with_peer_vote ... ok
[INFO] [stdout] test core::raft_node::tests::test_election_with_one_peer_timeout ... ok
[INFO] [stdout] test core::raft_core::tests::test_take_snapshot_basic ... ok
[INFO] [stdout] test core::raft_core::tests::test_vote_granted_candidate_log_has_higher_term ... ok
[INFO] [stdout] test core::raft_core::tests::test_vote_denied_candidate_log_is_shorter ... ok
[INFO] [stdout] test core::raft_core::tests::test_vote_granted_to_same_candidate_again ... ok
[INFO] [stdout] test core::raft_core::tests::test_vote_request_resets_voted_for_on_new_term ... ok
[INFO] [stdout] test core::raft_node::tests::test_follower_catches_up_after_snapshot ... ok
[INFO] [stdout] test core::raft_node::tests::test_heartbeat ... ok
[INFO] [stdout] test core::raft_node::tests::test_all_peers_timeout_election_fails ... ok
[INFO] [stdout] test core::raft_core::tests::test_vote_denied_candidate_has_lower_term ... ok
[INFO] [stdout] test core::raft_core::tests::test_vote_denied_candidate_log_has_older_term ... ok
[INFO] [stdout] test core::raft_core::tests::test_two_node_cluster_needs_both_votes ... ok
[INFO] [stdout] test core::raft_node::tests::test_heartbeat_sends_snapshot_to_lagging_follower ... ok
[INFO] [stdout] test core::raft_node::tests::test_heartbeat_catches_up_followers ... ok
[INFO] [stdout] test core::raft_node::tests::test_heartbeat_handles_snapshot_boundary ... ok
[INFO] [stdout] test core::raft_core::tests::test_vote_denied_already_voted_for_another ... ok
[INFO] [stdout] test core::raft_node::tests::test_heartbeat_with_timeout ... ok
[INFO] [stdout] test core::raft_node::tests::test_mixed_snapshot_and_log_replication ... ok
[INFO] [stdout] test core::raft_server::tests::test_election_via_server ... ok
[INFO] [stdout] test core::raft_node::tests::test_multiple_entries_replicated ... ok
[INFO] [stdout] test core::raft_node::tests::test_replicate_to_peers_after_snapshot ... ok
[INFO] [stdout] test core::raft_node::tests::test_replication_with_one_peer_timeout ... ok
[INFO] [stdout] test core::raft_server::tests::test_candidate_rejects_commands ... ok
[INFO] [stdout] test core::raft_server::tests::test_election_timeout_triggers_election ... ok
[INFO] [stdout] test core::raft_server::tests::test_client_command_flow ... ok
[INFO] [stdout] test core::raft_node::tests::test_replication ... ok
[INFO] [stdout] test core::raft_server::tests::test_entry_applied_when_quorum_reached ... ok
[INFO] [stdout] test core::raft_server::tests::test_fresh_node_catches_up_from_empty ... ok
[INFO] [stdout] test core::raft_server::tests::test_follower_applies_entry_on_commit_notification ... ok
[INFO] [stdout] test core::raft_server::tests::test_followers_snapshot_before_leader ... ok
[INFO] [stdout] test core::raft_server::tests::test_entry_not_applied_without_quorum ... ok
[INFO] [stdout] test core::raft_server::tests::test_command_fails_when_leader_loses_leadership ... ok
[INFO] [stdout] test core::raft_server::tests::test_leader_isolated_cannot_commit ... ok
[INFO] [stdout] test core::raft_server::tests::test_leader_failover ... ok
[INFO] [stdout] test core::raft_server::tests::test_invalid_command_rejected_before_log_append ... ok
[INFO] [stdout] test core::raft_server::tests::test_read_index_as_leader ... ok
[INFO] [stdout] test core::raft_server::tests::test_multiple_commands_in_sequence ... ok
[INFO] [stdout] test core::raft_server::tests::test_leader_snapshots_before_followers ... ok
[INFO] [stdout] test core::raft_server::tests::test_read_index_with_pending_entries ... ok
[INFO] [stdout] test core::raft_server::tests::test_replication_via_server ... ok
[INFO] [stdout] test core::raft_server::tests::test_read_index_leader_steps_down ... ok
[INFO] [stdout] test core::raft_server::tests::test_read_index_returns_last_applied ... ok
[INFO] [stdout] test core::raft_server::tests::test_partitioned_node_rejoins ... ok
[INFO] [stdout] test core::raft_server::tests::test_read_index_not_leader ... ok
[INFO] [stdout] test core::raft_server::tests::test_replication_continues_after_snapshot ... ok
[INFO] [stdout] test core::raft_server::tests::test_no_election_before_timeout ... ok
[INFO] [stdout] test core::raft_server::tests::test_server_not_leader ... ok
[INFO] [stdout] test core::raft_server::tests::test_read_index_timeout_no_majority ... ok
[INFO] [stdout] test core::raft_server::tests::test_split_brain_prevention ... ok
[INFO] [stdout] test state_machine::kv::tests::test_delete ... ok
[INFO] [stdout] test core::raft_server::tests::test_stale_leader_steps_down ... ok
[INFO] [stdout] test core::raft_server::tests::test_state_machine_recovery_after_reelection ... ok
[INFO] [stdout] test state_machine::kv::tests::test_overwrite ... ok
[INFO] [stdout] test state_machine::kv::tests::test_snapshot_and_restore ... ok
[INFO] [stdout] test state_machine::kv::tests::test_snapshot_empty_store ... ok
[INFO] [stdout] test core::raft_server::tests::test_uncommitted_entries_committed_via_noop ... ok
[INFO] [stdout] test state_machine::kv::tests::test_get_not_found ... ok
[INFO] [stdout] test core::raft_server::tests::test_snapshot_with_heartbeat_replication ... ok
[INFO] [stdout] test state_machine::kv::tests::test_delete_not_found ... ok
[INFO] [stdout] test state_machine::kv::tests::test_restore_overwrites_existing_data ... ok
[INFO] [stdout] test state_machine::kv::tests::test_restore_invalid_data ... ok
[INFO] [stdout] test state_machine::kv::tests::test_unknown_command ... ok
[INFO] [stdout] test state_machine::kv::tests::test_validate_delete_command ... ok
[INFO] [stdout] test state_machine::kv::tests::test_validate_noop_command ... ok
[INFO] [stdout] test state_machine::kv::tests::test_value_with_spaces ... ok
[INFO] [stdout] test state_machine::kv::tests::test_validate_invalid_command ... ok
[INFO] [stdout] test state_machine::kv::tests::test_validate_set_command ... ok
[INFO] [stdout] test storage::file::tests::test_crc32_basic ... ok
[INFO] [stdout] test api::client_http::tests::test_submit_invalid_command_returns_bad_request ... ok
[INFO] [stdout] test storage::file::tests::test_detects_corrupted_log_entry ... ok
[INFO] [stdout] test storage::file::tests::test_file_storage_log ... ok
[INFO] [stdout] test storage::file::tests::test_detects_corrupted_voted_for ... ok
[INFO] [stdout] test storage::memory::tests::test_memory_storage_compact_log ... ok
[INFO] [stdout] test storage::memory::tests::test_memory_storage_log_append ... ok
[INFO] [stdout] test core::raft_core::tests::test_snapshot_compacts_file_storage ... ok
[INFO] [stdout] test storage::file::tests::test_detects_corrupted_term ... ok
[INFO] [stdout] test storage::memory::tests::test_memory_storage_log_truncate ... ok
[INFO] [stdout] test storage::memory::tests::test_memory_storage_term ... ok
[INFO] [stdout] test storage::memory::tests::test_memory_storage_snapshot ... ok
[INFO] [stdout] test storage::file::tests::test_file_storage_snapshot ... ok
[INFO] [stdout] test storage::memory::tests::test_memory_storage_truncate_all ... ok
[INFO] [stdout] test storage::memory::tests::test_memory_storage_truncate_beyond_log ... ok
[INFO] [stdout] test storage::file::tests::test_file_storage_term ... ok
[INFO] [stdout] test transport::http::tests::test_http_node_not_found ... ok
[INFO] [stdout] test transport::inmemory::tests::test_append_entries_timeout ... ok
[INFO] [stdout] test storage::file::tests::test_detects_corrupted_snapshot ... ok
[INFO] [stdout] test transport::inmemory::tests::test_install_snapshot ... ok
[INFO] [stdout] test storage::file::tests::test_file_storage_persistence_across_instances ... ok
[INFO] [stdout] test storage::file::tests::test_file_storage_voted_for ... ok
[INFO] [stdout] test storage::file::tests::test_file_storage_snapshot_persistence ... ok
[INFO] [stdout] test storage::file::tests::test_file_storage_log_compact ... ok
[INFO] [stdout] test transport::inmemory::tests::test_mixed_responses_and_timeouts ... ok
[INFO] [stdout] test transport::inmemory::tests::test_node_not_found ... ok
[INFO] [stdout] test storage::file::tests::test_file_storage_log_truncate ... ok
[INFO] [stdout] test transport::http::tests::test_http_timeout ... ok
[INFO] [stdout] test transport::inmemory::tests::test_request_vote ... ok
[INFO] [stdout] test transport::inmemory::tests::test_append_entries ... ok
[INFO] [stdout] test transport::inmemory::tests::test_election_via_transport ... ok
[INFO] [stdout] test core::raft_server::tests::test_valid_command_accepted_after_validation ... ok
[INFO] [stdout] test transport::inmemory::tests::test_request_vote_timeout ... ok
[INFO] [stdout] test transport::http::tests::test_http_request_vote ... ok
[INFO] [stdout] test core::raft_core::tests::test_election_lost_all_denied ... ok
[INFO] [stdout] test state_machine::kv::tests::test_set_and_get ... ok
[INFO] [stdout] test storage::memory::tests::test_memory_storage_voted_for ... ok
[INFO] [stdout] test transport::http::tests::test_http_append_entries ... ok
[INFO] [stderr]      Running unittests src/bin/server.rs (/opt/rustwide/target/debug/deps/raft_server-cd799e70c0fa7b17)
[INFO] [stdout] test core::raft_server::tests::test_snapshot_with_file_storage ... ok
[INFO] [stderr]      Running tests/chaos_linearizability_test.rs (/opt/rustwide/target/debug/deps/chaos_linearizability_test-eeb0bbac5e32218e)
[INFO] [stdout] 
[INFO] [stderr]      Running tests/http_cluster.rs (/opt/rustwide/target/debug/deps/http_cluster-5814107b138df4d9)
[INFO] [stdout] test result: ok. 192 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.14s
[INFO] [stdout] 
[INFO] [stdout] 
[INFO] [stdout] running 0 tests
[INFO] [stdout] 
[INFO] [stdout] test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s
[INFO] [stdout] 
[INFO] [stdout] 
[INFO] [stdout] running 13 tests
[INFO] [stdout] test test_asymmetric_partition ... ignored
[INFO] [stdout] test test_follower_kill ... ignored
[INFO] [stdout] test test_leader_kill ... ignored
[INFO] [stdout] test test_leader_kill_loop ... ignored
[INFO] [stdout] test test_leader_partition ... ignored
[INFO] [stdout] test test_majority_crash_and_recover ... ignored
[INFO] [stdout] test test_minority_partition ... ignored
[INFO] [stdout] test test_packet_loss ... ignored
[INFO] [stdout] test test_partition_heal ... ignored
[INFO] [stdout] test test_rolling_restart ... ignored
[INFO] [stdout] test test_slow_network ... ignored
[INFO] [stdout] test test_storage_wipe ... ignored
[INFO] [stdout] test test_two_followers_kill ... ignored
[INFO] [stdout] 
[INFO] [stdout] test result: ok. 0 passed; 0 failed; 13 ignored; 0 measured; 0 filtered out; finished in 0.00s
[INFO] [stdout] 
[INFO] [stdout] 
[INFO] [stdout] running 13 tests
[INFO] [stdout] test test_cluster_elects_leader ... ok
[INFO] [stdout] test test_linearizable_read_after_writes ... ok
[INFO] [stdout] test test_linearizable_read_not_leader ... ok
[INFO] [stdout] test test_linearizable_read_missing_key ... ok
[INFO] [stdout] test test_linearizable_read_as_leader ... ok
[INFO] [stdout] test test_leader_failover ... ok
[INFO] [stdout] test test_follower_redirect ... ok
[INFO] [stdout] test test_concurrent_writes_to_same_key ... FAILED
[INFO] [stdout] test test_concurrent_client_commands ... ok
[INFO] [stdout] test test_concurrent_commands_during_leader_failover ... ok
[INFO] [stdout] test test_cluster_consistency ... ok
[INFO] [stdout] test test_concurrent_reads_and_writes ... ok
[INFO] [stdout] test test_rapid_fire_http_requests ... ok
[INFO] [stdout] 
[INFO] [stderr] error: test failed, to rerun pass `--test http_cluster`
[INFO] [stdout] failures:
[INFO] [stdout] 
[INFO] [stdout] ---- test_concurrent_writes_to_same_key stdout ----
[INFO] [stdout] [NODE 1] Became CANDIDATE for term 1
[INFO] [stdout] [NODE 1] Became LEADER for term 1
[INFO] [stdout] [NODE 1] Appending no-op entry 1
[INFO] [stdout] [NODE 2] Replicated entry 1 (term 1): NOOP
[INFO] [stdout] [NODE 3] Replicated entry 1 (term 1): NOOP
[INFO] [stdout] [NODE 1] Committed entry 1 (replicated to 2/3)
[INFO] [stdout] [NODE 2] Became CANDIDATE for term 2
[INFO] [stdout] [NODE 3] Became CANDIDATE for term 2
[INFO] [stdout] [NODE 1] Stepped down to FOLLOWER (was Leader, saw term 2)
[INFO] [stdout] [NODE 2] Became LEADER for term 2
[INFO] [stdout] [NODE 2] Appending no-op entry 2
[INFO] [stdout] [NODE 3] Replicated entry 2 (term 2): NOOP
[INFO] [stdout] [NODE 1] Replicated entry 2 (term 2): NOOP
[INFO] [stdout] [NODE 2] Committed entry 2 (replicated to 2/3)
[INFO] [stdout] 
[INFO] [stdout] thread 'test_concurrent_writes_to_same_key' (225) panicked at tests/http_cluster.rs:583:5:
[INFO] [stdout] assertion `left == right` failed: All concurrent writes should succeed
[INFO] [stdout]   left: 0
[INFO] [stdout]  right: 50
[INFO] [stdout] stack backtrace:
[INFO] [stdout]    0:     0x612f2d63c9d2 - std::backtrace_rs::backtrace::libunwind::trace::hff358a60abf734fc
[INFO] [stdout]                                at /rustc/4a4ef493e3a1488c6e321570238084b38948f6db/library/std/src/../../backtrace/src/backtrace/libunwind.rs:117:9
[INFO] [stdout]    1:     0x612f2d63c9d2 - std::backtrace_rs::backtrace::trace_unsynchronized::h3b121f916dd95ec6
[INFO] [stdout]                                at /rustc/4a4ef493e3a1488c6e321570238084b38948f6db/library/std/src/../../backtrace/src/backtrace/mod.rs:66:14
[INFO] [stdout]    2:     0x612f2d63c9d2 - std::sys::backtrace::_print_fmt::hde0a62ded68798e9
[INFO] [stdout]                                at /rustc/4a4ef493e3a1488c6e321570238084b38948f6db/library/std/src/sys/backtrace.rs:74:9
[INFO] [stdout]    3:     0x612f2d63c9d2 - <std::sys::backtrace::BacktraceLock::print::DisplayBacktrace as core::fmt::Display>::fmt::h93773fc827e3113d
[INFO] [stdout]                                at /rustc/4a4ef493e3a1488c6e321570238084b38948f6db/library/std/src/sys/backtrace.rs:44:26
[INFO] [stdout]    4:     0x612f2d64e98a - core::fmt::rt::Argument::fmt::h01eff69902dad97f
[INFO] [stdout]                                at /rustc/4a4ef493e3a1488c6e321570238084b38948f6db/library/core/src/fmt/rt.rs:152:76
[INFO] [stdout]    5:     0x612f2d64e98a - core::fmt::write::hed7b5c73d82ecb7c
[INFO] [stdout]    6:     0x612f2d609c26 - std::io::default_write_fmt::h2f696ff5b8bbaa4b
[INFO] [stdout]                                at /rustc/4a4ef493e3a1488c6e321570238084b38948f6db/library/std/src/io/mod.rs:639:11
[INFO] [stdout]    7:     0x612f2d609c26 - std::io::Write::write_fmt::h5e66814db8a9cfce
[INFO] [stdout]                                at /rustc/4a4ef493e3a1488c6e321570238084b38948f6db/library/std/src/io/mod.rs:1994:13
[INFO] [stdout]    8:     0x612f2d61b6a9 - std::sys::backtrace::BacktraceLock::print::h8b1d6fcc5a56d1a3
[INFO] [stdout]                                at /rustc/4a4ef493e3a1488c6e321570238084b38948f6db/library/std/src/sys/backtrace.rs:47:9
[INFO] [stdout]    9:     0x612f2d61b6a9 - std::panicking::default_hook::{{closure}}::h2be84df4f189ae36
[INFO] [stdout]                                at /rustc/4a4ef493e3a1488c6e321570238084b38948f6db/library/std/src/panicking.rs:292:27
[INFO] [stdout]   10:     0x612f2d61b541 - std::panicking::default_hook::hf0ea8939246f43a9
[INFO] [stdout]                                at /rustc/4a4ef493e3a1488c6e321570238084b38948f6db/library/std/src/panicking.rs:316:9
[INFO] [stdout]   11:     0x612f2cf9d57e - <alloc::boxed::Box<F,A> as core::ops::function::Fn<Args>>::call::h02d35e3888986252
[INFO] [stdout]                                at /rustc/4a4ef493e3a1488c6e321570238084b38948f6db/library/alloc/src/boxed.rs:2220:9
[INFO] [stdout]   12:     0x612f2cf9d57e - test::test_main_with_exit_callback::{{closure}}::h939640a27d647173
[INFO] [stdout]                                at /rustc/4a4ef493e3a1488c6e321570238084b38948f6db/library/test/src/lib.rs:145:21
[INFO] [stdout]   13:     0x612f2d61b9d2 - <alloc::boxed::Box<F,A> as core::ops::function::Fn<Args>>::call::hc9df09ff391af3e7
[INFO] [stdout]                                at /rustc/4a4ef493e3a1488c6e321570238084b38948f6db/library/alloc/src/boxed.rs:2220:9
[INFO] [stdout]   14:     0x612f2d61b9d2 - std::panicking::panic_with_hook::hb4bd9ac1123582a0
[INFO] [stdout]                                at /rustc/4a4ef493e3a1488c6e321570238084b38948f6db/library/std/src/panicking.rs:833:13
[INFO] [stdout]   15:     0x612f2d61b768 - std::panicking::panic_handler::{{closure}}::hde00dd15f5637fe2
[INFO] [stdout]                                at /rustc/4a4ef493e3a1488c6e321570238084b38948f6db/library/std/src/panicking.rs:698:13
[INFO] [stdout]   16:     0x612f2d6169c9 - std::sys::backtrace::__rust_end_short_backtrace::hb72197fa777c1785
[INFO] [stdout]                                at /rustc/4a4ef493e3a1488c6e321570238084b38948f6db/library/std/src/sys/backtrace.rs:182:18
[INFO] [stdout]   17:     0x612f2d5fe98d - __rustc[4425a7e20b4c8619]::rust_begin_unwind
[INFO] [stdout]                                at /rustc/4a4ef493e3a1488c6e321570238084b38948f6db/library/std/src/panicking.rs:689:5
[INFO] [stdout]   18:     0x612f2d65745c - core::panicking::panic_fmt::ha59b517dd231f4da
[INFO] [stdout]                                at /rustc/4a4ef493e3a1488c6e321570238084b38948f6db/library/core/src/panicking.rs:80:14
[INFO] [stdout]   19:     0x612f2d657373 - core::panicking::assert_failed_inner::h3c3301979dc81b81
[INFO] [stdout]                                at /rustc/4a4ef493e3a1488c6e321570238084b38948f6db/library/core/src/panicking.rs:434:23
[INFO] [stdout]   20:     0x612f2d6571a3 - core::panicking::assert_failed::h59d9a1fd9655a77e
[INFO] [stdout]                                at /rustc/4a4ef493e3a1488c6e321570238084b38948f6db/library/core/src/panicking.rs:394:5
[INFO] [stdout]   21:     0x612f2ceca949 - http_cluster::test_concurrent_writes_to_same_key::{{closure}}::h4afec8aa754478b4
[INFO] [stdout]                                at /opt/rustwide/workdir/tests/http_cluster.rs:583:5
[INFO] [stdout]   22:     0x612f2cf2f0b2 - <core::pin::Pin<P> as core::future::future::Future>::poll::h6df39a847b953799
[INFO] [stdout]                                at /rustc/4a4ef493e3a1488c6e321570238084b38948f6db/library/core/src/future/future.rs:133:9
[INFO] [stdout]   23:     0x612f2cf2f00d - <core::pin::Pin<P> as core::future::future::Future>::poll::h6c014d1c1b96babc
[INFO] [stdout]                                at /rustc/4a4ef493e3a1488c6e321570238084b38948f6db/library/core/src/future/future.rs:133:9
[INFO] [stdout]   24:     0x612f2cf2317d - tokio::runtime::scheduler::current_thread::CoreGuard::block_on::{{closure}}::{{closure}}::{{closure}}::h947a38db91353ab8
[INFO] [stdout]                                at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.48.0/src/runtime/scheduler/current_thread/mod.rs:742:70
[INFO] [stdout]   25:     0x612f2cf22f1b - tokio::task::coop::with_budget::hfe06af4e30780f41
[INFO] [stdout]                                at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.48.0/src/task/coop/mod.rs:167:5
[INFO] [stdout]   26:     0x612f2cf22f1b - tokio::task::coop::budget::he8851442b3df94e1
[INFO] [stdout]                                at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.48.0/src/task/coop/mod.rs:133:5
[INFO] [stdout]   27:     0x612f2cf22f1b - tokio::runtime::scheduler::current_thread::CoreGuard::block_on::{{closure}}::{{closure}}::h1dc8b289e4847262
[INFO] [stdout]                                at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.48.0/src/runtime/scheduler/current_thread/mod.rs:742:25
[INFO] [stdout]   28:     0x612f2cf204f0 - tokio::runtime::scheduler::current_thread::Context::enter::h727da6a02b84a025
[INFO] [stdout]                                at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.48.0/src/runtime/scheduler/current_thread/mod.rs:432:19
[INFO] [stdout]   29:     0x612f2cf21b5d - tokio::runtime::scheduler::current_thread::CoreGuard::block_on::{{closure}}::h1195d683e70ffbf8
[INFO] [stdout]                                at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.48.0/src/runtime/scheduler/current_thread/mod.rs:741:44
[INFO] [stdout]   30:     0x612f2cf217e4 - tokio::runtime::scheduler::current_thread::CoreGuard::enter::{{closure}}::hc2af3fba26a6cc65
[INFO] [stdout]                                at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.48.0/src/runtime/scheduler/current_thread/mod.rs:829:68
[INFO] [stdout]   31:     0x612f2cf7b50b - tokio::runtime::context::scoped::Scoped<T>::set::h98558c00a9e947c6
[INFO] [stdout]                                at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.48.0/src/runtime/context/scoped.rs:40:9
[INFO] [stdout]   32:     0x612f2cf054e9 - tokio::runtime::context::set_scheduler::{{closure}}::h019dcee98839ac58
[INFO] [stdout]                                at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.48.0/src/runtime/context.rs:176:38
[INFO] [stdout]   33:     0x612f2cf6f812 - std::thread::local::LocalKey<T>::try_with::hdb5568d90c9af336
[INFO] [stdout]                                at /rustc/4a4ef493e3a1488c6e321570238084b38948f6db/library/std/src/thread/local.rs:513:12
[INFO] [stdout]   34:     0x612f2cf6d22e - std::thread::local::LocalKey<T>::with::h3b2afc06cfd4ddb6
[INFO] [stdout]                                at /rustc/4a4ef493e3a1488c6e321570238084b38948f6db/library/std/src/thread/local.rs:477:20
[INFO] [stdout]   35:     0x612f2cf0545d - tokio::runtime::context::set_scheduler::h0807226c75984d14
[INFO] [stdout]                                at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.48.0/src/runtime/context.rs:176:17
[INFO] [stdout]   36:     0x612f2cf21150 - tokio::runtime::scheduler::current_thread::CoreGuard::enter::h2fc88dd655fd0b55
[INFO] [stdout]                                at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.48.0/src/runtime/scheduler/current_thread/mod.rs:829:27
[INFO] [stdout]   37:     0x612f2cf21826 - tokio::runtime::scheduler::current_thread::CoreGuard::block_on::h90b1f5a6bae5fb80
[INFO] [stdout]                                at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.48.0/src/runtime/scheduler/current_thread/mod.rs:729:24
[INFO] [stdout]   38:     0x612f2cf1c920 - tokio::runtime::scheduler::current_thread::CurrentThread::block_on::{{closure}}::hbc42551b25aab737
[INFO] [stdout]                                at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.48.0/src/runtime/scheduler/current_thread/mod.rs:200:33
[INFO] [stdout]   39:     0x612f2cf6a9f3 - tokio::runtime::context::runtime::enter_runtime::h8a9368506e5a3f6d
[INFO] [stdout]                                at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.48.0/src/runtime/context/runtime.rs:65:16
[INFO] [stdout]   40:     0x612f2cf1c631 - tokio::runtime::scheduler::current_thread::CurrentThread::block_on::h2ecddc55111acea2
[INFO] [stdout]                                at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.48.0/src/runtime/scheduler/current_thread/mod.rs:188:9
[INFO] [stdout]   41:     0x612f2ced45d4 - tokio::runtime::runtime::Runtime::block_on_inner::h2fc9adafcd83da35
[INFO] [stdout]                                at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.48.0/src/runtime/runtime.rs:368:52
[INFO] [stdout]   42:     0x612f2ced479c - tokio::runtime::runtime::Runtime::block_on::h8ec4d6b2935573a1
[INFO] [stdout]                                at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.48.0/src/runtime/runtime.rs:342:18
[INFO] [stdout]   43:     0x612f2cec9ef3 - http_cluster::test_concurrent_writes_to_same_key::h4776d4c98ef14e91
[INFO] [stdout]                                at /opt/rustwide/workdir/tests/http_cluster.rs:654:29
[INFO] [stdout]   44:     0x612f2cecc2f7 - http_cluster::test_concurrent_writes_to_same_key::{{closure}}::h4d5defd8fb53526c
[INFO] [stdout]                                at /opt/rustwide/workdir/tests/http_cluster.rs:551:46
[INFO] [stdout]   45:     0x612f2ceac1b6 - core::ops::function::FnOnce::call_once::hcb8ead5ca7a6b50a
[INFO] [stdout]                                at /rustc/4a4ef493e3a1488c6e321570238084b38948f6db/library/core/src/ops/function.rs:250:5
[INFO] [stdout]   46:     0x612f2cf9d33b - core::ops::function::FnOnce::call_once::hddb3cd395c36bfbb
[INFO] [stdout]                                at /rustc/4a4ef493e3a1488c6e321570238084b38948f6db/library/core/src/ops/function.rs:250:5
[INFO] [stdout]   47:     0x612f2cf9d33b - test::__rust_begin_short_backtrace::h0b6a7601d9750bfa
[INFO] [stdout]                                at /rustc/4a4ef493e3a1488c6e321570238084b38948f6db/library/test/src/lib.rs:663:18
[INFO] [stdout]   48:     0x612f2cfb0d4a - test::run_test_in_process::{{closure}}::h444209903f00b347
[INFO] [stdout]                                at /rustc/4a4ef493e3a1488c6e321570238084b38948f6db/library/test/src/lib.rs:686:74
[INFO] [stdout]   49:     0x612f2cfb0d4a - <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once::hfb3ba8f46e537649
[INFO] [stdout]                                at /rustc/4a4ef493e3a1488c6e321570238084b38948f6db/library/core/src/panic/unwind_safe.rs:274:9
[INFO] [stdout]   50:     0x612f2cfb0d4a - std::panicking::catch_unwind::do_call::h1672a45911fcf9dc
[INFO] [stdout]                                at /rustc/4a4ef493e3a1488c6e321570238084b38948f6db/library/std/src/panicking.rs:581:40
[INFO] [stdout]   51:     0x612f2cfb0d4a - std::panicking::catch_unwind::h0fadaee22787a6dd
[INFO] [stdout]                                at /rustc/4a4ef493e3a1488c6e321570238084b38948f6db/library/std/src/panicking.rs:544:19
[INFO] [stdout]   52:     0x612f2cfb0d4a - std::panic::catch_unwind::hc47ad190f2c8e188
[INFO] [stdout]                                at /rustc/4a4ef493e3a1488c6e321570238084b38948f6db/library/std/src/panic.rs:359:14
[INFO] [stdout]   53:     0x612f2cfb0d4a - test::run_test_in_process::hcd7faaf934f29999
[INFO] [stdout]                                at /rustc/4a4ef493e3a1488c6e321570238084b38948f6db/library/test/src/lib.rs:686:27
[INFO] [stdout]   54:     0x612f2cfb0d4a - test::run_test::{{closure}}::hddc4550da4871867
[INFO] [stdout]                                at /rustc/4a4ef493e3a1488c6e321570238084b38948f6db/library/test/src/lib.rs:607:43
[INFO] [stdout]   55:     0x612f2cf8b1a4 - test::run_test::{{closure}}::h98ae26689f13ed94
[INFO] [stdout]                                at /rustc/4a4ef493e3a1488c6e321570238084b38948f6db/library/test/src/lib.rs:637:41
[INFO] [stdout]   56:     0x612f2cf8b1a4 - std::sys::backtrace::__rust_begin_short_backtrace::h3ec4d1f21ad44588
[INFO] [stdout]                                at /rustc/4a4ef493e3a1488c6e321570238084b38948f6db/library/std/src/sys/backtrace.rs:166:18
[INFO] [stdout]   57:     0x612f2cf8eb42 - std::thread::lifecycle::spawn_unchecked::{{closure}}::{{closure}}::hd8bf2c73d479939f
[INFO] [stdout]                                at /rustc/4a4ef493e3a1488c6e321570238084b38948f6db/library/std/src/thread/lifecycle.rs:91:13
[INFO] [stdout]   58:     0x612f2cf8eb42 - <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once::h3b4248f358af2491
[INFO] [stdout]                                at /rustc/4a4ef493e3a1488c6e321570238084b38948f6db/library/core/src/panic/unwind_safe.rs:274:9
[INFO] [stdout]   59:     0x612f2cf8eb42 - std::panicking::catch_unwind::do_call::hda10a64f4b8daebe
[INFO] [stdout]                                at /rustc/4a4ef493e3a1488c6e321570238084b38948f6db/library/std/src/panicking.rs:581:40
[INFO] [stdout]   60:     0x612f2cf8eb42 - std::panicking::catch_unwind::h659e5ee8336ec8c7
[INFO] [stdout]                                at /rustc/4a4ef493e3a1488c6e321570238084b38948f6db/library/std/src/panicking.rs:544:19
[INFO] [stdout]   61:     0x612f2cf8eb42 - std::panic::catch_unwind::h9d4d66a538912c18
[INFO] [stdout]                                at /rustc/4a4ef493e3a1488c6e321570238084b38948f6db/library/std/src/panic.rs:359:14
[INFO] [stdout]   62:     0x612f2cf8eb42 - std::thread::lifecycle::spawn_unchecked::{{closure}}::h7557906bb51d79f1
[INFO] [stdout]                                at /rustc/4a4ef493e3a1488c6e321570238084b38948f6db/library/std/src/thread/lifecycle.rs:89:26
[INFO] [stdout]   63:     0x612f2cf8eb42 - core::ops::function::FnOnce::call_once{{vtable.shim}}::h9e460d1d7fc83d35
[INFO] [stdout]                                at /rustc/4a4ef493e3a1488c6e321570238084b38948f6db/library/core/src/ops/function.rs:250:5
[INFO] [stdout]   64:     0x612f2d611baf - <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once::h61282da819d64ea9
[INFO] [stdout]                                at /rustc/4a4ef493e3a1488c6e321570238084b38948f6db/library/alloc/src/boxed.rs:2206:9
[INFO] [stdout]   65:     0x612f2d611baf - std::sys::thread::unix::Thread::new::thread_start::h982f9ea829d1b5fb
[INFO] [stdout]                                at /rustc/4a4ef493e3a1488c6e321570238084b38948f6db/library/std/src/sys/thread/unix.rs:127:17
[INFO] [stdout]   66:     0x76b8ed656aa4 - <unknown>
[INFO] [stdout]   67:     0x76b8ed6e3a64 - clone
[INFO] [stdout]   68:                0x0 - <unknown>
[INFO] [stdout] 
[INFO] [stdout] 
[INFO] [stdout] failures:
[INFO] [stdout]     test_concurrent_writes_to_same_key
[INFO] [stdout] 
[INFO] [stdout] test result: FAILED. 12 passed; 1 failed; 0 ignored; 0 measured; 0 filtered out; finished in 1.06s
[INFO] [stdout] 
[INFO] running `Command { std: "docker" "inspect" "c017efbab1fe5117a8f95fcff13217933b1986f16ec55bdadfa07ac32a32290d", kill_on_drop: false }`
[INFO] running `Command { std: "docker" "rm" "-f" "c017efbab1fe5117a8f95fcff13217933b1986f16ec55bdadfa07ac32a32290d", kill_on_drop: false }`
[INFO] [stdout] c017efbab1fe5117a8f95fcff13217933b1986f16ec55bdadfa07ac32a32290d
