[INFO] fetching crate melin-server 0.15.0... [INFO] testing melin-server-0.15.0 against 1.100.0-beta.1+cargoflags=--release for beta-release-1.100-2 [INFO] extracting crate melin-server 0.15.0 into /workspace/builds/worker-7-tc2/source [INFO] started tweaking crates.io crate melin-server 0.15.0 [INFO] removed 0 missing tests [INFO] finished tweaking crates.io crate melin-server 0.15.0 [INFO] tweaked toml for crates.io crate melin-server 0.15.0 written to /workspace/builds/worker-7-tc2/source/Cargo.toml [INFO] validating manifest of crates.io crate melin-server 0.15.0 on toolchain 1.100.0-beta.1 [INFO] running `Command { std: CARGO_HOME="/workspace/cargo-home" RUSTUP_HOME="/workspace/rustup-home" "/workspace/cargo-home/bin/cargo" "+1.100.0-beta.1" "metadata" "--manifest-path" "Cargo.toml" "--no-deps", kill_on_drop: false }` [INFO] crate crates.io crate melin-server 0.15.0 already has a lockfile, it will not be regenerated [INFO] running `Command { std: CARGO_HOME="/workspace/cargo-home" RUSTUP_HOME="/workspace/rustup-home" "/workspace/cargo-home/bin/cargo" "+1.100.0-beta.1" "fetch" "--manifest-path" "Cargo.toml", kill_on_drop: false }` [INFO] [stderr] warning: `package.homepage` is redundant with `package.repository` [INFO] [stderr] --> Cargo.toml:154:12 [INFO] [stderr] | [INFO] [stderr] 154 | homepage = "https://github.com/melin-engine/exchange-core" [INFO] [stderr] | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [INFO] [stderr] ... [INFO] [stderr] 158 | repository = "https://github.com/melin-engine/exchange-core" [INFO] [stderr] | ----------------------------------------------- [INFO] [stderr] | [INFO] [stderr] = note: `cargo::redundant_homepage` is set to `warn` by default [INFO] [stderr] help: consider removing `package.homepage` [INFO] [stderr] warning: `melin-server` (manifest) generated 1 warning [INFO] running `Command { std: "docker" "create" "-v" "/var/lib/crater-agent-workspace/builds/worker-7-tc2/source:/opt/rustwide/workdir:ro,Z" "-v" "/var/lib/crater-agent-workspace/builds/worker-7-tc2/target:/opt/rustwide/target:rw,Z" "-v" "/var/lib/crater-agent-workspace/cargo-home:/opt/rustwide/cargo-home:ro,Z" "-v" "/var/lib/crater-agent-workspace/rustup-home:/opt/rustwide/rustup-home:ro,Z" "-m" "1610612736" "--network" "none" "ghcr.io/rust-lang/crates-build-env/linux@sha256:77db811e55d90add9212f6832c23229e3de5f1c1f9905e8cb07319ea15df73ac" "sleep" "infinity", kill_on_drop: false }` [INFO] [stdout] 10c669e04f2a16b15d21c73f6f2d08c97cbc473105ebd58f305f122aa46aa314 [INFO] running `Command { std: "docker" "start" "10c669e04f2a16b15d21c73f6f2d08c97cbc473105ebd58f305f122aa46aa314", kill_on_drop: false }` [INFO] running `Command { std: "docker" "inspect" "10c669e04f2a16b15d21c73f6f2d08c97cbc473105ebd58f305f122aa46aa314", kill_on_drop: false }` [INFO] running `Command { std: "docker" "exec" "-e" "SOURCE_DIR=/opt/rustwide/workdir" "-e" "CARGO_HOME=/opt/rustwide/cargo-home" "-e" "RUSTUP_HOME=/opt/rustwide/rustup-home" "-e" "CARGO_TARGET_DIR=/opt/rustwide/target" "-w" "/opt/rustwide/workdir" "--user" "0:0" "10c669e04f2a16b15d21c73f6f2d08c97cbc473105ebd58f305f122aa46aa314" "/opt/rustwide/cargo-home/bin/cargo" "+1.100.0-beta.1" "metadata" "--no-deps" "--format-version=1", kill_on_drop: false }` [INFO] running `Command { std: "docker" "inspect" "10c669e04f2a16b15d21c73f6f2d08c97cbc473105ebd58f305f122aa46aa314", kill_on_drop: false }` [INFO] running `Command { std: "docker" "exec" "-e" "SOURCE_DIR=/opt/rustwide/workdir" "-e" "CARGO_HOME=/opt/rustwide/cargo-home" "-e" "RUSTUP_HOME=/opt/rustwide/rustup-home" "-e" "CARGO_TARGET_DIR=/opt/rustwide/target" "-e" "CARGO_INCREMENTAL=0" "-e" "RUST_BACKTRACE=full" "-e" "RUSTFLAGS=--cap-lints=warn" "-e" "RUSTDOCFLAGS=--cap-lints=warn" "-w" "/opt/rustwide/workdir" "--user" "0:0" "10c669e04f2a16b15d21c73f6f2d08c97cbc473105ebd58f305f122aa46aa314" "/opt/rustwide/cargo-home/bin/cargo" "+1.100.0-beta.1" "build" "--frozen" "--message-format=json" "--release", kill_on_drop: false }` [INFO] [stderr] warning: `package.homepage` is redundant with `package.repository` [INFO] [stderr] --> Cargo.toml:154:12 [INFO] [stderr] | [INFO] [stderr] 154 | homepage = "https://github.com/melin-engine/exchange-core" [INFO] [stderr] | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [INFO] [stderr] ... [INFO] [stderr] 158 | repository = "https://github.com/melin-engine/exchange-core" [INFO] [stderr] | ----------------------------------------------- [INFO] [stderr] | [INFO] [stderr] = note: `cargo::redundant_homepage` is set to `warn` by default [INFO] [stderr] help: consider removing `package.homepage` [INFO] [stderr] warning: `melin-server` (manifest) generated 1 warning [INFO] [stderr] Compiling syn v2.0.119 [INFO] [stderr] Compiling syn v3.0.5 [INFO] [stderr] Compiling zerocopy v0.8.57 [INFO] [stderr] Compiling typenum v1.20.1 [INFO] [stderr] Compiling serde_core v1.0.229 [INFO] [stderr] Compiling arrayvec v0.7.8 [INFO] [stderr] Compiling serde v1.0.229 [INFO] [stderr] Compiling num-traits v0.2.19 [INFO] [stderr] Compiling futures-sink v0.3.34 [INFO] [stderr] Compiling futures-core v0.3.34 [INFO] [stderr] Compiling getrandom v0.2.17 [INFO] [stderr] Compiling blake3 v1.8.7 [INFO] [stderr] Compiling crc32c v0.6.8 [INFO] [stderr] Compiling curve25519-dalek v4.1.3 [INFO] [stderr] Compiling thiserror v2.0.20 [INFO] [stderr] Compiling slab v0.4.12 [INFO] [stderr] Compiling bitflags v2.13.2 [INFO] [stderr] Compiling rust_decimal v1.43.0 [INFO] [stderr] Compiling signature v2.2.0 [INFO] [stderr] Compiling futures-channel v0.3.34 [INFO] [stderr] Compiling rand_core v0.6.4 [INFO] [stderr] Compiling zeroize v1.9.0 [INFO] [stderr] Compiling io-uring v0.7.15 [INFO] [stderr] Compiling iana-time-zone v0.1.65 [INFO] [stderr] Compiling rustix v1.1.4 [INFO] [stderr] Compiling clap_builder v4.6.6 [INFO] [stderr] Compiling ed25519 v2.2.3 [INFO] [stderr] Compiling mio v1.2.3 [INFO] [stderr] Compiling memmap2 v0.9.11 [INFO] [stderr] Compiling melin-pipeline v0.15.0 [INFO] [stderr] Compiling melin-types v0.15.0 [INFO] [stderr] Compiling utf8-width v0.1.9 [INFO] [stderr] Compiling tikv-jemalloc-sys v0.6.1+5.3.0-1-ge13ca993e8ccb9ba9847cc330696e02839f328f7 [INFO] [stderr] Compiling maplit v1.0.2 [INFO] [stderr] Compiling melin-wire-protocol v0.15.0 [INFO] [stderr] Compiling regex-automata v0.4.18 [INFO] [stderr] Compiling log v0.4.34 [INFO] [stderr] Compiling chrono v0.4.45 [INFO] [stderr] Compiling astenn v0.1.0 [INFO] [stderr] Compiling tracing-log v0.2.0 [INFO] [stderr] Compiling rand v0.10.2 [INFO] [stderr] Compiling thread_local v1.1.10 [INFO] [stderr] Compiling generic-array v0.14.7 [INFO] [stderr] Compiling smallvec v1.16.0 [INFO] [stderr] Compiling crypto-common v0.1.7 [INFO] [stderr] Compiling block-buffer v0.10.4 [INFO] [stderr] Compiling digest v0.10.7 [INFO] [stderr] Compiling sha2 v0.10.9 [INFO] [stderr] Compiling byte-unit v5.2.5 [INFO] [stderr] Compiling matchers v0.2.0 [INFO] [stderr] Compiling serde_derive v1.0.229 [INFO] [stderr] Compiling tracing-attributes v0.1.31 [INFO] [stderr] Compiling zerocopy-derive v0.8.57 [INFO] [stderr] Compiling futures-macro v0.3.34 [INFO] [stderr] Compiling curve25519-dalek-derive v0.1.1 [INFO] [stderr] Compiling pin-project-internal v1.1.13 [INFO] [stderr] Compiling thiserror-impl v2.0.20 [INFO] [stderr] Compiling derive_more-impl v1.0.0 [INFO] [stderr] Compiling thiserror-impl v1.0.69 [INFO] [stderr] Compiling clap_derive v4.6.4 [INFO] [stderr] Compiling tokio-macros v2.7.2 [INFO] [stderr] Compiling openraft-macros v0.9.25 [INFO] [stderr] Compiling tokio v1.53.1 [INFO] [stderr] Compiling futures-util v0.3.34 [INFO] [stderr] Compiling pin-project v1.1.13 [INFO] [stderr] Compiling tracing v0.1.44 [INFO] [stderr] Compiling melin-app v0.15.0 [INFO] [stderr] Compiling tracing-futures v0.2.5 [INFO] [stderr] Compiling tracing-subscriber v0.3.23 [INFO] [stderr] Compiling thiserror v1.0.69 [INFO] [stderr] Compiling melin-trading v0.15.0 [INFO] [stderr] Compiling melin-exchange-core v0.15.0 [INFO] [stderr] Compiling cobs v0.3.0 [INFO] [stderr] Compiling clap v4.6.6 [INFO] [stderr] Compiling derive_more v1.0.0 [INFO] [stderr] Compiling anyerror v0.1.13 [INFO] [stderr] Compiling ed25519-dalek v2.2.0 [INFO] [stderr] Compiling postcard v1.1.3 [INFO] [stderr] Compiling validit v0.2.6 [INFO] [stderr] Compiling futures-executor v0.3.34 [INFO] [stderr] Compiling futures v0.3.34 [INFO] [stderr] Compiling ppv-lite86 v0.2.21 [INFO] [stderr] Compiling melin-journal v0.15.0 [INFO] [stderr] Compiling melin-protocol v0.15.0 [INFO] [stderr] Compiling rand_chacha v0.3.1 [INFO] [stderr] Compiling melin-market-data v0.15.0 [INFO] [stderr] Compiling rand v0.8.8 [INFO] [stderr] Compiling melin-transport-core v0.15.0 [INFO] [stderr] Compiling openraft v0.9.25 [INFO] [stderr] Compiling tikv-jemallocator v0.6.1 [INFO] [stderr] Compiling melin-raft v0.15.0 [INFO] [stderr] Compiling melin-server-runtime v0.15.0 [INFO] [stderr] Compiling melin-server v0.15.0 (/opt/rustwide/workdir) [INFO] [stderr] Finished `release` profile [optimized] target(s) in 2m 55s [INFO] running `Command { std: "docker" "inspect" "10c669e04f2a16b15d21c73f6f2d08c97cbc473105ebd58f305f122aa46aa314", kill_on_drop: false }` [INFO] running `Command { std: "docker" "exec" "-e" "SOURCE_DIR=/opt/rustwide/workdir" "-e" "CARGO_HOME=/opt/rustwide/cargo-home" "-e" "RUSTUP_HOME=/opt/rustwide/rustup-home" "-e" "CARGO_TARGET_DIR=/opt/rustwide/target" "-e" "CARGO_INCREMENTAL=0" "-e" "RUST_BACKTRACE=full" "-e" "RUSTFLAGS=--cap-lints=warn" "-e" "RUSTDOCFLAGS=--cap-lints=warn" "-w" "/opt/rustwide/workdir" "--user" "0:0" "10c669e04f2a16b15d21c73f6f2d08c97cbc473105ebd58f305f122aa46aa314" "/opt/rustwide/cargo-home/bin/cargo" "+1.100.0-beta.1" "test" "--frozen" "--no-run" "--message-format=json" "--release", kill_on_drop: false }` [INFO] [stderr] warning: `package.homepage` is redundant with `package.repository` [INFO] [stderr] --> Cargo.toml:154:12 [INFO] [stderr] | [INFO] [stderr] 154 | homepage = "https://github.com/melin-engine/exchange-core" [INFO] [stderr] | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [INFO] [stderr] ... [INFO] [stderr] 158 | repository = "https://github.com/melin-engine/exchange-core" [INFO] [stderr] | ----------------------------------------------- [INFO] [stderr] | [INFO] [stderr] = note: `cargo::redundant_homepage` is set to `warn` by default [INFO] [stderr] help: consider removing `package.homepage` [INFO] [stderr] warning: `melin-server` (manifest) generated 1 warning [INFO] [stderr] Compiling getrandom v0.3.4 [INFO] [stderr] Compiling melin-journal v0.15.0 [INFO] [stderr] Compiling indexmap v2.14.2 [INFO] [stderr] Compiling toml_datetime v0.6.3 [INFO] [stderr] Compiling winnow v0.5.40 [INFO] [stderr] Compiling pretty-hex v0.4.2 [INFO] [stderr] Compiling serial_test_derive v3.5.0 [INFO] [stderr] Compiling fastrand v2.5.0 [INFO] [stderr] Compiling melin-client v0.15.0 [INFO] [stderr] Compiling parking_lot_core v0.9.12 [INFO] [stderr] Compiling rand_core v0.9.5 [INFO] [stderr] Compiling parking_lot v0.12.5 [INFO] [stderr] Compiling tempfile v3.27.0 [INFO] [stderr] Compiling rand_xoshiro v0.7.0 [INFO] [stderr] Compiling rand_chacha v0.9.0 [INFO] [stderr] Compiling serial_test v3.5.0 [INFO] [stderr] Compiling rand v0.9.5 [INFO] [stderr] Compiling melin-transport-core v0.15.0 [INFO] [stderr] Compiling toml_edit v0.20.2 [INFO] [stderr] Compiling melin-raft v0.15.0 [INFO] [stderr] Compiling proc-macro-crate v2.0.2 [INFO] [stderr] Compiling bolero-generator-derive v0.13.4 [INFO] [stderr] Compiling melin-server-runtime v0.15.0 [INFO] [stderr] Compiling bolero-generator v0.13.5 [INFO] [stderr] Compiling melin-server v0.15.0 (/opt/rustwide/workdir) [INFO] [stderr] Compiling bolero-engine v0.13.4 [INFO] [stderr] Compiling bolero v0.13.4 [INFO] [stderr] Finished `release` profile [optimized] target(s) in 50.79s [INFO] running `Command { std: "docker" "inspect" "10c669e04f2a16b15d21c73f6f2d08c97cbc473105ebd58f305f122aa46aa314", kill_on_drop: false }` [INFO] running `Command { std: "docker" "exec" "-e" "SOURCE_DIR=/opt/rustwide/workdir" "-e" "CARGO_HOME=/opt/rustwide/cargo-home" "-e" "RUSTUP_HOME=/opt/rustwide/rustup-home" "-e" "CARGO_TARGET_DIR=/opt/rustwide/target" "-e" "CARGO_INCREMENTAL=0" "-e" "RUST_BACKTRACE=full" "-e" "RUSTFLAGS=--cap-lints=warn" "-e" "RUSTDOCFLAGS=--cap-lints=warn" "-w" "/opt/rustwide/workdir" "--user" "0:0" "10c669e04f2a16b15d21c73f6f2d08c97cbc473105ebd58f305f122aa46aa314" "/opt/rustwide/cargo-home/bin/cargo" "+1.100.0-beta.1" "test" "--frozen" "--release", kill_on_drop: false }` [INFO] [stderr] warning: `package.homepage` is redundant with `package.repository` [INFO] [stderr] --> Cargo.toml:154:12 [INFO] [stderr] | [INFO] [stderr] 154 | homepage = "https://github.com/melin-engine/exchange-core" [INFO] [stderr] | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [INFO] [stderr] ... [INFO] [stderr] 158 | repository = "https://github.com/melin-engine/exchange-core" [INFO] [stderr] | ----------------------------------------------- [INFO] [stderr] | [INFO] [stderr] = note: `cargo::redundant_homepage` is set to `warn` by default [INFO] [stderr] help: consider removing `package.homepage` [INFO] [stderr] warning: `melin-server` (manifest) generated 1 warning [INFO] [stderr] Finished `release` profile [optimized] target(s) in 0.18s [INFO] [stderr] Running unittests src/lib.rs (/opt/rustwide/target/release/build/melin-server/d6408d389933277a/out/melin_server-d6408d389933277a) [INFO] [stdout] [INFO] [stdout] running 49 tests [INFO] [stdout] test event_publisher::tests::frame_encoding_with_sequence_prefix ... ok [INFO] [stdout] test event_publisher::tests::multiple_subscribers_receive_same_frame ... ok [INFO] [stdout] test event_publisher::tests::payload_to_response_all_variants ... ok [INFO] [stdout] test event_publisher::tests::report_symbol_extracts_all_variants ... ok [INFO] [stdout] test app_factory::tests::seed_events_count_matches_config ... ok [INFO] [stdout] test exchange_app::tests::apply_withdraw_emits_rejection_on_failure ... ok [INFO] [stdout] test exchange_app::tests::apply_query_request_seq_returns_per_key_hwm ... ok [INFO] [stdout] test event_publisher::tests::slow_subscriber_disconnected ... ok [INFO] [stdout] test app_factory::tests::seed_events_empty_when_no_accounts_or_instruments ... ok [INFO] [stdout] test exchange_app::tests::check_request_seq_rejects_duplicates ... ok [INFO] [stdout] test event_publisher::tests::monotonic_sequence_numbers_in_frames ... ok [INFO] [stdout] test exchange_app::tests::snapshot_restore_round_trip_preserves_state ... ok [INFO] [stdout] test exchange_app::tests::apply_submit_order_produces_placed_report ... ok [INFO] [stdout] test request_decoder::tests::add_instrument_as_operator_is_permitted ... ok [INFO] [stdout] test request_decoder::tests::add_instrument_as_trader_is_denied ... ok [INFO] [stdout] test request_decoder::tests::challenge_response_is_filtered ... ok [INFO] [stdout] test request_decoder::tests::deposit_as_trader_is_denied ... ok [INFO] [stdout] test app_factory::tests::seed_events_order_is_instruments_then_accounts ... ok [INFO] [stdout] test exchange_app::tests::tick_advances_scheduler_clock ... ok [INFO] [stdout] test request_decoder::tests::heartbeat_panics_if_not_filtered - should panic ... ok [INFO] [stdout] test exchange_app::tests::build_reject_maps_transport_reasons ... ok [INFO] [stdout] test request_decoder::tests::maps_cancel_all ... ok [INFO] [stdout] test request_decoder::tests::maps_cancel_order ... ok [INFO] [stdout] test request_decoder::tests::maps_deposit ... ok [INFO] [stdout] test request_decoder::tests::maps_set_fee_schedule ... ok [INFO] [stdout] test request_decoder::tests::maps_cancel_replace ... ok [INFO] [stdout] test request_decoder::tests::maps_add_instrument ... ok [INFO] [stdout] test request_decoder::tests::maps_set_risk_limits ... ok [INFO] [stdout] test request_decoder::tests::maps_query_stats ... ok [INFO] [stdout] test request_decoder::tests::subscribe_is_filtered ... ok [INFO] [stdout] test request_decoder::tests::malformed_frame_yields_decode_error ... ok [INFO] [stdout] test request_decoder::tests::challenge_response_panics_if_not_filtered - should panic ... ok [INFO] [stdout] test request_decoder::tests::deposit_as_custodian_is_permitted ... ok [INFO] [stdout] test request_decoder::tests::heartbeat_is_filtered ... ok [INFO] [stdout] test response_encoder::tests::encodes_query_position ... ok [INFO] [stdout] test response_encoder::tests::encodes_query_stats ... ok [INFO] [stdout] test request_decoder::tests::maps_set_circuit_breaker ... ok [INFO] [stdout] test request_decoder::tests::query_stats_is_permitted_and_flagged ... ok [INFO] [stdout] test response_encoder::tests::encodes_report ... ok [INFO] [stdout] test request_decoder::tests::submit_order_as_trader_is_permitted ... ok [INFO] [stdout] test request_decoder::tests::maps_submit_order ... ok [INFO] [stdout] test request_decoder::tests::submit_order_as_readonly_is_denied ... ok [INFO] [stdout] test response_encoder::tests::encodes_query_request_seq_hwm ... ok [INFO] [stdout] test event_publisher::tests::failed_subscriber_does_not_affect_others ... ok [INFO] [stdout] test event_publisher::tests::shutdown_stops_publisher ... ok [INFO] [stdout] test app_factory::tests::apply_operator_policy_overrides_default ... ok [INFO] [stdout] test app_factory::tests::empty_does_not_apply_policy ... ok [INFO] [stdout] test event_publisher::tests::send_snapshot_empty_book ... ok [INFO] [stderr] Running unittests src/bin/journal-diff.rs (/opt/rustwide/target/release/build/melin-server/a06381534622baf8/out/journal_diff-a06381534622baf8) [INFO] [stdout] test event_publisher::tests::send_snapshot_with_levels ... ok [INFO] [stdout] [INFO] [stdout] test result: ok. 49 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 1.02s [INFO] [stdout] [INFO] [stdout] [INFO] [stdout] running 0 tests [INFO] [stdout] [INFO] [stdout] test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s [INFO] [stdout] [INFO] [stderr] Running unittests src/bin/journal-verify.rs (/opt/rustwide/target/release/build/melin-server/8175f7f53a27afbf/out/journal_verify-8175f7f53a27afbf) [INFO] [stderr] Running unittests src/main.rs (/opt/rustwide/target/release/build/melin-server/13f5406e918055ff/out/melin_server-13f5406e918055ff) [INFO] [stdout] [INFO] [stdout] running 0 tests [INFO] [stdout] [INFO] [stdout] test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s [INFO] [stdout] [INFO] [stdout] [INFO] [stderr] Running tests/engine_fuzz.rs (/opt/rustwide/target/release/build/melin-server/591884f2ab0cd724/out/engine_fuzz-591884f2ab0cd724) [INFO] [stdout] running 0 tests [INFO] [stdout] [INFO] [stdout] test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s [INFO] [stdout] [INFO] [stdout] [INFO] [stdout] running 4 tests [INFO] [stdout] test fuzz_snapshot_decode ... ok [INFO] [stdout] test fuzz_journal_decode ... ok [INFO] [stdout] test fuzz_journal_file_header ... ok [INFO] [stderr] Running tests/failover.rs (/opt/rustwide/target/release/build/melin-server/85eb41ab37658f0e/out/failover-85eb41ab37658f0e) [INFO] [stdout] test fuzz_journal_roundtrip ... ok [INFO] [stdout] [INFO] [stdout] test result: ok. 4 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 1.01s [INFO] [stdout] [INFO] [stdout] [INFO] [stdout] running 24 tests [INFO] [stderr] [INFO] [stderr] thread 'response' (7568) panicked at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/melin-server-runtime-0.15.0/src/response.rs:300:10: [INFO] [stderr] failed to create io_uring instance for response stage: Os { code: 1, kind: PermissionDenied, message: "Operation not permitted" } [INFO] [stderr] stack backtrace: [INFO] [stderr] 0: 0x5c47cd3d29a1 - <::print::DisplayBacktrace as core[a373ad73a504971]::fmt::Display>::fmt [INFO] [stderr] 1: 0x5c47cd3ea63a - core[a373ad73a504971]::fmt::write [INFO] [stderr] 2: 0x5c47cd3d86ec - ::write_fmt [INFO] [stderr] 3: 0x5c47cd3b31d6 - std[617be2ca0a6fd877]::panicking::default_hook::{closure#0} [INFO] [stderr] 4: 0x5c47cd3c9d41 - std[617be2ca0a6fd877]::panicking::default_hook [INFO] [stderr] 5: 0x5c47cd3ca06b - std[617be2ca0a6fd877]::panicking::panic_with_hook [INFO] [stderr] 6: 0x5c47cd3b3282 - std[617be2ca0a6fd877]::panicking::panic_handler::{closure#0} [INFO] [stderr] 7: 0x5c47cd3abcf9 - std[617be2ca0a6fd877]::sys::backtrace::__rust_end_short_backtrace:: [INFO] [stderr] 8: 0x5c47cd3b403d - __rustc[ea4f61d10b84890b]::rust_begin_unwind [INFO] [stderr] 9: 0x5c47cd3eae1c - core[a373ad73a504971]::panicking::panic_fmt [INFO] [stderr] 10: 0x5c47cd3eabf2 - core[a373ad73a504971]::result::unwrap_failed [INFO] [stderr] 11: 0x5c47ccff740f - melin_server_runtime[439fe4d1bbbbe025]::response::run:: [INFO] [stderr] 12: 0x5c47ccfdee1f - std[617be2ca0a6fd877]::sys::backtrace::__rust_begin_short_backtrace::::{closure#5}, ()> [INFO] [stderr] 13: 0x5c47ccfeaa84 - ::{closure#5}, ()>::{closure#1} as core[a373ad73a504971]::ops::function::FnOnce<()>>::call_once::{shim:vtable#0} [INFO] [stderr] 14: 0x5c47cd3d12ff - ::new::thread_start [INFO] [stderr] 15: 0x77f776cb6dfa - [INFO] [stderr] 16: 0x77f776d4a3d4 - clone [INFO] [stderr] 17: 0x0 - [INFO] [stderr] [INFO] [stdout] 2026-10-18T11:53:14.533852Z ERROR main melin_server_runtime::server: pipeline thread died, initiating shutdown [INFO] [stderr] thread 'uring-reader' (7580) panicked at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/melin-server-runtime-0.15.0/src/reader.rs:421:10: [INFO] [stdout] 2026-10-18T11:53:14.535217Z ERROR main melin_server_runtime::reader: reader thread panicked failed to create io_uring instance: Os { code: 1, kind: PermissionDenied, message: "Operation not permitted" } [INFO] [stderr] failed to create io_uring instance: Os { code: 1, kind: PermissionDenied, message: "Operation not permitted" } [INFO] [stderr] stack backtrace: [INFO] [stderr] 0: 0x5c47cd3d29a1 - <::print::DisplayBacktrace as core[a373ad73a504971]::fmt::Display>::fmt [INFO] [stderr] 1: 0x5c47cd3ea63a - core[a373ad73a504971]::fmt::write [INFO] [stderr] 2: 0x5c47cd3d86ec - ::write_fmt [INFO] [stderr] 3: 0x5c47cd3b31d6 - std[617be2ca0a6fd877]::panicking::default_hook::{closure#0} [INFO] [stderr] 4: 0x5c47cd3c9d41 - std[617be2ca0a6fd877]::panicking::default_hook [INFO] [stderr] 5: 0x5c47cd3ca06b - std[617be2ca0a6fd877]::panicking::panic_with_hook [INFO] [stderr] 6: 0x5c47cd3b3282 - std[617be2ca0a6fd877]::panicking::panic_handler::{closure#0} [INFO] [stderr] 7: 0x5c47cd3abcf9 - std[617be2ca0a6fd877]::sys::backtrace::__rust_end_short_backtrace:: [INFO] [stderr] 8: 0x5c47cd3b403d - __rustc[ea4f61d10b84890b]::rust_begin_unwind [INFO] [stderr] 9: 0x5c47cd3eae1c - core[a373ad73a504971]::panicking::panic_fmt [INFO] [stderr] 10: 0x5c47cd3eabf2 - core[a373ad73a504971]::result::unwrap_failed [INFO] [stderr] 11: 0x5c47cd007678 - melin_server_runtime[439fe4d1bbbbe025]::reader::reader_loop:: [INFO] [stderr] 12: 0x5c47ccfde495 - std[617be2ca0a6fd877]::sys::backtrace::__rust_begin_short_backtrace::::{closure#0}, ()> [INFO] [stderr] 13: 0x5c47ccfea2be - ::{closure#0}, ()>::{closure#1} as core[a373ad73a504971]::ops::function::FnOnce<()>>::call_once::{shim:vtable#0} [INFO] [stderr] 14: 0x5c47cd3d12ff - ::new::thread_start [INFO] [stderr] 15: 0x77f776cb6dfa - [INFO] [stderr] 16: 0x77f776d4a3d4 - clone [INFO] [stderr] 17: 0x0 - [INFO] [stdout] 2026-10-18T11:53:15.212285Z ERROR replica-receiver melin_server_runtime::replication::tcp_receiver: UringTransport init failed error=io_uring init failed: Operation not permitted (os error 1) [INFO] [stdout] 2026-10-18T11:53:15.353637Z ERROR replica-receiver melin_server_runtime::replication::tcp_receiver: UringTransport init failed error=io_uring init failed: Operation not permitted (os error 1) [INFO] [stdout] 2026-10-18T11:53:15.484944Z ERROR main melin_server_runtime::server: pipeline thread panicked thread="response" failed to create io_uring instance for response stage: Os { code: 1, kind: PermissionDenied, message: "Operation not permitted" } [INFO] [stdout] 2026-10-18T11:53:15.485048Z ERROR main melin_server_runtime::server: shutdown complete (with pipeline failure) [INFO] [stderr] Error: "pipeline failure" [INFO] [stdout] test evicted_replica_catchup_under_load_preserves_dense_lineage ... FAILED [INFO] [stderr] [INFO] [stderr] thread 'response' (7599) panicked at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/melin-server-runtime-0.15.0/src/response.rs:300:10: [INFO] [stderr] failed to create io_uring instance for response stage: Os { code: 1, kind: PermissionDenied, message: "Operation not permitted" } [INFO] [stderr] stack backtrace: [INFO] [stderr] 0: 0x5a4dba76a9a1 - <::print::DisplayBacktrace as core[a373ad73a504971]::fmt::Display>::fmt [INFO] [stderr] 1: 0x5a4dba78263a - core[a373ad73a504971]::fmt::write [INFO] [stderr] 2: 0x5a4dba7706ec - ::write_fmt [INFO] [stderr] 3: 0x5a4dba74b1d6 - std[617be2ca0a6fd877]::panicking::default_hook::{closure#0} [INFO] [stderr] 4: 0x5a4dba761d41 - std[617be2ca0a6fd877]::panicking::default_hook [INFO] [stderr] 5: 0x5a4dba76206b - std[617be2ca0a6fd877]::panicking::panic_with_hook [INFO] [stderr] 6: 0x5a4dba74b282 - std[617be2ca0a6fd877]::panicking::panic_handler::{closure#0} [INFO] [stderr] 7: 0x5a4dba743cf9 - std[617be2ca0a6fd877]::sys::backtrace::__rust_end_short_backtrace:: [INFO] [stderr] 8: 0x5a4dba74c03d - __rustc[ea4f61d10b84890b]::rust_begin_unwind [INFO] [stderr] 9: 0x5a4dba782e1c - core[a373ad73a504971]::panicking::panic_fmt [INFO] [stderr] 10: 0x5a4dba782bf2 - core[a373ad73a504971]::result::unwrap_failed [INFO] [stderr] 11: 0x5a4dba38f40f - melin_server_runtime[439fe4d1bbbbe025]::response::run:: [INFO] [stderr] 12: 0x5a4dba376e1f - std[617be2ca0a6fd877]::sys::backtrace::__rust_begin_short_backtrace::::{closure#5}, ()> [INFO] [stderr] 13: 0x5a4dba382a84 - ::{closure#5}, ()>::{closure#1} as core[a373ad73a504971]::ops::function::FnOnce<()>>::call_once::{shim:vtable#0} [INFO] [stderr] 14: 0x5a4dba7692ff - ::new::thread_start [INFO] [stderr] 15: 0x787c1f14ddfa - [INFO] [stderr] 16: 0x787c1f1e13d4 - clone [INFO] [stderr] 17: 0x0 - [INFO] [stdout] 2026-10-18T11:53:45.594889Z ERROR main melin_server_runtime::server: pipeline thread died, initiating shutdown [INFO] [stderr] [INFO] [stderr] thread 'uring-reader' (7611) panicked at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/melin-server-runtime-0.15.0/src/reader.rs:421:10: [INFO] [stderr] failed to create io_uring instance: Os { code: 1, kind: PermissionDenied, message: "Operation not permitted" } [INFO] [stderr] stack backtrace: [INFO] [stderr] 0: 0x5a4dba76a9a1 - <::print::DisplayBacktrace as core[a373ad73a504971]::fmt::Display>::fmt [INFO] [stderr] 1: 0x5a4dba78263a - core[a373ad73a504971]::fmt::write [INFO] [stderr] 2: 0x5a4dba7706ec - ::write_fmt [INFO] [stderr] 3: 0x5a4dba74b1d6 - std[617be2ca0a6fd877]::panicking::default_hook::{closure#0} [INFO] [stderr] 4: 0x5a4dba761d41 - std[617be2ca0a6fd877]::panicking::default_hook [INFO] [stderr] 5: 0x5a4dba76206b - std[617be2ca0a6fd877]::panicking::panic_with_hook [INFO] [stderr] 6: 0x5a4dba74b282 - std[617be2ca0a6fd877]::panicking::panic_handler::{closure#0} [INFO] [stderr] 7: 0x5a4dba743cf9 - std[617be2ca0a6fd877]::sys::backtrace::__rust_end_short_backtrace:: [INFO] [stderr] 8: 0x5a4dba74c03d - __rustc[ea4f61d10b84890b]::rust_begin_unwind [INFO] [stderr] 9: 0x5a4dba782e1c - core[a373ad73a504971]::panicking::panic_fmt [INFO] [stderr] 10: 0x5a4dba782bf2 - core[a373ad73a504971]::result::unwrap_failed [INFO] [stderr] 11: 0x5a4dba39f678 - melin_server_runtime[439fe4d1bbbbe025]::reader::reader_loop:: [INFO] [stderr] 12: 0x5a4dba376495 - std[617be2ca0a6fd877]::sys::backtrace::__rust_begin_short_backtrace::::{closure#0}, ()> [INFO] [stderr] 13: 0x5a4dba3822be - ::{closure#0}, ()>::{closure#1} as core[a373ad73a504971]::ops::function::FnOnce<()>>::call_once::{shim:vtable#0} [INFO] [stderr] 14: 0x5a4dba7692ff - ::new::thread_start [INFO] [stderr] 15: 0x787c1f14ddfa - [INFO] [stdout] 2026-10-18T11:53:45.598772Z ERROR main melin_server_runtime::reader: reader thread panicked failed to create io_uring instance: Os { code: 1, kind: PermissionDenied, message: "Operation not permitted" } [INFO] [stderr] 16: 0x787c1f1e13d4 - clone [INFO] [stderr] 17: 0x0 - [INFO] [stdout] 2026-10-18T11:53:45.791601Z ERROR main melin_server_runtime::server: pipeline thread panicked thread="response" failed to create io_uring instance for response stage: Os { code: 1, kind: PermissionDenied, message: "Operation not permitted" } [INFO] [stderr] Error: "pipeline failure" [INFO] [stdout] 2026-10-18T11:53:45.791827Z ERROR main melin_server_runtime::server: shutdown complete (with pipeline failure) [INFO] [stdout] 2026-10-18T11:53:46.068819Z ERROR replica-receiver melin_server_runtime::replication::tcp_receiver: UringTransport init failed error=io_uring init failed: Operation not permitted (os error 1) [INFO] [stdout] 2026-10-18T11:53:46.288939Z ERROR replica-receiver melin_server_runtime::replication::tcp_receiver: UringTransport init failed error=io_uring init failed: Operation not permitted (os error 1) [INFO] [stdout] test catchup_then_immediate_failover has been running for over 60 seconds [INFO] [stdout] test catchup_with_fills_during_gap has been running for over 60 seconds [INFO] [stdout] test crashed_primary_recovers_from_journal has been running for over 60 seconds [INFO] [stdout] test disk_ram_gate_stalls_while_replica_frozen has been running for over 60 seconds [INFO] [stdout] test dual_replication_halts_when_both_disconnect has been running for over 60 seconds [INFO] [stdout] test dual_replication_promote_replica1_after_replica2_dies has been running for over 60 seconds [INFO] [stdout] test dual_replication_survives_one_replica_failure has been running for over 60 seconds [INFO] [stdout] test dual_replication_with_fills_then_failover has been running for over 60 seconds [INFO] [stdout] test fresh_replica_full_catchup has been running for over 60 seconds [INFO] [stdout] test higher_epoch_handshake_fences_stale_primary has been running for over 60 seconds [INFO] [stdout] test in_memory_cursor_runs_ahead_of_persisted_under_sustained_traffic has been running for over 60 seconds [INFO] [stdout] test journals_contiguous_across_replication has been running for over 60 seconds [INFO] [stdout] test kill_during_active_fills has been running for over 60 seconds [INFO] [stdout] test kill_primary_promote_replica_no_data_loss has been running for over 60 seconds [INFO] [stdout] test kill_without_waiting_for_replication has been running for over 60 seconds [INFO] [stdout] test catchup_with_fills_during_gap ... FAILED [INFO] [stderr] [INFO] [stderr] thread 'response' (7630) panicked at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/melin-server-runtime-0.15.0/src/response.rs:300:10: [INFO] [stderr] failed to create io_uring instance for response stage: Os { code: 1, kind: PermissionDenied, message: "Operation not permitted" } [INFO] [stderr] stack backtrace: [INFO] [stderr] 0: 0x5667c2bd99a1 - <::print::DisplayBacktrace as core[a373ad73a504971]::fmt::Display>::fmt [INFO] [stderr] 1: 0x5667c2bf163a - core[a373ad73a504971]::fmt::write [INFO] [stderr] 2: 0x5667c2bdf6ec - ::write_fmt [INFO] [stderr] 3: 0x5667c2bba1d6 - std[617be2ca0a6fd877]::panicking::default_hook::{closure#0} [INFO] [stderr] 4: 0x5667c2bd0d41 - std[617be2ca0a6fd877]::panicking::default_hook [INFO] [stderr] 5: 0x5667c2bd106b - std[617be2ca0a6fd877]::panicking::panic_with_hook [INFO] [stderr] 6: 0x5667c2bba282 - std[617be2ca0a6fd877]::panicking::panic_handler::{closure#0} [INFO] [stderr] 7: 0x5667c2bb2cf9 - std[617be2ca0a6fd877]::sys::backtrace::__rust_end_short_backtrace:: [INFO] [stderr] 8: 0x5667c2bbb03d - __rustc[ea4f61d10b84890b]::rust_begin_unwind [INFO] [stderr] 9: 0x5667c2bf1e1c - core[a373ad73a504971]::panicking::panic_fmt [INFO] [stderr] 10: 0x5667c2bf1bf2 - core[a373ad73a504971]::result::unwrap_failed [INFO] [stderr] 11: 0x5667c27fe40f - melin_server_runtime[439fe4d1bbbbe025]::response::run:: [INFO] [stderr] 12: 0x5667c27e5e1f - std[617be2ca0a6fd877]::sys::backtrace::__rust_begin_short_backtrace::::{closure#5}, ()> [INFO] [stderr] 13: 0x5667c27f1a84 - ::{closure#5}, ()>::{closure#1} as core[a373ad73a504971]::ops::function::FnOnce<()>>::call_once::{shim:vtable#0} [INFO] [stderr] 14: 0x5667c2bd82ff - ::new::thread_start [INFO] [stderr] 15: 0x772eec24edfa - [INFO] [stderr] 16: 0x772eec2e23d4 - clone [INFO] [stderr] 17: 0x0 - [INFO] [stdout] 2026-10-18T11:54:17.193723Z ERROR main melin_server_runtime::server: pipeline thread died, initiating shutdown [INFO] [stderr] [INFO] [stderr] thread 'uring-reader' (7639) panicked at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/melin-server-runtime-0.15.0/src/reader.rs:421:10: [INFO] [stderr] failed to create io_uring instance: Os { code: 1, kind: PermissionDenied, message: "Operation not permitted" } [INFO] [stderr] stack backtrace: [INFO] [stderr] 0: 0x5667c2bd99a1 - <::print::DisplayBacktrace as core[a373ad73a504971]::fmt::Display>::fmt [INFO] [stderr] 1: 0x5667c2bf163a - core[a373ad73a504971]::fmt::write [INFO] [stderr] 2: 0x5667c2bdf6ec - ::write_fmt [INFO] [stderr] 3: 0x5667c2bba1d6 - std[617be2ca0a6fd877]::panicking::default_hook::{closure#0} [INFO] [stderr] 4: 0x5667c2bd0d41 - std[617be2ca0a6fd877]::panicking::default_hook [INFO] [stderr] 5: 0x5667c2bd106b - std[617be2ca0a6fd877]::panicking::panic_with_hook [INFO] [stderr] 6: 0x5667c2bba282 - std[617be2ca0a6fd877]::panicking::panic_handler::{closure#0} [INFO] [stderr] 7: 0x5667c2bb2cf9 - std[617be2ca0a6fd877]::sys::backtrace::__rust_end_short_backtrace:: [INFO] [stderr] 8: 0x5667c2bbb03d - __rustc[ea4f61d10b84890b]::rust_begin_unwind [INFO] [stderr] 9: 0x5667c2bf1e1c - core[a373ad73a504971]::panicking::panic_fmt [INFO] [stderr] 10: 0x5667c2bf1bf2 - core[a373ad73a504971]::result::unwrap_failed [INFO] [stderr] 11: 0x5667c280e678 - melin_server_runtime[439fe4d1bbbbe025]::reader::reader_loop:: [INFO] [stderr] 12: 0x5667c27e5495 - std[617be2ca0a6fd877]::sys::backtrace::__rust_begin_short_backtrace::::{closure#0}, ()> [INFO] [stderr] 13: 0x5667c27f12be - ::{closure#0}, ()>::{closure#1} as core[a373ad73a504971]::ops::function::FnOnce<()>>::call_once::{shim:vtable#0} [INFO] [stderr] 14: 0x5667c2bd82ff - ::new::thread_start [INFO] [stderr] 15: 0x772eec24edfa - [INFO] [stderr] 16: 0x772eec2e23d4 - clone [INFO] [stderr] 17: 0x0 - [INFO] [stdout] 2026-10-18T11:54:17.206583Z ERROR main melin_server_runtime::reader: reader thread panicked failed to create io_uring instance: Os { code: 1, kind: PermissionDenied, message: "Operation not permitted" } [INFO] [stdout] 2026-10-18T11:54:17.701479Z ERROR replica-receiver melin_server_runtime::replication::tcp_receiver: UringTransport init failed error=io_uring init failed: Operation not permitted (os error 1) [INFO] [stderr] Error: "pipeline failure" [INFO] [stdout] 2026-10-18T11:54:18.131079Z ERROR main melin_server_runtime::server: pipeline thread panicked thread="response" failed to create io_uring instance for response stage: Os { code: 1, kind: PermissionDenied, message: "Operation not permitted" } [INFO] [stdout] 2026-10-18T11:54:18.131117Z ERROR main melin_server_runtime::server: shutdown complete (with pipeline failure) [INFO] [stdout] test policy_degraded_gauge_transitions_with_cluster_shape has been running for over 60 seconds [INFO] [stdout] test kill_without_waiting_for_replication ... FAILED [INFO] [stderr] [INFO] [stderr] thread 'response' (7651) panicked at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/melin-server-runtime-0.15.0/src/response.rs:300:10: [INFO] [stderr] failed to create io_uring instance for response stage: Os { code: 1, kind: PermissionDenied, message: "Operation not permitted" } [INFO] [stderr] stack backtrace: [INFO] [stderr] 0: 0x5c1220fff9a1 - <::print::DisplayBacktrace as core[a373ad73a504971]::fmt::Display>::fmt [INFO] [stderr] 1: 0x5c122101763a - core[a373ad73a504971]::fmt::write [INFO] [stderr] 2: 0x5c12210056ec - ::write_fmt [INFO] [stderr] 3: 0x5c1220fe01d6 - std[617be2ca0a6fd877]::panicking::default_hook::{closure#0} [INFO] [stderr] 4: 0x5c1220ff6d41 - std[617be2ca0a6fd877]::panicking::default_hook [INFO] [stderr] 5: 0x5c1220ff706b - std[617be2ca0a6fd877]::panicking::panic_with_hook [INFO] [stderr] 6: 0x5c1220fe0282 - std[617be2ca0a6fd877]::panicking::panic_handler::{closure#0} [INFO] [stderr] 7: 0x5c1220fd8cf9 - std[617be2ca0a6fd877]::sys::backtrace::__rust_end_short_backtrace:: [INFO] [stderr] 8: 0x5c1220fe103d - __rustc[ea4f61d10b84890b]::rust_begin_unwind [INFO] [stderr] 9: 0x5c1221017e1c - core[a373ad73a504971]::panicking::panic_fmt [INFO] [stderr] 10: 0x5c1221017bf2 - core[a373ad73a504971]::result::unwrap_failed [INFO] [stderr] 11: 0x5c1220c2440f - melin_server_runtime[439fe4d1bbbbe025]::response::run:: [INFO] [stderr] 12: 0x5c1220c0be1f - std[617be2ca0a6fd877]::sys::backtrace::__rust_begin_short_backtrace::::{closure#5}, ()> [INFO] [stderr] 13: 0x5c1220c17a84 - ::{closure#5}, ()>::{closure#1} as core[a373ad73a504971]::ops::function::FnOnce<()>>::call_once::{shim:vtable#0} [INFO] [stderr] 14: 0x5c1220ffe2ff - ::new::thread_start [INFO] [stderr] 15: 0x77a39d3d3dfa - [INFO] [stderr] 16: 0x77a39d4673d4 - clone [INFO] [stderr] 17: 0x0 - [INFO] [stderr] [INFO] [stdout] 2026-10-18T11:54:48.327736Z ERROR main melin_server_runtime::server: pipeline thread died, initiating shutdown [INFO] [stderr] thread 'uring-reader' (7660) panicked at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/melin-server-runtime-0.15.0/src/reader.rs:421:10: [INFO] [stderr] failed to create io_uring instance: Os { code: 1, kind: PermissionDenied, message: "Operation not permitted" } [INFO] [stderr] stack backtrace: [INFO] [stderr] 0: 0x5c1220fff9a1 - <::print::DisplayBacktrace as core[a373ad73a504971]::fmt::Display>::fmt [INFO] [stderr] 1: 0x5c122101763a - core[a373ad73a504971]::fmt::write [INFO] [stderr] 2: 0x5c12210056ec - ::write_fmt [INFO] [stderr] 3: 0x5c1220fe01d6 - std[617be2ca0a6fd877]::panicking::default_hook::{closure#0} [INFO] [stderr] 4: 0x5c1220ff6d41 - std[617be2ca0a6fd877]::panicking::default_hook [INFO] [stderr] 5: 0x5c1220ff706b - std[617be2ca0a6fd877]::panicking::panic_with_hook [INFO] [stderr] 6: 0x5c1220fe0282 - std[617be2ca0a6fd877]::panicking::panic_handler::{closure#0} [INFO] [stderr] 7: 0x5c1220fd8cf9 - std[617be2ca0a6fd877]::sys::backtrace::__rust_end_short_backtrace:: [INFO] [stderr] 8: 0x5c1220fe103d - __rustc[ea4f61d10b84890b]::rust_begin_unwind [INFO] [stderr] 9: 0x5c1221017e1c - core[a373ad73a504971]::panicking::panic_fmt [INFO] [stderr] 10: 0x5c1221017bf2 - core[a373ad73a504971]::result::unwrap_failed [INFO] [stderr] 11: 0x5c1220c34678 - melin_server_runtime[439fe4d1bbbbe025]::reader::reader_loop:: [INFO] [stderr] 12: 0x5c1220c0b495 - std[617be2ca0a6fd877]::sys::backtrace::__rust_begin_short_backtrace::::{closure#0}, ()> [INFO] [stderr] 13: 0x5c1220c172be - ::{closure#0}, ()>::{closure#1} as core[a373ad73a504971]::ops::function::FnOnce<()>>::call_once::{shim:vtable#0} [INFO] [stderr] 14: 0x5c1220ffe2ff - ::new::thread_start [INFO] [stderr] 15: 0x77a39d3d3dfa - [INFO] [stderr] 16: 0x77a39d4673d4 - clone [INFO] [stderr] 17: 0x0 - [INFO] [stdout] 2026-10-18T11:54:48.332750Z ERROR main melin_server_runtime::reader: reader thread panicked failed to create io_uring instance: Os { code: 1, kind: PermissionDenied, message: "Operation not permitted" } [INFO] [stdout] 2026-10-18T11:54:48.866495Z ERROR replica-receiver melin_server_runtime::replication::tcp_receiver: UringTransport init failed error=io_uring init failed: Operation not permitted (os error 1) [INFO] [stderr] Error: "pipeline failure" [INFO] [stdout] 2026-10-18T11:54:49.270590Z ERROR main melin_server_runtime::server: pipeline thread panicked thread="response" failed to create io_uring instance for response stage: Os { code: 1, kind: PermissionDenied, message: "Operation not permitted" } [INFO] [stdout] 2026-10-18T11:54:49.270690Z ERROR main melin_server_runtime::server: shutdown complete (with pipeline failure) [INFO] [stdout] test recovered_primary_ack_gate_holds has been running for over 60 seconds [INFO] [stdout] test kill_primary_promote_replica_no_data_loss ... FAILED [INFO] [stderr] [INFO] [stderr] thread 'response' (7672) panicked at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/melin-server-runtime-0.15.0/src/response.rs:300:10: [INFO] [stderr] failed to create io_uring instance for response stage: Os { code: 1, kind: PermissionDenied, message: "Operation not permitted" } [INFO] [stderr] stack backtrace: [INFO] [stderr] 0: 0x5eb78b7079a1 - <::print::DisplayBacktrace as core[a373ad73a504971]::fmt::Display>::fmt [INFO] [stderr] 1: 0x5eb78b71f63a - core[a373ad73a504971]::fmt::write [INFO] [stderr] 2: 0x5eb78b70d6ec - ::write_fmt [INFO] [stderr] 3: 0x5eb78b6e81d6 - std[617be2ca0a6fd877]::panicking::default_hook::{closure#0} [INFO] [stderr] 4: 0x5eb78b6fed41 - std[617be2ca0a6fd877]::panicking::default_hook [INFO] [stderr] 5: 0x5eb78b6ff06b - std[617be2ca0a6fd877]::panicking::panic_with_hook [INFO] [stderr] 6: 0x5eb78b6e8282 - std[617be2ca0a6fd877]::panicking::panic_handler::{closure#0} [INFO] [stderr] 7: 0x5eb78b6e0cf9 - std[617be2ca0a6fd877]::sys::backtrace::__rust_end_short_backtrace:: [INFO] [stderr] 8: 0x5eb78b6e903d - __rustc[ea4f61d10b84890b]::rust_begin_unwind [INFO] [stderr] 9: 0x5eb78b71fe1c - core[a373ad73a504971]::panicking::panic_fmt [INFO] [stderr] 10: 0x5eb78b71fbf2 - core[a373ad73a504971]::result::unwrap_failed [INFO] [stderr] 11: 0x5eb78b32c40f - melin_server_runtime[439fe4d1bbbbe025]::response::run:: [INFO] [stderr] 12: 0x5eb78b313e1f - std[617be2ca0a6fd877]::sys::backtrace::__rust_begin_short_backtrace::::{closure#5}, ()> [INFO] [stderr] 13: 0x5eb78b31fa84 - ::{closure#5}, ()>::{closure#1} as core[a373ad73a504971]::ops::function::FnOnce<()>>::call_once::{shim:vtable#0} [INFO] [stderr] 14: 0x5eb78b7062ff - ::new::thread_start [INFO] [stderr] 15: 0x7ece7d018dfa - [INFO] [stderr] 16: 0x7ece7d0ac3d4 - clone [INFO] [stderr] 17: 0x0 - [INFO] [stdout] 2026-10-18T11:55:19.567677Z ERROR main melin_server_runtime::server: pipeline thread died, initiating shutdown [INFO] [stderr] [INFO] [stdout] 2026-10-18T11:55:19.569855Z ERROR main melin_server_runtime::reader: reader thread panicked failed to create io_uring instance: Os { code: 1, kind: PermissionDenied, message: "Operation not permitted" } [INFO] [stderr] thread 'uring-reader' (7684) panicked at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/melin-server-runtime-0.15.0/src/reader.rs:421:10: [INFO] [stderr] failed to create io_uring instance: Os { code: 1, kind: PermissionDenied, message: "Operation not permitted" } [INFO] [stderr] stack backtrace: [INFO] [stderr] 0: 0x5eb78b7079a1 - <::print::DisplayBacktrace as core[a373ad73a504971]::fmt::Display>::fmt [INFO] [stderr] 1: 0x5eb78b71f63a - core[a373ad73a504971]::fmt::write [INFO] [stderr] 2: 0x5eb78b70d6ec - ::write_fmt [INFO] [stderr] 3: 0x5eb78b6e81d6 - std[617be2ca0a6fd877]::panicking::default_hook::{closure#0} [INFO] [stderr] 4: 0x5eb78b6fed41 - std[617be2ca0a6fd877]::panicking::default_hook [INFO] [stderr] 5: 0x5eb78b6ff06b - std[617be2ca0a6fd877]::panicking::panic_with_hook [INFO] [stderr] 6: 0x5eb78b6e8282 - std[617be2ca0a6fd877]::panicking::panic_handler::{closure#0} [INFO] [stderr] 7: 0x5eb78b6e0cf9 - std[617be2ca0a6fd877]::sys::backtrace::__rust_end_short_backtrace:: [INFO] [stderr] 8: 0x5eb78b6e903d - __rustc[ea4f61d10b84890b]::rust_begin_unwind [INFO] [stderr] 9: 0x5eb78b71fe1c - core[a373ad73a504971]::panicking::panic_fmt [INFO] [stderr] 10: 0x5eb78b71fbf2 - core[a373ad73a504971]::result::unwrap_failed [INFO] [stderr] 11: 0x5eb78b33c678 - melin_server_runtime[439fe4d1bbbbe025]::reader::reader_loop:: [INFO] [stderr] 12: 0x5eb78b313495 - std[617be2ca0a6fd877]::sys::backtrace::__rust_begin_short_backtrace::::{closure#0}, ()> [INFO] [stderr] 13: 0x5eb78b31f2be - ::{closure#0}, ()>::{closure#1} as core[a373ad73a504971]::ops::function::FnOnce<()>>::call_once::{shim:vtable#0} [INFO] [stderr] 14: 0x5eb78b7062ff - ::new::thread_start [INFO] [stderr] 15: 0x7ece7d018dfa - [INFO] [stderr] 16: 0x7ece7d0ac3d4 - clone [INFO] [stderr] 17: 0x0 - [INFO] [stdout] 2026-10-18T11:55:20.250747Z ERROR replica-receiver melin_server_runtime::replication::tcp_receiver: UringTransport init failed error=io_uring init failed: Operation not permitted (os error 1) [INFO] [stdout] 2026-10-18T11:55:20.351845Z ERROR replica-receiver melin_server_runtime::replication::tcp_receiver: UringTransport init failed error=io_uring init failed: Operation not permitted (os error 1) [INFO] [stderr] Error: "pipeline failure" [INFO] [stdout] 2026-10-18T11:55:20.569812Z ERROR main melin_server_runtime::server: pipeline thread panicked thread="response" failed to create io_uring instance for response stage: Os { code: 1, kind: PermissionDenied, message: "Operation not permitted" } [INFO] [stdout] 2026-10-18T11:55:20.569847Z ERROR main melin_server_runtime::server: shutdown complete (with pipeline failure) [INFO] [stdout] test replacement_replica_catches_up_from_journal has been running for over 60 seconds [INFO] [stdout] test dual_replication_with_fills_then_failover ... FAILED [INFO] [stderr] [INFO] [stderr] thread 'response' (7703) panicked at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/melin-server-runtime-0.15.0/src/response.rs:300:10: [INFO] [stderr] failed to create io_uring instance for response stage: Os { code: 1, kind: PermissionDenied, message: "Operation not permitted" } [INFO] [stderr] stack backtrace: [INFO] [stderr] 0: 0x5c5f6f56b9a1 - <::print::DisplayBacktrace as core[a373ad73a504971]::fmt::Display>::fmt [INFO] [stderr] 1: 0x5c5f6f58363a - core[a373ad73a504971]::fmt::write [INFO] [stderr] 2: 0x5c5f6f5716ec - ::write_fmt [INFO] [stderr] 3: 0x5c5f6f54c1d6 - std[617be2ca0a6fd877]::panicking::default_hook::{closure#0} [INFO] [stderr] 4: 0x5c5f6f562d41 - std[617be2ca0a6fd877]::panicking::default_hook [INFO] [stderr] 5: 0x5c5f6f56306b - std[617be2ca0a6fd877]::panicking::panic_with_hook [INFO] [stderr] 6: 0x5c5f6f54c282 - std[617be2ca0a6fd877]::panicking::panic_handler::{closure#0} [INFO] [stderr] 7: 0x5c5f6f544cf9 - std[617be2ca0a6fd877]::sys::backtrace::__rust_end_short_backtrace:: [INFO] [stderr] 8: 0x5c5f6f54d03d - __rustc[ea4f61d10b84890b]::rust_begin_unwind [INFO] [stderr] 9: 0x5c5f6f583e1c - core[a373ad73a504971]::panicking::panic_fmt [INFO] [stderr] 10: 0x5c5f6f583bf2 - core[a373ad73a504971]::result::unwrap_failed [INFO] [stderr] 11: 0x5c5f6f19040f - melin_server_runtime[439fe4d1bbbbe025]::response::run:: [INFO] [stderr] 12: 0x5c5f6f177e1f - std[617be2ca0a6fd877]::sys::backtrace::__rust_begin_short_backtrace::::{closure#5}, ()> [INFO] [stderr] 13: 0x5c5f6f183a84 - ::{closure#5}, ()>::{closure#1} as core[a373ad73a504971]::ops::function::FnOnce<()>>::call_once::{shim:vtable#0} [INFO] [stderr] 14: 0x5c5f6f56a2ff - ::new::thread_start [INFO] [stderr] 15: 0x737e5da9cdfa - [INFO] [stderr] 16: 0x737e5db303d4 - clone [INFO] [stderr] 17: 0x0 - [INFO] [stdout] 2026-10-18T11:55:51.255673Z ERROR main melin_server_runtime::server: pipeline thread died, initiating shutdown [INFO] [stderr] [INFO] [stderr] thread 'uring-reader' (7712) panicked at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/melin-server-runtime-0.15.0/src/reader.rs:421:10: [INFO] [stderr] failed to create io_uring instance: Os { code: 1, kind: PermissionDenied, message: "Operation not permitted" } [INFO] [stderr] stack backtrace: [INFO] [stderr] 0: 0x5c5f6f56b9a1 - <::print::DisplayBacktrace as core[a373ad73a504971]::fmt::Display>::fmt [INFO] [stderr] 1: 0x5c5f6f58363a - core[a373ad73a504971]::fmt::write [INFO] [stderr] 2: 0x5c5f6f5716ec - ::write_fmt [INFO] [stderr] 3: 0x5c5f6f54c1d6 - std[617be2ca0a6fd877]::panicking::default_hook::{closure#0} [INFO] [stderr] 4: 0x5c5f6f562d41 - std[617be2ca0a6fd877]::panicking::default_hook [INFO] [stderr] 5: 0x5c5f6f56306b - std[617be2ca0a6fd877]::panicking::panic_with_hook [INFO] [stderr] 6: 0x5c5f6f54c282 - std[617be2ca0a6fd877]::panicking::panic_handler::{closure#0} [INFO] [stderr] 7: 0x5c5f6f544cf9 - std[617be2ca0a6fd877]::sys::backtrace::__rust_end_short_backtrace:: [INFO] [stderr] 8: 0x5c5f6f54d03d - __rustc[ea4f61d10b84890b]::rust_begin_unwind [INFO] [stderr] 9: 0x5c5f6f583e1c - core[a373ad73a504971]::panicking::panic_fmt [INFO] [stderr] 10: 0x5c5f6f583bf2 - core[a373ad73a504971]::result::unwrap_failed [INFO] [stderr] 11: 0x5c5f6f1a0678 - melin_server_runtime[439fe4d1bbbbe025]::reader::reader_loop:: [INFO] [stderr] 12: 0x5c5f6f177495 - std[617be2ca0a6fd877]::sys::backtrace::__rust_begin_short_backtrace::::{closure#0}, ()> [INFO] [stderr] 13: 0x5c5f6f1832be - ::{closure#0}, ()>::{closure#1} as core[a373ad73a504971]::ops::function::FnOnce<()>>::call_once::{shim:vtable#0} [INFO] [stderr] 14: 0x5c5f6f56a2ff - ::new::thread_start [INFO] [stderr] 15: 0x737e5da9cdfa - [INFO] [stderr] 16: 0x737e5db303d4 - clone [INFO] [stderr] 17: 0x0 - [INFO] [stdout] 2026-10-18T11:55:51.274647Z ERROR main melin_server_runtime::reader: reader thread panicked failed to create io_uring instance: Os { code: 1, kind: PermissionDenied, message: "Operation not permitted" } [INFO] [stdout] 2026-10-18T11:55:51.724829Z ERROR replica-receiver melin_server_runtime::replication::tcp_receiver: UringTransport init failed error=io_uring init failed: Operation not permitted (os error 1) [INFO] [stdout] 2026-10-18T11:55:52.189876Z ERROR main melin_server_runtime::server: pipeline thread panicked thread="response" failed to create io_uring instance for response stage: Os { code: 1, kind: PermissionDenied, message: "Operation not permitted" } [INFO] [stdout] 2026-10-18T11:55:52.189991Z ERROR main melin_server_runtime::server: shutdown complete (with pipeline failure) [INFO] [stderr] Error: "pipeline failure" [INFO] [stdout] test replica_reconnects_after_primary_restart_without_journal_wipe has been running for over 60 seconds [INFO] [stdout] test disk_ram_gate_stalls_while_replica_frozen ... FAILED [INFO] [stderr] [INFO] [stderr] thread 'response' (7724) panicked at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/melin-server-runtime-0.15.0/src/response.rs:300:10: [INFO] [stderr] failed to create io_uring instance for response stage: Os { code: 1, kind: PermissionDenied, message: "Operation not permitted" } [INFO] [stderr] stack backtrace: [INFO] [stderr] 0: 0x5ecf30da89a1 - <::print::DisplayBacktrace as core[a373ad73a504971]::fmt::Display>::fmt [INFO] [stderr] 1: 0x5ecf30dc063a - core[a373ad73a504971]::fmt::write [INFO] [stderr] 2: 0x5ecf30dae6ec - ::write_fmt [INFO] [stderr] 3: 0x5ecf30d891d6 - std[617be2ca0a6fd877]::panicking::default_hook::{closure#0} [INFO] [stderr] 4: 0x5ecf30d9fd41 - std[617be2ca0a6fd877]::panicking::default_hook [INFO] [stderr] 5: 0x5ecf30da006b - std[617be2ca0a6fd877]::panicking::panic_with_hook [INFO] [stderr] 6: 0x5ecf30d89282 - std[617be2ca0a6fd877]::panicking::panic_handler::{closure#0} [INFO] [stderr] 7: 0x5ecf30d81cf9 - std[617be2ca0a6fd877]::sys::backtrace::__rust_end_short_backtrace:: [INFO] [stderr] 8: 0x5ecf30d8a03d - __rustc[ea4f61d10b84890b]::rust_begin_unwind [INFO] [stderr] 9: 0x5ecf30dc0e1c - core[a373ad73a504971]::panicking::panic_fmt [INFO] [stderr] 10: 0x5ecf30dc0bf2 - core[a373ad73a504971]::result::unwrap_failed [INFO] [stderr] 11: 0x5ecf309cd40f - melin_server_runtime[439fe4d1bbbbe025]::response::run:: [INFO] [stderr] 12: 0x5ecf309b4e1f - std[617be2ca0a6fd877]::sys::backtrace::__rust_begin_short_backtrace::::{closure#5}, ()> [INFO] [stderr] 13: 0x5ecf309c0a84 - ::{closure#5}, ()>::{closure#1} as core[a373ad73a504971]::ops::function::FnOnce<()>>::call_once::{shim:vtable#0} [INFO] [stderr] 14: 0x5ecf30da72ff - ::new::thread_start [INFO] [stderr] 15: 0x7191d120adfa - [INFO] [stderr] 16: 0x7191d129e3d4 - clone [INFO] [stderr] 17: 0x0 - [INFO] [stdout] 2026-10-18T11:56:22.214731Z ERROR main melin_server_runtime::server: pipeline thread died, initiating shutdown [INFO] [stderr] [INFO] [stderr] thread 'uring-reader' (7733) panicked at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/melin-server-runtime-0.15.0/src/reader.rs:421:10: [INFO] [stderr] failed to create io_uring instance: Os { code: 1, kind: PermissionDenied, message: "Operation not permitted" } [INFO] [stderr] stack backtrace: [INFO] [stdout] 2026-10-18T11:56:22.223537Z ERROR main melin_server_runtime::reader: reader thread panicked failed to create io_uring instance: Os { code: 1, kind: PermissionDenied, message: "Operation not permitted" } [INFO] [stderr] 0: 0x5ecf30da89a1 - <::print::DisplayBacktrace as core[a373ad73a504971]::fmt::Display>::fmt [INFO] [stderr] 1: 0x5ecf30dc063a - core[a373ad73a504971]::fmt::write [INFO] [stderr] 2: 0x5ecf30dae6ec - ::write_fmt [INFO] [stderr] 3: 0x5ecf30d891d6 - std[617be2ca0a6fd877]::panicking::default_hook::{closure#0} [INFO] [stderr] 4: 0x5ecf30d9fd41 - std[617be2ca0a6fd877]::panicking::default_hook [INFO] [stderr] 5: 0x5ecf30da006b - std[617be2ca0a6fd877]::panicking::panic_with_hook [INFO] [stderr] 6: 0x5ecf30d89282 - std[617be2ca0a6fd877]::panicking::panic_handler::{closure#0} [INFO] [stderr] 7: 0x5ecf30d81cf9 - std[617be2ca0a6fd877]::sys::backtrace::__rust_end_short_backtrace:: [INFO] [stderr] 8: 0x5ecf30d8a03d - __rustc[ea4f61d10b84890b]::rust_begin_unwind [INFO] [stderr] 9: 0x5ecf30dc0e1c - core[a373ad73a504971]::panicking::panic_fmt [INFO] [stderr] 10: 0x5ecf30dc0bf2 - core[a373ad73a504971]::result::unwrap_failed [INFO] [stderr] 11: 0x5ecf309dd678 - melin_server_runtime[439fe4d1bbbbe025]::reader::reader_loop:: [INFO] [stderr] 12: 0x5ecf309b4495 - std[617be2ca0a6fd877]::sys::backtrace::__rust_begin_short_backtrace::::{closure#0}, ()> [INFO] [stderr] 13: 0x5ecf309c02be - ::{closure#0}, ()>::{closure#1} as core[a373ad73a504971]::ops::function::FnOnce<()>>::call_once::{shim:vtable#0} [INFO] [stderr] 14: 0x5ecf30da72ff - ::new::thread_start [INFO] [stderr] 15: 0x7191d120adfa - [INFO] [stderr] 16: 0x7191d129e3d4 - clone [INFO] [stderr] 17: 0x0 - [INFO] [stdout] 2026-10-18T11:56:22.652828Z ERROR replica-receiver melin_server_runtime::replication::tcp_receiver: UringTransport init failed error=io_uring init failed: Operation not permitted (os error 1) [INFO] [stdout] 2026-10-18T11:56:23.459072Z ERROR main melin_server_runtime::server: pipeline thread panicked thread="response" failed to create io_uring instance for response stage: Os { code: 1, kind: PermissionDenied, message: "Operation not permitted" } [INFO] [stdout] 2026-10-18T11:56:23.459180Z ERROR main melin_server_runtime::server: shutdown complete (with pipeline failure) [INFO] [stderr] Error: "pipeline failure" [INFO] [stdout] test rotation_soak_under_load has been running for over 60 seconds [INFO] [stdout] test recovered_primary_ack_gate_holds ... FAILED [INFO] [stderr] [INFO] [stderr] thread 'response' (7745) panicked at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/melin-server-runtime-0.15.0/src/response.rs:300:10: [INFO] [stderr] failed to create io_uring instance for response stage: Os { code: 1, kind: PermissionDenied, message: "Operation not permitted" } [INFO] [stderr] stack backtrace: [INFO] [stderr] 0: 0x5b307641b9a1 - <::print::DisplayBacktrace as core[a373ad73a504971]::fmt::Display>::fmt [INFO] [stderr] 1: 0x5b307643363a - core[a373ad73a504971]::fmt::write [INFO] [stderr] 2: 0x5b30764216ec - ::write_fmt [INFO] [stderr] 3: 0x5b30763fc1d6 - std[617be2ca0a6fd877]::panicking::default_hook::{closure#0} [INFO] [stderr] 4: 0x5b3076412d41 - std[617be2ca0a6fd877]::panicking::default_hook [INFO] [stderr] 5: 0x5b307641306b - std[617be2ca0a6fd877]::panicking::panic_with_hook [INFO] [stderr] 6: 0x5b30763fc282 - std[617be2ca0a6fd877]::panicking::panic_handler::{closure#0} [INFO] [stderr] 7: 0x5b30763f4cf9 - std[617be2ca0a6fd877]::sys::backtrace::__rust_end_short_backtrace:: [INFO] [stderr] 8: 0x5b30763fd03d - __rustc[ea4f61d10b84890b]::rust_begin_unwind [INFO] [stderr] 9: 0x5b3076433e1c - core[a373ad73a504971]::panicking::panic_fmt [INFO] [stderr] 10: 0x5b3076433bf2 - core[a373ad73a504971]::result::unwrap_failed [INFO] [stderr] 11: 0x5b307604040f - melin_server_runtime[439fe4d1bbbbe025]::response::run:: [INFO] [stderr] 12: 0x5b3076027e1f - std[617be2ca0a6fd877]::sys::backtrace::__rust_begin_short_backtrace::::{closure#5}, ()> [INFO] [stderr] 13: 0x5b3076033a84 - ::{closure#5}, ()>::{closure#1} as core[a373ad73a504971]::ops::function::FnOnce<()>>::call_once::{shim:vtable#0} [INFO] [stderr] 14: 0x5b307641a2ff - ::new::thread_start [INFO] [stderr] 15: 0x79f243cc0dfa - [INFO] [stderr] 16: 0x79f243d543d4 - clone [INFO] [stderr] 17: 0x0 - [INFO] [stdout] 2026-10-18T11:56:53.809767Z ERROR main melin_server_runtime::server: pipeline thread died, initiating shutdown [INFO] [stdout] 2026-10-18T11:56:53.811999Z ERROR main melin_server_runtime::reader: reader thread panicked failed to create io_uring instance: Os { code: 1, kind: PermissionDenied, message: "Operation not permitted" } [INFO] [stderr] [INFO] [stderr] thread 'uring-reader' (7757) panicked at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/melin-server-runtime-0.15.0/src/reader.rs:421:10: [INFO] [stderr] failed to create io_uring instance: Os { code: 1, kind: PermissionDenied, message: "Operation not permitted" } [INFO] [stderr] stack backtrace: [INFO] [stderr] 0: 0x5b307641b9a1 - <::print::DisplayBacktrace as core[a373ad73a504971]::fmt::Display>::fmt [INFO] [stderr] 1: 0x5b307643363a - core[a373ad73a504971]::fmt::write [INFO] [stderr] 2: 0x5b30764216ec - ::write_fmt [INFO] [stderr] 3: 0x5b30763fc1d6 - std[617be2ca0a6fd877]::panicking::default_hook::{closure#0} [INFO] [stderr] 4: 0x5b3076412d41 - std[617be2ca0a6fd877]::panicking::default_hook [INFO] [stderr] 5: 0x5b307641306b - std[617be2ca0a6fd877]::panicking::panic_with_hook [INFO] [stderr] 6: 0x5b30763fc282 - std[617be2ca0a6fd877]::panicking::panic_handler::{closure#0} [INFO] [stderr] 7: 0x5b30763f4cf9 - std[617be2ca0a6fd877]::sys::backtrace::__rust_end_short_backtrace:: [INFO] [stderr] 8: 0x5b30763fd03d - __rustc[ea4f61d10b84890b]::rust_begin_unwind [INFO] [stderr] 9: 0x5b3076433e1c - core[a373ad73a504971]::panicking::panic_fmt [INFO] [stderr] 10: 0x5b3076433bf2 - core[a373ad73a504971]::result::unwrap_failed [INFO] [stderr] 11: 0x5b3076050678 - melin_server_runtime[439fe4d1bbbbe025]::reader::reader_loop:: [INFO] [stderr] 12: 0x5b3076027495 - std[617be2ca0a6fd877]::sys::backtrace::__rust_begin_short_backtrace::::{closure#0}, ()> [INFO] [stderr] 13: 0x5b30760332be - ::{closure#0}, ()>::{closure#1} as core[a373ad73a504971]::ops::function::FnOnce<()>>::call_once::{shim:vtable#0} [INFO] [stderr] 14: 0x5b307641a2ff - ::new::thread_start [INFO] [stderr] 15: 0x79f243cc0dfa - [INFO] [stderr] 16: 0x79f243d543d4 - clone [INFO] [stderr] 17: 0x0 - [INFO] [stdout] 2026-10-18T11:56:54.412289Z ERROR replica-receiver melin_server_runtime::replication::tcp_receiver: UringTransport init failed error=io_uring init failed: Operation not permitted (os error 1) [INFO] [stdout] 2026-10-18T11:56:55.420988Z ERROR replica-receiver melin_server_runtime::replication::tcp_receiver: UringTransport init failed error=io_uring init failed: Operation not permitted (os error 1) [INFO] [stdout] test same_key_request_seq_hwm_survives_failover has been running for over 60 seconds [INFO] [stdout] test fresh_replica_full_catchup ... FAILED [INFO] [stderr] [INFO] [stderr] thread 'response' (7776) panicked at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/melin-server-runtime-0.15.0/src/response.rs:300:10: [INFO] [stderr] failed to create io_uring instance for response stage: Os { code: 1, kind: PermissionDenied, message: "Operation not permitted" } [INFO] [stderr] stack backtrace: [INFO] [stderr] 0: 0x5952184df9a1 - <::print::DisplayBacktrace as core[a373ad73a504971]::fmt::Display>::fmt [INFO] [stderr] 1: 0x5952184f763a - core[a373ad73a504971]::fmt::write [INFO] [stderr] 2: 0x5952184e56ec - ::write_fmt [INFO] [stderr] 3: 0x5952184c01d6 - std[617be2ca0a6fd877]::panicking::default_hook::{closure#0} [INFO] [stderr] 4: 0x5952184d6d41 - std[617be2ca0a6fd877]::panicking::default_hook [INFO] [stderr] 5: 0x5952184d706b - std[617be2ca0a6fd877]::panicking::panic_with_hook [INFO] [stderr] 6: 0x5952184c0282 - std[617be2ca0a6fd877]::panicking::panic_handler::{closure#0} [INFO] [stderr] 7: 0x5952184b8cf9 - std[617be2ca0a6fd877]::sys::backtrace::__rust_end_short_backtrace:: [INFO] [stderr] 8: 0x5952184c103d - __rustc[ea4f61d10b84890b]::rust_begin_unwind [INFO] [stderr] 9: 0x5952184f7e1c - core[a373ad73a504971]::panicking::panic_fmt [INFO] [stderr] 10: 0x5952184f7bf2 - core[a373ad73a504971]::result::unwrap_failed [INFO] [stderr] 11: 0x59521810440f - melin_server_runtime[439fe4d1bbbbe025]::response::run:: [INFO] [stderr] 12: 0x5952180ebe1f - std[617be2ca0a6fd877]::sys::backtrace::__rust_begin_short_backtrace::::{closure#5}, ()> [INFO] [stderr] 13: 0x5952180f7a84 - ::{closure#5}, ()>::{closure#1} as core[a373ad73a504971]::ops::function::FnOnce<()>>::call_once::{shim:vtable#0} [INFO] [stderr] 14: 0x5952184de2ff - ::new::thread_start [INFO] [stderr] 15: 0x7fd53adbddfa - [INFO] [stderr] 16: 0x7fd53ae513d4 - clone [INFO] [stderr] 17: 0x0 - [INFO] [stdout] 2026-10-18T11:57:25.161644Z ERROR main melin_server_runtime::server: pipeline thread died, initiating shutdown [INFO] [stdout] 2026-10-18T11:57:25.164888Z ERROR main melin_server_runtime::reader: reader thread panicked failed to create io_uring instance: Os { code: 1, kind: PermissionDenied, message: "Operation not permitted" } [INFO] [stderr] [INFO] [stderr] thread 'uring-reader' (7788) panicked at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/melin-server-runtime-0.15.0/src/reader.rs:421:10: [INFO] [stderr] failed to create io_uring instance: Os { code: 1, kind: PermissionDenied, message: "Operation not permitted" } [INFO] [stderr] stack backtrace: [INFO] [stderr] 0: 0x5952184df9a1 - <::print::DisplayBacktrace as core[a373ad73a504971]::fmt::Display>::fmt [INFO] [stderr] 1: 0x5952184f763a - core[a373ad73a504971]::fmt::write [INFO] [stderr] 2: 0x5952184e56ec - ::write_fmt [INFO] [stderr] 3: 0x5952184c01d6 - std[617be2ca0a6fd877]::panicking::default_hook::{closure#0} [INFO] [stderr] 4: 0x5952184d6d41 - std[617be2ca0a6fd877]::panicking::default_hook [INFO] [stderr] 5: 0x5952184d706b - std[617be2ca0a6fd877]::panicking::panic_with_hook [INFO] [stderr] 6: 0x5952184c0282 - std[617be2ca0a6fd877]::panicking::panic_handler::{closure#0} [INFO] [stderr] 7: 0x5952184b8cf9 - std[617be2ca0a6fd877]::sys::backtrace::__rust_end_short_backtrace:: [INFO] [stderr] 8: 0x5952184c103d - __rustc[ea4f61d10b84890b]::rust_begin_unwind [INFO] [stderr] 9: 0x5952184f7e1c - core[a373ad73a504971]::panicking::panic_fmt [INFO] [stderr] 10: 0x5952184f7bf2 - core[a373ad73a504971]::result::unwrap_failed [INFO] [stderr] 11: 0x595218114678 - melin_server_runtime[439fe4d1bbbbe025]::reader::reader_loop:: [INFO] [stderr] 12: 0x5952180eb495 - std[617be2ca0a6fd877]::sys::backtrace::__rust_begin_short_backtrace::::{closure#0}, ()> [INFO] [stderr] 13: 0x5952180f72be - ::{closure#0}, ()>::{closure#1} as core[a373ad73a504971]::ops::function::FnOnce<()>>::call_once::{shim:vtable#0} [INFO] [stderr] 14: 0x5952184de2ff - ::new::thread_start [INFO] [stderr] 15: 0x7fd53adbddfa - [INFO] [stderr] 16: 0x7fd53ae513d4 - clone [INFO] [stderr] 17: 0x0 - [INFO] [stdout] 2026-10-18T11:57:25.587844Z ERROR replica-receiver melin_server_runtime::replication::tcp_receiver: UringTransport init failed error=io_uring init failed: Operation not permitted (os error 1) [INFO] [stdout] 2026-10-18T11:57:25.752821Z ERROR replica-receiver melin_server_runtime::replication::tcp_receiver: UringTransport init failed error=io_uring init failed: Operation not permitted (os error 1) [INFO] [stdout] 2026-10-18T11:57:26.080828Z ERROR main melin_server_runtime::server: pipeline thread panicked thread="response" failed to create io_uring instance for response stage: Os { code: 1, kind: PermissionDenied, message: "Operation not permitted" } [INFO] [stderr] Error: "pipeline failure" [INFO] [stdout] 2026-10-18T11:57:26.080926Z ERROR main melin_server_runtime::server: shutdown complete (with pipeline failure) [INFO] [stdout] test sec04_rate_limit_replicates_to_replica has been running for over 60 seconds [INFO] [stdout] test dual_replication_halts_when_both_disconnect ... FAILED [INFO] [stderr] [INFO] [stderr] thread 'response' (7807) panicked at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/melin-server-runtime-0.15.0/src/response.rs:300:10: [INFO] [stderr] failed to create io_uring instance for response stage: Os { code: 1, kind: PermissionDenied, message: "Operation not permitted" } [INFO] [stderr] stack backtrace: [INFO] [stderr] 0: 0x581a01e1b9a1 - <::print::DisplayBacktrace as core[a373ad73a504971]::fmt::Display>::fmt [INFO] [stderr] 1: 0x581a01e3363a - core[a373ad73a504971]::fmt::write [INFO] [stderr] 2: 0x581a01e216ec - ::write_fmt [INFO] [stderr] 3: 0x581a01dfc1d6 - std[617be2ca0a6fd877]::panicking::default_hook::{closure#0} [INFO] [stderr] 4: 0x581a01e12d41 - std[617be2ca0a6fd877]::panicking::default_hook [INFO] [stderr] 5: 0x581a01e1306b - std[617be2ca0a6fd877]::panicking::panic_with_hook [INFO] [stderr] 6: 0x581a01dfc282 - std[617be2ca0a6fd877]::panicking::panic_handler::{closure#0} [INFO] [stderr] 7: 0x581a01df4cf9 - std[617be2ca0a6fd877]::sys::backtrace::__rust_end_short_backtrace:: [INFO] [stderr] 8: 0x581a01dfd03d - __rustc[ea4f61d10b84890b]::rust_begin_unwind [INFO] [stderr] 9: 0x581a01e33e1c - core[a373ad73a504971]::panicking::panic_fmt [INFO] [stderr] 10: 0x581a01e33bf2 - core[a373ad73a504971]::result::unwrap_failed [INFO] [stderr] 11: 0x581a01a4040f - melin_server_runtime[439fe4d1bbbbe025]::response::run:: [INFO] [stderr] 12: 0x581a01a27e1f - std[617be2ca0a6fd877]::sys::backtrace::__rust_begin_short_backtrace::::{closure#5}, ()> [INFO] [stderr] 13: 0x581a01a33a84 - ::{closure#5}, ()>::{closure#1} as core[a373ad73a504971]::ops::function::FnOnce<()>>::call_once::{shim:vtable#0} [INFO] [stderr] 14: 0x581a01e1a2ff - ::new::thread_start [INFO] [stderr] 15: 0x79b492bc2dfa - [INFO] [stderr] 16: 0x79b492c563d4 - clone [INFO] [stderr] 17: 0x0 - [INFO] [stderr] [INFO] [stdout] 2026-10-18T11:57:56.576692Z ERROR main melin_server_runtime::server: pipeline thread died, initiating shutdown [INFO] [stderr] thread 'uring-reader' (7815) panicked at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/melin-server-runtime-0.15.0/src/reader.rs:421:10: [INFO] [stdout] 2026-10-18T11:57:56.577867Z ERROR main melin_server_runtime::reader: reader thread panicked failed to create io_uring instance: Os { code: 1, kind: PermissionDenied, message: "Operation not permitted" } [INFO] [stderr] failed to create io_uring instance: Os { code: 1, kind: PermissionDenied, message: "Operation not permitted" } [INFO] [stderr] stack backtrace: [INFO] [stderr] 0: 0x581a01e1b9a1 - <::print::DisplayBacktrace as core[a373ad73a504971]::fmt::Display>::fmt [INFO] [stderr] 1: 0x581a01e3363a - core[a373ad73a504971]::fmt::write [INFO] [stderr] 2: 0x581a01e216ec - ::write_fmt [INFO] [stderr] 3: 0x581a01dfc1d6 - std[617be2ca0a6fd877]::panicking::default_hook::{closure#0} [INFO] [stderr] 4: 0x581a01e12d41 - std[617be2ca0a6fd877]::panicking::default_hook [INFO] [stderr] 5: 0x581a01e1306b - std[617be2ca0a6fd877]::panicking::panic_with_hook [INFO] [stderr] 6: 0x581a01dfc282 - std[617be2ca0a6fd877]::panicking::panic_handler::{closure#0} [INFO] [stderr] 7: 0x581a01df4cf9 - std[617be2ca0a6fd877]::sys::backtrace::__rust_end_short_backtrace:: [INFO] [stderr] 8: 0x581a01dfd03d - __rustc[ea4f61d10b84890b]::rust_begin_unwind [INFO] [stderr] 9: 0x581a01e33e1c - core[a373ad73a504971]::panicking::panic_fmt [INFO] [stderr] 10: 0x581a01e33bf2 - core[a373ad73a504971]::result::unwrap_failed [INFO] [stderr] 11: 0x581a01a50678 - melin_server_runtime[439fe4d1bbbbe025]::reader::reader_loop:: [INFO] [stderr] 12: 0x581a01a27495 - std[617be2ca0a6fd877]::sys::backtrace::__rust_begin_short_backtrace::::{closure#0}, ()> [INFO] [stderr] 13: 0x581a01a332be - ::{closure#0}, ()>::{closure#1} as core[a373ad73a504971]::ops::function::FnOnce<()>>::call_once::{shim:vtable#0} [INFO] [stderr] 14: 0x581a01e1a2ff - ::new::thread_start [INFO] [stderr] 15: 0x79b492bc2dfa - [INFO] [stderr] 16: 0x79b492c563d4 - clone [INFO] [stderr] 17: 0x0 - [INFO] [stdout] 2026-10-18T11:57:56.707856Z ERROR main melin_server_runtime::server: pipeline thread panicked thread="response" failed to create io_uring instance for response stage: Os { code: 1, kind: PermissionDenied, message: "Operation not permitted" } [INFO] [stdout] 2026-10-18T11:57:56.728229Z ERROR main melin_server_runtime::server: shutdown complete (with pipeline failure) [INFO] [stderr] Error: "pipeline failure" [INFO] [stdout] 2026-10-18T11:57:56.969502Z ERROR replica-receiver melin_server_runtime::replication::tcp_receiver: UringTransport init failed error=io_uring init failed: Operation not permitted (os error 1) [INFO] [stdout] test snapshot_transfer_when_archives_purged has been running for over 60 seconds [INFO] [stdout] test rotation_soak_under_load ... FAILED [INFO] [stderr] [INFO] [stderr] thread 'response' (7826) panicked at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/melin-server-runtime-0.15.0/src/response.rs:300:10: [INFO] [stderr] failed to create io_uring instance for response stage: Os { code: 1, kind: PermissionDenied, message: "Operation not permitted" } [INFO] [stderr] stack backtrace: [INFO] [stderr] 0: 0x6129924119a1 - <::print::DisplayBacktrace as core[a373ad73a504971]::fmt::Display>::fmt [INFO] [stderr] 1: 0x61299242963a - core[a373ad73a504971]::fmt::write [INFO] [stderr] 2: 0x6129924176ec - ::write_fmt [INFO] [stderr] 3: 0x6129923f21d6 - std[617be2ca0a6fd877]::panicking::default_hook::{closure#0} [INFO] [stderr] 4: 0x612992408d41 - std[617be2ca0a6fd877]::panicking::default_hook [INFO] [stderr] 5: 0x61299240906b - std[617be2ca0a6fd877]::panicking::panic_with_hook [INFO] [stderr] 6: 0x6129923f2282 - std[617be2ca0a6fd877]::panicking::panic_handler::{closure#0} [INFO] [stderr] 7: 0x6129923eacf9 - std[617be2ca0a6fd877]::sys::backtrace::__rust_end_short_backtrace:: [INFO] [stderr] 8: 0x6129923f303d - __rustc[ea4f61d10b84890b]::rust_begin_unwind [INFO] [stderr] 9: 0x612992429e1c - core[a373ad73a504971]::panicking::panic_fmt [INFO] [stderr] 10: 0x612992429bf2 - core[a373ad73a504971]::result::unwrap_failed [INFO] [stderr] 11: 0x61299203640f - melin_server_runtime[439fe4d1bbbbe025]::response::run:: [INFO] [stderr] 12: 0x61299201de1f - std[617be2ca0a6fd877]::sys::backtrace::__rust_begin_short_backtrace::::{closure#5}, ()> [INFO] [stderr] 13: 0x612992029a84 - ::{closure#5}, ()>::{closure#1} as core[a373ad73a504971]::ops::function::FnOnce<()>>::call_once::{shim:vtable#0} [INFO] [stderr] 14: 0x6129924102ff - ::new::thread_start [INFO] [stderr] 15: 0x7b69d08e4dfa - [INFO] [stderr] 16: 0x7b69d09783d4 - clone [INFO] [stderr] 17: 0x0 - [INFO] [stderr] [INFO] [stdout] 2026-10-18T11:58:27.995669Z ERROR main melin_server_runtime::server: pipeline thread died, initiating shutdown [INFO] [stderr] thread 'uring-reader' (7835) panicked at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/melin-server-runtime-0.15.0/src/reader.rs:421:10: [INFO] [stdout] 2026-10-18T11:58:27.996936Z ERROR main melin_server_runtime::reader: reader thread panicked failed to create io_uring instance: Os { code: 1, kind: PermissionDenied, message: "Operation not permitted" } [INFO] [stderr] failed to create io_uring instance: Os { code: 1, kind: PermissionDenied, message: "Operation not permitted" } [INFO] [stderr] stack backtrace: [INFO] [stderr] 0: 0x6129924119a1 - <::print::DisplayBacktrace as core[a373ad73a504971]::fmt::Display>::fmt [INFO] [stderr] 1: 0x61299242963a - core[a373ad73a504971]::fmt::write [INFO] [stderr] 2: 0x6129924176ec - ::write_fmt [INFO] [stderr] 3: 0x6129923f21d6 - std[617be2ca0a6fd877]::panicking::default_hook::{closure#0} [INFO] [stderr] 4: 0x612992408d41 - std[617be2ca0a6fd877]::panicking::default_hook [INFO] [stderr] 5: 0x61299240906b - std[617be2ca0a6fd877]::panicking::panic_with_hook [INFO] [stderr] 6: 0x6129923f2282 - std[617be2ca0a6fd877]::panicking::panic_handler::{closure#0} [INFO] [stderr] 7: 0x6129923eacf9 - std[617be2ca0a6fd877]::sys::backtrace::__rust_end_short_backtrace:: [INFO] [stderr] 8: 0x6129923f303d - __rustc[ea4f61d10b84890b]::rust_begin_unwind [INFO] [stderr] 9: 0x612992429e1c - core[a373ad73a504971]::panicking::panic_fmt [INFO] [stderr] 10: 0x612992429bf2 - core[a373ad73a504971]::result::unwrap_failed [INFO] [stderr] 11: 0x612992046678 - melin_server_runtime[439fe4d1bbbbe025]::reader::reader_loop:: [INFO] [stderr] 12: 0x61299201d495 - std[617be2ca0a6fd877]::sys::backtrace::__rust_begin_short_backtrace::::{closure#0}, ()> [INFO] [stderr] 13: 0x6129920292be - ::{closure#0}, ()>::{closure#1} as core[a373ad73a504971]::ops::function::FnOnce<()>>::call_once::{shim:vtable#0} [INFO] [stderr] 14: 0x6129924102ff - ::new::thread_start [INFO] [stderr] 15: 0x7b69d08e4dfa - [INFO] [stderr] 16: 0x7b69d09783d4 - clone [INFO] [stderr] 17: 0x0 - [INFO] [stdout] 2026-10-18T11:58:28.527572Z ERROR replica-receiver melin_server_runtime::replication::tcp_receiver: UringTransport init failed error=io_uring init failed: Operation not permitted (os error 1) [INFO] [stdout] 2026-10-18T11:58:28.944101Z ERROR main melin_server_runtime::server: pipeline thread panicked thread="response" failed to create io_uring instance for response stage: Os { code: 1, kind: PermissionDenied, message: "Operation not permitted" } [INFO] [stdout] 2026-10-18T11:58:28.950578Z ERROR main melin_server_runtime::server: shutdown complete (with pipeline failure) [INFO] [stderr] Error: "pipeline failure" [INFO] [stdout] test journals_contiguous_across_replication ... FAILED [INFO] [stderr] [INFO] [stderr] thread 'response' (7846) panicked at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/melin-server-runtime-0.15.0/src/response.rs:300:10: [INFO] [stderr] failed to create io_uring instance for response stage: Os { code: 1, kind: PermissionDenied, message: "Operation not permitted" } [INFO] [stderr] stack backtrace: [INFO] [stderr] 0: 0x56cc294b19a1 - <::print::DisplayBacktrace as core[a373ad73a504971]::fmt::Display>::fmt [INFO] [stderr] 1: 0x56cc294c963a - core[a373ad73a504971]::fmt::write [INFO] [stderr] 2: 0x56cc294b76ec - ::write_fmt [INFO] [stderr] 3: 0x56cc294921d6 - std[617be2ca0a6fd877]::panicking::default_hook::{closure#0} [INFO] [stderr] 4: 0x56cc294a8d41 - std[617be2ca0a6fd877]::panicking::default_hook [INFO] [stderr] 5: 0x56cc294a906b - std[617be2ca0a6fd877]::panicking::panic_with_hook [INFO] [stderr] 6: 0x56cc29492282 - std[617be2ca0a6fd877]::panicking::panic_handler::{closure#0} [INFO] [stderr] 7: 0x56cc2948acf9 - std[617be2ca0a6fd877]::sys::backtrace::__rust_end_short_backtrace:: [INFO] [stderr] 8: 0x56cc2949303d - __rustc[ea4f61d10b84890b]::rust_begin_unwind [INFO] [stderr] 9: 0x56cc294c9e1c - core[a373ad73a504971]::panicking::panic_fmt [INFO] [stderr] 10: 0x56cc294c9bf2 - core[a373ad73a504971]::result::unwrap_failed [INFO] [stderr] 11: 0x56cc290d640f - melin_server_runtime[439fe4d1bbbbe025]::response::run:: [INFO] [stderr] 12: 0x56cc290bde1f - std[617be2ca0a6fd877]::sys::backtrace::__rust_begin_short_backtrace::::{closure#5}, ()> [INFO] [stderr] 13: 0x56cc290c9a84 - ::{closure#5}, ()>::{closure#1} as core[a373ad73a504971]::ops::function::FnOnce<()>>::call_once::{shim:vtable#0} [INFO] [stderr] 14: 0x56cc294b02ff - ::new::thread_start [INFO] [stderr] 15: 0x71d943d83dfa - [INFO] [stderr] 16: 0x71d943e173d4 - clone [INFO] [stderr] 17: 0x0 - [INFO] [stdout] 2026-10-18T11:58:59.855345Z ERROR replica-receiver melin_server_runtime::replication::tcp_receiver: UringTransport init failed error=io_uring init failed: Operation not permitted (os error 1) [INFO] [stdout] test in_memory_cursor_runs_ahead_of_persisted_under_sustained_traffic ... FAILED [INFO] [stderr] [INFO] [stderr] thread 'response' (7868) panicked at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/melin-server-runtime-0.15.0/src/response.rs:300:10: [INFO] [stderr] failed to create io_uring instance for response stage: Os { code: 1, kind: PermissionDenied, message: "Operation not permitted" } [INFO] [stderr] stack backtrace: [INFO] [stderr] 0: 0x6297705049a1 - <::print::DisplayBacktrace as core[a373ad73a504971]::fmt::Display>::fmt [INFO] [stderr] 1: 0x62977051c63a - core[a373ad73a504971]::fmt::write [INFO] [stderr] 2: 0x62977050a6ec - ::write_fmt [INFO] [stderr] 3: 0x6297704e51d6 - std[617be2ca0a6fd877]::panicking::default_hook::{closure#0} [INFO] [stderr] 4: 0x6297704fbd41 - std[617be2ca0a6fd877]::panicking::default_hook [INFO] [stderr] 5: 0x6297704fc06b - std[617be2ca0a6fd877]::panicking::panic_with_hook [INFO] [stderr] 6: 0x6297704e5282 - std[617be2ca0a6fd877]::panicking::panic_handler::{closure#0} [INFO] [stderr] 7: 0x6297704ddcf9 - std[617be2ca0a6fd877]::sys::backtrace::__rust_end_short_backtrace:: [INFO] [stderr] 8: 0x6297704e603d - __rustc[ea4f61d10b84890b]::rust_begin_unwind [INFO] [stderr] 9: 0x62977051ce1c - core[a373ad73a504971]::panicking::panic_fmt [INFO] [stderr] 10: 0x62977051cbf2 - core[a373ad73a504971]::result::unwrap_failed [INFO] [stderr] 11: 0x62977012940f - melin_server_runtime[439fe4d1bbbbe025]::response::run:: [INFO] [stderr] 12: 0x629770110e1f - std[617be2ca0a6fd877]::sys::backtrace::__rust_begin_short_backtrace::::{closure#5}, ()> [INFO] [stderr] 13: 0x62977011ca84 - ::{closure#5}, ()>::{closure#1} as core[a373ad73a504971]::ops::function::FnOnce<()>>::call_once::{shim:vtable#0} [INFO] [stderr] 14: 0x6297705032ff - ::new::thread_start [INFO] [stderr] 15: 0x794811199dfa - [INFO] [stderr] 16: 0x79481122d3d4 - clone [INFO] [stderr] 17: 0x0 - [INFO] [stderr] [INFO] [stdout] 2026-10-18T11:59:30.509971Z ERROR main melin_server_runtime::server: pipeline thread died, initiating shutdown [INFO] [stderr] thread 'uring-reader' (7880) panicked at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/melin-server-runtime-0.15.0/src/reader.rs:421:10: [INFO] [stderr] failed to create io_uring instance: Os { code: 1, kind: PermissionDenied, message: "Operation not permitted" } [INFO] [stderr] stack backtrace: [INFO] [stderr] 0: 0x6297705049a1 - <::print::DisplayBacktrace as core[a373ad73a504971]::fmt::Display>::fmt [INFO] [stderr] 1: 0x62977051c63a - core[a373ad73a504971]::fmt::write [INFO] [stderr] 2: 0x62977050a6ec - ::write_fmt [INFO] [stderr] 3: 0x6297704e51d6 - std[617be2ca0a6fd877]::panicking::default_hook::{closure#0} [INFO] [stderr] 4: 0x6297704fbd41 - std[617be2ca0a6fd877]::panicking::default_hook [INFO] [stderr] 5: 0x6297704fc06b - std[617be2ca0a6fd877]::panicking::panic_with_hook [INFO] [stderr] 6: 0x6297704e5282 - std[617be2ca0a6fd877]::panicking::panic_handler::{closure#0} [INFO] [stderr] 7: 0x6297704ddcf9 - std[617be2ca0a6fd877]::sys::backtrace::__rust_end_short_backtrace:: [INFO] [stdout] 2026-10-18T11:59:30.511970Z ERROR main melin_server_runtime::reader: reader thread panicked failed to create io_uring instance: Os { code: 1, kind: PermissionDenied, message: "Operation not permitted" } [INFO] [stderr] 8: 0x6297704e603d - __rustc[ea4f61d10b84890b]::rust_begin_unwind [INFO] [stderr] 9: 0x62977051ce1c - core[a373ad73a504971]::panicking::panic_fmt [INFO] [stderr] 10: 0x62977051cbf2 - core[a373ad73a504971]::result::unwrap_failed [INFO] [stderr] 11: 0x629770139678 - melin_server_runtime[439fe4d1bbbbe025]::reader::reader_loop:: [INFO] [stderr] 12: 0x629770110495 - std[617be2ca0a6fd877]::sys::backtrace::__rust_begin_short_backtrace::::{closure#0}, ()> [INFO] [stderr] 13: 0x62977011c2be - ::{closure#0}, ()>::{closure#1} as core[a373ad73a504971]::ops::function::FnOnce<()>>::call_once::{shim:vtable#0} [INFO] [stderr] 14: 0x6297705032ff - ::new::thread_start [INFO] [stderr] 15: 0x794811199dfa - [INFO] [stderr] 16: 0x79481122d3d4 - clone [INFO] [stderr] 17: 0x0 - [INFO] [stdout] 2026-10-18T11:59:31.020137Z ERROR replica-receiver melin_server_runtime::replication::tcp_receiver: UringTransport init failed error=io_uring init failed: Operation not permitted (os error 1) [INFO] [stdout] 2026-10-18T11:59:31.223862Z ERROR replica-receiver melin_server_runtime::replication::tcp_receiver: UringTransport init failed error=io_uring init failed: Operation not permitted (os error 1) [INFO] [stderr] Error: "pipeline failure" [INFO] [stdout] 2026-10-18T11:59:31.457034Z ERROR main melin_server_runtime::server: pipeline thread panicked thread="response" failed to create io_uring instance for response stage: Os { code: 1, kind: PermissionDenied, message: "Operation not permitted" } [INFO] [stdout] 2026-10-18T11:59:31.457068Z ERROR main melin_server_runtime::server: shutdown complete (with pipeline failure) [INFO] [stdout] test catchup_then_immediate_failover ... FAILED [INFO] [stderr] [INFO] [stderr] thread 'response' (7898) panicked at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/melin-server-runtime-0.15.0/src/response.rs:300:10: [INFO] [stderr] failed to create io_uring instance for response stage: Os { code: 1, kind: PermissionDenied, message: "Operation not permitted" } [INFO] [stderr] stack backtrace: [INFO] [stderr] 0: 0x601712eee9a1 - <::print::DisplayBacktrace as core[a373ad73a504971]::fmt::Display>::fmt [INFO] [stderr] 1: 0x601712f0663a - core[a373ad73a504971]::fmt::write [INFO] [stderr] 2: 0x601712ef46ec - ::write_fmt [INFO] [stderr] 3: 0x601712ecf1d6 - std[617be2ca0a6fd877]::panicking::default_hook::{closure#0} [INFO] [stderr] 4: 0x601712ee5d41 - std[617be2ca0a6fd877]::panicking::default_hook [INFO] [stderr] 5: 0x601712ee606b - std[617be2ca0a6fd877]::panicking::panic_with_hook [INFO] [stderr] 6: 0x601712ecf282 - std[617be2ca0a6fd877]::panicking::panic_handler::{closure#0} [INFO] [stderr] 7: 0x601712ec7cf9 - std[617be2ca0a6fd877]::sys::backtrace::__rust_end_short_backtrace:: [INFO] [stderr] 8: 0x601712ed003d - __rustc[ea4f61d10b84890b]::rust_begin_unwind [INFO] [stderr] 9: 0x601712f06e1c - core[a373ad73a504971]::panicking::panic_fmt [INFO] [stderr] 10: 0x601712f06bf2 - core[a373ad73a504971]::result::unwrap_failed [INFO] [stderr] 11: 0x601712b1340f - melin_server_runtime[439fe4d1bbbbe025]::response::run:: [INFO] [stderr] 12: 0x601712afae1f - std[617be2ca0a6fd877]::sys::backtrace::__rust_begin_short_backtrace::::{closure#5}, ()> [INFO] [stderr] 13: 0x601712b06a84 - ::{closure#5}, ()>::{closure#1} as core[a373ad73a504971]::ops::function::FnOnce<()>>::call_once::{shim:vtable#0} [INFO] [stderr] 14: 0x601712eed2ff - ::new::thread_start [INFO] [stderr] 15: 0x7b6a1bdf6dfa - [INFO] [stderr] 16: 0x7b6a1be8a3d4 - clone [INFO] [stderr] 17: 0x0 - [INFO] [stdout] 2026-10-18T12:00:01.950730Z ERROR main melin_server_runtime::server: pipeline thread died, initiating shutdown [INFO] [stderr] [INFO] [stderr] thread 'uring-reader' (7910) panicked at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/melin-server-runtime-0.15.0/src/reader.rs:421:10: [INFO] [stderr] failed to create io_uring instance: Os { code: 1, kind: PermissionDenied, message: "Operation not permitted" } [INFO] [stderr] stack backtrace: [INFO] [stderr] 0: 0x601712eee9a1 - <::print::DisplayBacktrace as core[a373ad73a504971]::fmt::Display>::fmt [INFO] [stderr] 1: 0x601712f0663a - core[a373ad73a504971]::fmt::write [INFO] [stderr] 2: 0x601712ef46ec - ::write_fmt [INFO] [stderr] 3: 0x601712ecf1d6 - std[617be2ca0a6fd877]::panicking::default_hook::{closure#0} [INFO] [stderr] 4: 0x601712ee5d41 - std[617be2ca0a6fd877]::panicking::default_hook [INFO] [stderr] 5: 0x601712ee606b - std[617be2ca0a6fd877]::panicking::panic_with_hook [INFO] [stderr] 6: 0x601712ecf282 - std[617be2ca0a6fd877]::panicking::panic_handler::{closure#0} [INFO] [stderr] 7: 0x601712ec7cf9 - std[617be2ca0a6fd877]::sys::backtrace::__rust_end_short_backtrace:: [INFO] [stderr] 8: 0x601712ed003d - __rustc[ea4f61d10b84890b]::rust_begin_unwind [INFO] [stderr] 9: 0x601712f06e1c - core[a373ad73a504971]::panicking::panic_fmt [INFO] [stderr] 10: 0x601712f06bf2 - core[a373ad73a504971]::result::unwrap_failed [INFO] [stderr] 11: 0x601712b23678 - melin_server_runtime[439fe4d1bbbbe025]::reader::reader_loop:: [INFO] [stderr] 12: 0x601712afa495 - std[617be2ca0a6fd877]::sys::backtrace::__rust_begin_short_backtrace::::{closure#0}, ()> [INFO] [stdout] 2026-10-18T12:00:01.952953Z ERROR main melin_server_runtime::reader: reader thread panicked failed to create io_uring instance: Os { code: 1, kind: PermissionDenied, message: "Operation not permitted" } [INFO] [stderr] 13: 0x601712b062be - ::{closure#0}, ()>::{closure#1} as core[a373ad73a504971]::ops::function::FnOnce<()>>::call_once::{shim:vtable#0} [INFO] [stderr] 14: 0x601712eed2ff - ::new::thread_start [INFO] [stderr] 15: 0x7b6a1bdf6dfa - [INFO] [stderr] 16: 0x7b6a1be8a3d4 - clone [INFO] [stderr] 17: 0x0 - [INFO] [stdout] 2026-10-18T12:00:02.628800Z ERROR replica-receiver melin_server_runtime::replication::tcp_receiver: UringTransport init failed error=io_uring init failed: Operation not permitted (os error 1) [INFO] [stdout] 2026-10-18T12:00:02.863960Z ERROR replica-receiver melin_server_runtime::replication::tcp_receiver: UringTransport init failed error=io_uring init failed: Operation not permitted (os error 1) [INFO] [stdout] 2026-10-18T12:00:03.018144Z ERROR main melin_server_runtime::server: pipeline thread panicked thread="response" failed to create io_uring instance for response stage: Os { code: 1, kind: PermissionDenied, message: "Operation not permitted" } [INFO] [stderr] Error: "pipeline failure" [INFO] [stdout] 2026-10-18T12:00:03.024639Z ERROR main melin_server_runtime::server: shutdown complete (with pipeline failure) [INFO] [stdout] test replacement_replica_catches_up_from_journal ... FAILED [INFO] [stderr] [INFO] [stderr] thread 'response' (7928) panicked at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/melin-server-runtime-0.15.0/src/response.rs:300:10: [INFO] [stderr] failed to create io_uring instance for response stage: Os { code: 1, kind: PermissionDenied, message: "Operation not permitted" } [INFO] [stderr] stack backtrace: [INFO] [stderr] 0: 0x5c00371279a1 - <::print::DisplayBacktrace as core[a373ad73a504971]::fmt::Display>::fmt [INFO] [stderr] 1: 0x5c003713f63a - core[a373ad73a504971]::fmt::write [INFO] [stderr] 2: 0x5c003712d6ec - ::write_fmt [INFO] [stderr] 3: 0x5c00371081d6 - std[617be2ca0a6fd877]::panicking::default_hook::{closure#0} [INFO] [stderr] 4: 0x5c003711ed41 - std[617be2ca0a6fd877]::panicking::default_hook [INFO] [stderr] 5: 0x5c003711f06b - std[617be2ca0a6fd877]::panicking::panic_with_hook [INFO] [stderr] 6: 0x5c0037108282 - std[617be2ca0a6fd877]::panicking::panic_handler::{closure#0} [INFO] [stderr] 7: 0x5c0037100cf9 - std[617be2ca0a6fd877]::sys::backtrace::__rust_end_short_backtrace:: [INFO] [stderr] 8: 0x5c003710903d - __rustc[ea4f61d10b84890b]::rust_begin_unwind [INFO] [stderr] 9: 0x5c003713fe1c - core[a373ad73a504971]::panicking::panic_fmt [INFO] [stderr] 10: 0x5c003713fbf2 - core[a373ad73a504971]::result::unwrap_failed [INFO] [stderr] 11: 0x5c0036d4c40f - melin_server_runtime[439fe4d1bbbbe025]::response::run:: [INFO] [stderr] 12: 0x5c0036d33e1f - std[617be2ca0a6fd877]::sys::backtrace::__rust_begin_short_backtrace::::{closure#5}, ()> [INFO] [stderr] 13: 0x5c0036d3fa84 - ::{closure#5}, ()>::{closure#1} as core[a373ad73a504971]::ops::function::FnOnce<()>>::call_once::{shim:vtable#0} [INFO] [stderr] 14: 0x5c00371262ff - ::new::thread_start [INFO] [stderr] 15: 0x7affc2c6edfa - [INFO] [stderr] 16: 0x7affc2d023d4 - clone [INFO] [stderr] 17: 0x0 - [INFO] [stdout] 2026-10-18T12:00:32.940015Z ERROR main melin_server_runtime::server: pipeline thread died, initiating shutdown [INFO] [stderr] [INFO] [stderr] thread 'uring-reader' (7933) panicked at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/melin-server-runtime-0.15.0/src/reader.rs:421:10: [INFO] [stderr] failed to create io_uring instance: Os { code: 1, kind: PermissionDenied, message: "Operation not permitted" } [INFO] [stderr] stack backtrace: [INFO] [stderr] 0: 0x5c00371279a1 - <::print::DisplayBacktrace as core[a373ad73a504971]::fmt::Display>::fmt [INFO] [stderr] 1: 0x5c003713f63a - core[a373ad73a504971]::fmt::write [INFO] [stderr] 2: 0x5c003712d6ec - ::write_fmt [INFO] [stderr] 3: 0x5c00371081d6 - std[617be2ca0a6fd877]::panicking::default_hook::{closure#0} [INFO] [stderr] 4: 0x5c003711ed41 - std[617be2ca0a6fd877]::panicking::default_hook [INFO] [stderr] 5: 0x5c003711f06b - std[617be2ca0a6fd877]::panicking::panic_with_hook [INFO] [stderr] 6: 0x5c0037108282 - std[617be2ca0a6fd877]::panicking::panic_handler::{closure#0} [INFO] [stderr] 7: 0x5c0037100cf9 - std[617be2ca0a6fd877]::sys::backtrace::__rust_end_short_backtrace:: [INFO] [stderr] 8: 0x5c003710903d - __rustc[ea4f61d10b84890b]::rust_begin_unwind [INFO] [stderr] 9: 0x5c003713fe1c - core[a373ad73a504971]::panicking::panic_fmt [INFO] [stderr] 10: 0x5c003713fbf2 - core[a373ad73a504971]::result::unwrap_failed [INFO] [stderr] 11: 0x5c0036d5c678 - melin_server_runtime[439fe4d1bbbbe025]::reader::reader_loop:: [INFO] [stderr] 12: 0x5c0036d33495 - std[617be2ca0a6fd877]::sys::backtrace::__rust_begin_short_backtrace::::{closure#0}, ()> [INFO] [stderr] 13: 0x5c0036d3f2be - ::{closure#0}, ()>::{closure#1} as core[a373ad73a504971]::ops::function::FnOnce<()>>::call_once::{shim:vtable#0} [INFO] [stderr] 14: 0x5c00371262ff - ::new::thread_start [INFO] [stderr] 15: 0x7affc2c6edfa - [INFO] [stderr] 16: 0x7affc2d023d4 - clone [INFO] [stderr] 17: 0x0 - [INFO] [stdout] 2026-10-18T12:00:32.946086Z ERROR main melin_server_runtime::reader: reader thread panicked failed to create io_uring instance: Os { code: 1, kind: PermissionDenied, message: "Operation not permitted" } [INFO] [stdout] 2026-10-18T12:00:33.047390Z ERROR main melin_server_runtime::server: pipeline thread panicked thread="response" failed to create io_uring instance for response stage: Os { code: 1, kind: PermissionDenied, message: "Operation not permitted" } [INFO] [stderr] Error: "pipeline failure" [INFO] [stdout] 2026-10-18T12:00:33.047491Z ERROR main melin_server_runtime::server: shutdown complete (with pipeline failure) [INFO] [stdout] test snapshot_transfer_when_archives_purged ... FAILED [INFO] [stderr] [INFO] [stderr] thread 'response' (7937) panicked at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/melin-server-runtime-0.15.0/src/response.rs:300:10: [INFO] [stderr] failed to create io_uring instance for response stage: Os { code: 1, kind: PermissionDenied, message: "Operation not permitted" } [INFO] [stderr] stack backtrace: [INFO] [stderr] 0: 0x5ea8891489a1 - <::print::DisplayBacktrace as core[a373ad73a504971]::fmt::Display>::fmt [INFO] [stderr] 1: 0x5ea88916063a - core[a373ad73a504971]::fmt::write [INFO] [stderr] 2: 0x5ea88914e6ec - ::write_fmt [INFO] [stderr] 3: 0x5ea8891291d6 - std[617be2ca0a6fd877]::panicking::default_hook::{closure#0} [INFO] [stderr] 4: 0x5ea88913fd41 - std[617be2ca0a6fd877]::panicking::default_hook [INFO] [stderr] 5: 0x5ea88914006b - std[617be2ca0a6fd877]::panicking::panic_with_hook [INFO] [stderr] 6: 0x5ea889129282 - std[617be2ca0a6fd877]::panicking::panic_handler::{closure#0} [INFO] [stderr] 7: 0x5ea889121cf9 - std[617be2ca0a6fd877]::sys::backtrace::__rust_end_short_backtrace:: [INFO] [stderr] 8: 0x5ea88912a03d - __rustc[ea4f61d10b84890b]::rust_begin_unwind [INFO] [stderr] 9: 0x5ea889160e1c - core[a373ad73a504971]::panicking::panic_fmt [INFO] [stderr] 10: 0x5ea889160bf2 - core[a373ad73a504971]::result::unwrap_failed [INFO] [stderr] 11: 0x5ea888d6d40f - melin_server_runtime[439fe4d1bbbbe025]::response::run:: [INFO] [stderr] 12: 0x5ea888d54e1f - std[617be2ca0a6fd877]::sys::backtrace::__rust_begin_short_backtrace::::{closure#5}, ()> [INFO] [stderr] 13: 0x5ea888d60a84 - ::{closure#5}, ()>::{closure#1} as core[a373ad73a504971]::ops::function::FnOnce<()>>::call_once::{shim:vtable#0} [INFO] [stderr] 14: 0x5ea8891472ff - ::new::thread_start [INFO] [stderr] 15: 0x7ba4c1a6adfa - [INFO] [stderr] 16: 0x7ba4c1afe3d4 - clone [INFO] [stderr] 17: 0x0 - [INFO] [stderr] [INFO] [stdout] 2026-10-18T12:01:03.657442Z ERROR main melin_server_runtime::server: pipeline thread died, initiating shutdown [INFO] [stderr] thread 'uring-reader' (7949) panicked at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/melin-server-runtime-0.15.0/src/reader.rs:421:10: [INFO] [stderr] failed to create io_uring instance: Os { code: 1, kind: PermissionDenied, message: "Operation not permitted" } [INFO] [stderr] stack backtrace: [INFO] [stderr] 0: 0x5ea8891489a1 - <::print::DisplayBacktrace as core[a373ad73a504971]::fmt::Display>::fmt [INFO] [stdout] 2026-10-18T12:01:03.659991Z ERROR main melin_server_runtime::reader: reader thread panicked failed to create io_uring instance: Os { code: 1, kind: PermissionDenied, message: "Operation not permitted" } [INFO] [stderr] 1: 0x5ea88916063a - core[a373ad73a504971]::fmt::write [INFO] [stderr] 2: 0x5ea88914e6ec - ::write_fmt [INFO] [stderr] 3: 0x5ea8891291d6 - std[617be2ca0a6fd877]::panicking::default_hook::{closure#0} [INFO] [stderr] 4: 0x5ea88913fd41 - std[617be2ca0a6fd877]::panicking::default_hook [INFO] [stderr] 5: 0x5ea88914006b - std[617be2ca0a6fd877]::panicking::panic_with_hook [INFO] [stderr] 6: 0x5ea889129282 - std[617be2ca0a6fd877]::panicking::panic_handler::{closure#0} [INFO] [stderr] 7: 0x5ea889121cf9 - std[617be2ca0a6fd877]::sys::backtrace::__rust_end_short_backtrace:: [INFO] [stderr] 8: 0x5ea88912a03d - __rustc[ea4f61d10b84890b]::rust_begin_unwind [INFO] [stderr] 9: 0x5ea889160e1c - core[a373ad73a504971]::panicking::panic_fmt [INFO] [stderr] 10: 0x5ea889160bf2 - core[a373ad73a504971]::result::unwrap_failed [INFO] [stderr] 11: 0x5ea888d7d678 - melin_server_runtime[439fe4d1bbbbe025]::reader::reader_loop:: [INFO] [stderr] 12: 0x5ea888d54495 - std[617be2ca0a6fd877]::sys::backtrace::__rust_begin_short_backtrace::::{closure#0}, ()> [INFO] [stderr] 13: 0x5ea888d602be - ::{closure#0}, ()>::{closure#1} as core[a373ad73a504971]::ops::function::FnOnce<()>>::call_once::{shim:vtable#0} [INFO] [stderr] 14: 0x5ea8891472ff - ::new::thread_start [INFO] [stderr] 15: 0x7ba4c1a6adfa - [INFO] [stderr] 16: 0x7ba4c1afe3d4 - clone [INFO] [stderr] 17: 0x0 - [INFO] [stdout] 2026-10-18T12:01:03.764817Z ERROR replica-receiver melin_server_runtime::replication::tcp_receiver: UringTransport init failed error=io_uring init failed: Operation not permitted (os error 1) [INFO] [stdout] 2026-10-18T12:01:04.673623Z ERROR main melin_server_runtime::server: pipeline thread panicked thread="response" failed to create io_uring instance for response stage: Os { code: 1, kind: PermissionDenied, message: "Operation not permitted" } [INFO] [stderr] Error: "pipeline failure" [INFO] [stdout] 2026-10-18T12:01:04.673655Z ERROR main melin_server_runtime::server: shutdown complete (with pipeline failure) [INFO] [stdout] 2026-10-18T12:01:04.693583Z ERROR replica-receiver melin_server_runtime::replication::tcp_receiver: UringTransport init failed error=io_uring init failed: Operation not permitted (os error 1) [INFO] [stdout] test policy_degraded_gauge_transitions_with_cluster_shape ... FAILED [INFO] [stderr] [INFO] [stderr] thread 'response' (7967) panicked at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/melin-server-runtime-0.15.0/src/response.rs:300:10: [INFO] [stderr] failed to create io_uring instance for response stage: Os { code: 1, kind: PermissionDenied, message: "Operation not permitted" } [INFO] [stderr] stack backtrace: [INFO] [stderr] 0: 0x55f87b8ef9a1 - <::print::DisplayBacktrace as core[a373ad73a504971]::fmt::Display>::fmt [INFO] [stderr] 1: 0x55f87b90763a - core[a373ad73a504971]::fmt::write [INFO] [stderr] 2: 0x55f87b8f56ec - ::write_fmt [INFO] [stderr] 3: 0x55f87b8d01d6 - std[617be2ca0a6fd877]::panicking::default_hook::{closure#0} [INFO] [stderr] 4: 0x55f87b8e6d41 - std[617be2ca0a6fd877]::panicking::default_hook [INFO] [stderr] 5: 0x55f87b8e706b - std[617be2ca0a6fd877]::panicking::panic_with_hook [INFO] [stderr] 6: 0x55f87b8d0282 - std[617be2ca0a6fd877]::panicking::panic_handler::{closure#0} [INFO] [stderr] 7: 0x55f87b8c8cf9 - std[617be2ca0a6fd877]::sys::backtrace::__rust_end_short_backtrace:: [INFO] [stderr] 8: 0x55f87b8d103d - __rustc[ea4f61d10b84890b]::rust_begin_unwind [INFO] [stderr] 9: 0x55f87b907e1c - core[a373ad73a504971]::panicking::panic_fmt [INFO] [stderr] 10: 0x55f87b907bf2 - core[a373ad73a504971]::result::unwrap_failed [INFO] [stderr] 11: 0x55f87b51440f - melin_server_runtime[439fe4d1bbbbe025]::response::run:: [INFO] [stderr] 12: 0x55f87b4fbe1f - std[617be2ca0a6fd877]::sys::backtrace::__rust_begin_short_backtrace::::{closure#5}, ()> [INFO] [stderr] 13: 0x55f87b507a84 - ::{closure#5}, ()>::{closure#1} as core[a373ad73a504971]::ops::function::FnOnce<()>>::call_once::{shim:vtable#0} [INFO] [stderr] 14: 0x55f87b8ee2ff - ::new::thread_start [INFO] [stderr] 15: 0x7ff5d5fa8dfa - [INFO] [stderr] 16: 0x7ff5d603c3d4 - clone [INFO] [stderr] 17: 0x0 - [INFO] [stdout] 2026-10-18T12:01:34.876027Z ERROR main melin_server_runtime::server: pipeline thread died, initiating shutdown [INFO] [stderr] [INFO] [stdout] 2026-10-18T12:01:34.877947Z ERROR main melin_server_runtime::reader: reader thread panicked failed to create io_uring instance: Os { code: 1, kind: PermissionDenied, message: "Operation not permitted" } [INFO] [stderr] thread 'uring-reader' (7976) panicked at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/melin-server-runtime-0.15.0/src/reader.rs:421:10: [INFO] [stderr] failed to create io_uring instance: Os { code: 1, kind: PermissionDenied, message: "Operation not permitted" } [INFO] [stderr] stack backtrace: [INFO] [stderr] 0: 0x55f87b8ef9a1 - <::print::DisplayBacktrace as core[a373ad73a504971]::fmt::Display>::fmt [INFO] [stderr] 1: 0x55f87b90763a - core[a373ad73a504971]::fmt::write [INFO] [stderr] 2: 0x55f87b8f56ec - ::write_fmt [INFO] [stderr] 3: 0x55f87b8d01d6 - std[617be2ca0a6fd877]::panicking::default_hook::{closure#0} [INFO] [stderr] 4: 0x55f87b8e6d41 - std[617be2ca0a6fd877]::panicking::default_hook [INFO] [stderr] 5: 0x55f87b8e706b - std[617be2ca0a6fd877]::panicking::panic_with_hook [INFO] [stderr] 6: 0x55f87b8d0282 - std[617be2ca0a6fd877]::panicking::panic_handler::{closure#0} [INFO] [stderr] 7: 0x55f87b8c8cf9 - std[617be2ca0a6fd877]::sys::backtrace::__rust_end_short_backtrace:: [INFO] [stderr] 8: 0x55f87b8d103d - __rustc[ea4f61d10b84890b]::rust_begin_unwind [INFO] [stderr] 9: 0x55f87b907e1c - core[a373ad73a504971]::panicking::panic_fmt [INFO] [stderr] 10: 0x55f87b907bf2 - core[a373ad73a504971]::result::unwrap_failed [INFO] [stderr] 11: 0x55f87b524678 - melin_server_runtime[439fe4d1bbbbe025]::reader::reader_loop:: [INFO] [stderr] 12: 0x55f87b4fb495 - std[617be2ca0a6fd877]::sys::backtrace::__rust_begin_short_backtrace::::{closure#0}, ()> [INFO] [stderr] 13: 0x55f87b5072be - ::{closure#0}, ()>::{closure#1} as core[a373ad73a504971]::ops::function::FnOnce<()>>::call_once::{shim:vtable#0} [INFO] [stderr] 14: 0x55f87b8ee2ff - ::new::thread_start [INFO] [stderr] 15: 0x7ff5d5fa8dfa - [INFO] [stderr] 16: 0x7ff5d603c3d4 - clone [INFO] [stderr] 17: 0x0 - [INFO] [stdout] 2026-10-18T12:01:35.358815Z ERROR replica-receiver melin_server_runtime::replication::tcp_receiver: UringTransport init failed error=io_uring init failed: Operation not permitted (os error 1) [INFO] [stderr] Error: "pipeline failure" [INFO] [stdout] 2026-10-18T12:01:36.139682Z ERROR main melin_server_runtime::server: pipeline thread panicked thread="response" failed to create io_uring instance for response stage: Os { code: 1, kind: PermissionDenied, message: "Operation not permitted" } [INFO] [stdout] 2026-10-18T12:01:36.139788Z ERROR main melin_server_runtime::server: shutdown complete (with pipeline failure) [INFO] [stdout] test replica_reconnects_after_primary_restart_without_journal_wipe ... FAILED [INFO] [stderr] [INFO] [stderr] thread 'response' (7987) panicked at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/melin-server-runtime-0.15.0/src/response.rs:300:10: [INFO] [stderr] failed to create io_uring instance for response stage: Os { code: 1, kind: PermissionDenied, message: "Operation not permitted" } [INFO] [stderr] stack backtrace: [INFO] [stderr] 0: 0x5d4d3e2859a1 - <::print::DisplayBacktrace as core[a373ad73a504971]::fmt::Display>::fmt [INFO] [stderr] 1: 0x5d4d3e29d63a - core[a373ad73a504971]::fmt::write [INFO] [stderr] 2: 0x5d4d3e28b6ec - ::write_fmt [INFO] [stderr] 3: 0x5d4d3e2661d6 - std[617be2ca0a6fd877]::panicking::default_hook::{closure#0} [INFO] [stderr] 4: 0x5d4d3e27cd41 - std[617be2ca0a6fd877]::panicking::default_hook [INFO] [stderr] 5: 0x5d4d3e27d06b - std[617be2ca0a6fd877]::panicking::panic_with_hook [INFO] [stderr] 6: 0x5d4d3e266282 - std[617be2ca0a6fd877]::panicking::panic_handler::{closure#0} [INFO] [stderr] 7: 0x5d4d3e25ecf9 - std[617be2ca0a6fd877]::sys::backtrace::__rust_end_short_backtrace:: [INFO] [stderr] 8: 0x5d4d3e26703d - __rustc[ea4f61d10b84890b]::rust_begin_unwind [INFO] [stderr] 9: 0x5d4d3e29de1c - core[a373ad73a504971]::panicking::panic_fmt [INFO] [stderr] 10: 0x5d4d3e29dbf2 - core[a373ad73a504971]::result::unwrap_failed [INFO] [stderr] 11: 0x5d4d3deaa40f - melin_server_runtime[439fe4d1bbbbe025]::response::run:: [INFO] [stderr] 12: 0x5d4d3de91e1f - std[617be2ca0a6fd877]::sys::backtrace::__rust_begin_short_backtrace::::{closure#5}, ()> [INFO] [stderr] 13: 0x5d4d3de9da84 - ::{closure#5}, ()>::{closure#1} as core[a373ad73a504971]::ops::function::FnOnce<()>>::call_once::{shim:vtable#0} [INFO] [stderr] 14: 0x5d4d3e2842ff - ::new::thread_start [INFO] [stderr] 15: 0x73355d69adfa - [INFO] [stderr] 16: 0x73355d72e3d4 - clone [INFO] [stderr] 17: 0x0 - [INFO] [stdout] 2026-10-18T12:02:06.391690Z ERROR main melin_server_runtime::server: pipeline thread died, initiating shutdown [INFO] [stdout] 2026-10-18T12:02:06.433866Z ERROR main melin_server_runtime::reader: reader thread panicked failed to create io_uring instance: Os { code: 1, kind: PermissionDenied, message: "Operation not permitted" } [INFO] [stderr] [INFO] [stderr] thread 'uring-reader' (7999) panicked at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/melin-server-runtime-0.15.0/src/reader.rs:421:10: [INFO] [stderr] failed to create io_uring instance: Os { code: 1, kind: PermissionDenied, message: "Operation not permitted" } [INFO] [stderr] stack backtrace: [INFO] [stderr] 0: 0x5d4d3e2859a1 - <::print::DisplayBacktrace as core[a373ad73a504971]::fmt::Display>::fmt [INFO] [stderr] 1: 0x5d4d3e29d63a - core[a373ad73a504971]::fmt::write [INFO] [stderr] 2: 0x5d4d3e28b6ec - ::write_fmt [INFO] [stderr] 3: 0x5d4d3e2661d6 - std[617be2ca0a6fd877]::panicking::default_hook::{closure#0} [INFO] [stderr] 4: 0x5d4d3e27cd41 - std[617be2ca0a6fd877]::panicking::default_hook [INFO] [stderr] 5: 0x5d4d3e27d06b - std[617be2ca0a6fd877]::panicking::panic_with_hook [INFO] [stderr] 6: 0x5d4d3e266282 - std[617be2ca0a6fd877]::panicking::panic_handler::{closure#0} [INFO] [stderr] 7: 0x5d4d3e25ecf9 - std[617be2ca0a6fd877]::sys::backtrace::__rust_end_short_backtrace:: [INFO] [stderr] 8: 0x5d4d3e26703d - __rustc[ea4f61d10b84890b]::rust_begin_unwind [INFO] [stderr] 9: 0x5d4d3e29de1c - core[a373ad73a504971]::panicking::panic_fmt [INFO] [stderr] 10: 0x5d4d3e29dbf2 - core[a373ad73a504971]::result::unwrap_failed [INFO] [stderr] 11: 0x5d4d3deba678 - melin_server_runtime[439fe4d1bbbbe025]::reader::reader_loop:: [INFO] [stderr] 12: 0x5d4d3de91495 - std[617be2ca0a6fd877]::sys::backtrace::__rust_begin_short_backtrace::::{closure#0}, ()> [INFO] [stderr] 13: 0x5d4d3de9d2be - ::{closure#0}, ()>::{closure#1} as core[a373ad73a504971]::ops::function::FnOnce<()>>::call_once::{shim:vtable#0} [INFO] [stderr] 14: 0x5d4d3e2842ff - ::new::thread_start [INFO] [stderr] 15: 0x73355d69adfa - [INFO] [stderr] 16: 0x73355d72e3d4 - clone [INFO] [stderr] 17: 0x0 - [INFO] [stdout] 2026-10-18T12:02:07.037848Z ERROR replica-receiver melin_server_runtime::replication::tcp_receiver: UringTransport init failed error=io_uring init failed: Operation not permitted (os error 1) [INFO] [stdout] 2026-10-18T12:02:07.275885Z ERROR replica-receiver melin_server_runtime::replication::tcp_receiver: UringTransport init failed error=io_uring init failed: Operation not permitted (os error 1) [INFO] [stdout] 2026-10-18T12:02:07.386890Z ERROR main melin_server_runtime::server: pipeline thread panicked thread="response" failed to create io_uring instance for response stage: Os { code: 1, kind: PermissionDenied, message: "Operation not permitted" } [INFO] [stderr] Error: "pipeline failure" [INFO] [stdout] 2026-10-18T12:02:07.386921Z ERROR main melin_server_runtime::server: shutdown complete (with pipeline failure) [INFO] [stdout] test dual_replication_promote_replica1_after_replica2_dies ... FAILED [INFO] [stderr] [INFO] [stderr] thread 'response' (8017) panicked at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/melin-server-runtime-0.15.0/src/response.rs:300:10: [INFO] [stderr] failed to create io_uring instance for response stage: Os { code: 1, kind: PermissionDenied, message: "Operation not permitted" } [INFO] [stderr] stack backtrace: [INFO] [stderr] 0: 0x5af48c7b99a1 - <::print::DisplayBacktrace as core[a373ad73a504971]::fmt::Display>::fmt [INFO] [stderr] 1: 0x5af48c7d163a - core[a373ad73a504971]::fmt::write [INFO] [stderr] 2: 0x5af48c7bf6ec - ::write_fmt [INFO] [stderr] 3: 0x5af48c79a1d6 - std[617be2ca0a6fd877]::panicking::default_hook::{closure#0} [INFO] [stderr] 4: 0x5af48c7b0d41 - std[617be2ca0a6fd877]::panicking::default_hook [INFO] [stderr] 5: 0x5af48c7b106b - std[617be2ca0a6fd877]::panicking::panic_with_hook [INFO] [stderr] 6: 0x5af48c79a282 - std[617be2ca0a6fd877]::panicking::panic_handler::{closure#0} [INFO] [stderr] 7: 0x5af48c792cf9 - std[617be2ca0a6fd877]::sys::backtrace::__rust_end_short_backtrace:: [INFO] [stderr] 8: 0x5af48c79b03d - __rustc[ea4f61d10b84890b]::rust_begin_unwind [INFO] [stderr] 9: 0x5af48c7d1e1c - core[a373ad73a504971]::panicking::panic_fmt [INFO] [stderr] 10: 0x5af48c7d1bf2 - core[a373ad73a504971]::result::unwrap_failed [INFO] [stderr] 11: 0x5af48c3de40f - melin_server_runtime[439fe4d1bbbbe025]::response::run:: [INFO] [stderr] 12: 0x5af48c3c5e1f - std[617be2ca0a6fd877]::sys::backtrace::__rust_begin_short_backtrace::::{closure#5}, ()> [INFO] [stderr] 13: 0x5af48c3d1a84 - ::{closure#5}, ()>::{closure#1} as core[a373ad73a504971]::ops::function::FnOnce<()>>::call_once::{shim:vtable#0} [INFO] [stderr] 14: 0x5af48c7b82ff - ::new::thread_start [INFO] [stderr] 15: 0x795090e40dfa - [INFO] [stderr] 16: 0x795090ed43d4 - clone [INFO] [stderr] 17: 0x0 - [INFO] [stdout] 2026-10-18T12:02:38.284647Z ERROR main melin_server_runtime::server: pipeline thread died, initiating shutdown [INFO] [stderr] [INFO] [stdout] 2026-10-18T12:02:38.285834Z ERROR main melin_server_runtime::reader: reader thread panicked failed to create io_uring instance: Os { code: 1, kind: PermissionDenied, message: "Operation not permitted" } [INFO] [stderr] thread 'uring-reader' (8029) panicked at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/melin-server-runtime-0.15.0/src/reader.rs:421:10: [INFO] [stderr] failed to create io_uring instance: Os { code: 1, kind: PermissionDenied, message: "Operation not permitted" } [INFO] [stderr] stack backtrace: [INFO] [stderr] 0: 0x5af48c7b99a1 - <::print::DisplayBacktrace as core[a373ad73a504971]::fmt::Display>::fmt [INFO] [stderr] 1: 0x5af48c7d163a - core[a373ad73a504971]::fmt::write [INFO] [stderr] 2: 0x5af48c7bf6ec - ::write_fmt [INFO] [stderr] 3: 0x5af48c79a1d6 - std[617be2ca0a6fd877]::panicking::default_hook::{closure#0} [INFO] [stderr] 4: 0x5af48c7b0d41 - std[617be2ca0a6fd877]::panicking::default_hook [INFO] [stderr] 5: 0x5af48c7b106b - std[617be2ca0a6fd877]::panicking::panic_with_hook [INFO] [stderr] 6: 0x5af48c79a282 - std[617be2ca0a6fd877]::panicking::panic_handler::{closure#0} [INFO] [stderr] 7: 0x5af48c792cf9 - std[617be2ca0a6fd877]::sys::backtrace::__rust_end_short_backtrace:: [INFO] [stderr] 8: 0x5af48c79b03d - __rustc[ea4f61d10b84890b]::rust_begin_unwind [INFO] [stderr] 9: 0x5af48c7d1e1c - core[a373ad73a504971]::panicking::panic_fmt [INFO] [stderr] 10: 0x5af48c7d1bf2 - core[a373ad73a504971]::result::unwrap_failed [INFO] [stderr] 11: 0x5af48c3ee678 - melin_server_runtime[439fe4d1bbbbe025]::reader::reader_loop:: [INFO] [stderr] 12: 0x5af48c3c5495 - std[617be2ca0a6fd877]::sys::backtrace::__rust_begin_short_backtrace::::{closure#0}, ()> [INFO] [stderr] 13: 0x5af48c3d12be - ::{closure#0}, ()>::{closure#1} as core[a373ad73a504971]::ops::function::FnOnce<()>>::call_once::{shim:vtable#0} [INFO] [stderr] 14: 0x5af48c7b82ff - ::new::thread_start [INFO] [stderr] 15: 0x795090e40dfa - [INFO] [stderr] 16: 0x795090ed43d4 - clone [INFO] [stderr] 17: 0x0 - [INFO] [stdout] 2026-10-18T12:02:38.782685Z ERROR replica-receiver melin_server_runtime::replication::tcp_receiver: UringTransport init failed error=io_uring init failed: Operation not permitted (os error 1) [INFO] [stdout] 2026-10-18T12:02:38.825811Z ERROR replica-receiver melin_server_runtime::replication::tcp_receiver: UringTransport init failed error=io_uring init failed: Operation not permitted (os error 1) [INFO] [stderr] Error: "pipeline failure" [INFO] [stdout] 2026-10-18T12:02:39.263588Z ERROR main melin_server_runtime::server: pipeline thread panicked thread="response" failed to create io_uring instance for response stage: Os { code: 1, kind: PermissionDenied, message: "Operation not permitted" } [INFO] [stdout] 2026-10-18T12:02:39.263697Z ERROR main melin_server_runtime::server: shutdown complete (with pipeline failure) [INFO] [stdout] test higher_epoch_handshake_fences_stale_primary ... FAILED [INFO] [stderr] [INFO] [stderr] thread 'response' (8047) panicked at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/melin-server-runtime-0.15.0/src/response.rs:300:10: [INFO] [stderr] failed to create io_uring instance for response stage: Os { code: 1, kind: PermissionDenied, message: "Operation not permitted" } [INFO] [stderr] stack backtrace: [INFO] [stderr] 0: 0x5e7dbb7689a1 - <::print::DisplayBacktrace as core[a373ad73a504971]::fmt::Display>::fmt [INFO] [stderr] 1: 0x5e7dbb78063a - core[a373ad73a504971]::fmt::write [INFO] [stderr] 2: 0x5e7dbb76e6ec - ::write_fmt [INFO] [stderr] 3: 0x5e7dbb7491d6 - std[617be2ca0a6fd877]::panicking::default_hook::{closure#0} [INFO] [stderr] 4: 0x5e7dbb75fd41 - std[617be2ca0a6fd877]::panicking::default_hook [INFO] [stderr] 5: 0x5e7dbb76006b - std[617be2ca0a6fd877]::panicking::panic_with_hook [INFO] [stderr] 6: 0x5e7dbb749282 - std[617be2ca0a6fd877]::panicking::panic_handler::{closure#0} [INFO] [stderr] 7: 0x5e7dbb741cf9 - std[617be2ca0a6fd877]::sys::backtrace::__rust_end_short_backtrace:: [INFO] [stderr] 8: 0x5e7dbb74a03d - __rustc[ea4f61d10b84890b]::rust_begin_unwind [INFO] [stderr] 9: 0x5e7dbb780e1c - core[a373ad73a504971]::panicking::panic_fmt [INFO] [stderr] 10: 0x5e7dbb780bf2 - core[a373ad73a504971]::result::unwrap_failed [INFO] [stderr] 11: 0x5e7dbb38d40f - melin_server_runtime[439fe4d1bbbbe025]::response::run:: [INFO] [stderr] 12: 0x5e7dbb374e1f - std[617be2ca0a6fd877]::sys::backtrace::__rust_begin_short_backtrace::::{closure#5}, ()> [INFO] [stderr] 13: 0x5e7dbb380a84 - ::{closure#5}, ()>::{closure#1} as core[a373ad73a504971]::ops::function::FnOnce<()>>::call_once::{shim:vtable#0} [INFO] [stderr] 14: 0x5e7dbb7672ff - ::new::thread_start [INFO] [stderr] 15: 0x7223be528dfa - [INFO] [stderr] 16: 0x7223be5bc3d4 - clone [INFO] [stderr] 17: 0x0 - [INFO] [stderr] [INFO] [stderr] thread 'uring-reader' (8056) panicked at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/melin-server-runtime-0.15.0/src/reader.rs:421:10: [INFO] [stderr] failed to create io_uring instance: Os { code: 1, kind: PermissionDenied, message: "Operation not permitted" } [INFO] [stderr] stack backtrace: [INFO] [stdout] 2026-10-18T12:03:09.917713Z ERROR main melin_server_runtime::server: pipeline thread died, initiating shutdown [INFO] [stderr] 0: 0x5e7dbb7689a1 - <::print::DisplayBacktrace as core[a373ad73a504971]::fmt::Display>::fmt [INFO] [stderr] 1: 0x5e7dbb78063a - core[a373ad73a504971]::fmt::write [INFO] [stderr] 2: 0x5e7dbb76e6ec - ::write_fmt [INFO] [stderr] 3: 0x5e7dbb7491d6 - std[617be2ca0a6fd877]::panicking::default_hook::{closure#0} [INFO] [stderr] 4: 0x5e7dbb75fd41 - std[617be2ca0a6fd877]::panicking::default_hook [INFO] [stderr] 5: 0x5e7dbb76006b - std[617be2ca0a6fd877]::panicking::panic_with_hook [INFO] [stderr] 6: 0x5e7dbb749282 - std[617be2ca0a6fd877]::panicking::panic_handler::{closure#0} [INFO] [stderr] 7: 0x5e7dbb741cf9 - std[617be2ca0a6fd877]::sys::backtrace::__rust_end_short_backtrace:: [INFO] [stderr] 8: 0x5e7dbb74a03d - __rustc[ea4f61d10b84890b]::rust_begin_unwind [INFO] [stderr] 9: 0x5e7dbb780e1c - core[a373ad73a504971]::panicking::panic_fmt [INFO] [stderr] 10: 0x5e7dbb780bf2 - core[a373ad73a504971]::result::unwrap_failed [INFO] [stderr] 11: 0x5e7dbb39d678 - melin_server_runtime[439fe4d1bbbbe025]::reader::reader_loop:: [INFO] [stderr] 12: 0x5e7dbb374495 - std[617be2ca0a6fd877]::sys::backtrace::__rust_begin_short_backtrace::::{closure#0}, ()> [INFO] [stderr] 13: 0x5e7dbb3802be - ::{closure#0}, ()>::{closure#1} as core[a373ad73a504971]::ops::function::FnOnce<()>>::call_once::{shim:vtable#0} [INFO] [stderr] 14: 0x5e7dbb7672ff - ::new::thread_start [INFO] [stderr] 15: 0x7223be528dfa - [INFO] [stderr] 16: 0x7223be5bc3d4 - clone [INFO] [stderr] 17: 0x0 - [INFO] [stdout] 2026-10-18T12:03:09.933665Z ERROR main melin_server_runtime::reader: reader thread panicked failed to create io_uring instance: Os { code: 1, kind: PermissionDenied, message: "Operation not permitted" } [INFO] [stdout] 2026-10-18T12:03:10.456390Z ERROR replica-receiver melin_server_runtime::replication::tcp_receiver: UringTransport init failed error=io_uring init failed: Operation not permitted (os error 1) [INFO] [stderr] Error: "pipeline failure" [INFO] [stdout] 2026-10-18T12:03:10.852906Z ERROR main melin_server_runtime::server: pipeline thread panicked thread="response" failed to create io_uring instance for response stage: Os { code: 1, kind: PermissionDenied, message: "Operation not permitted" } [INFO] [stdout] 2026-10-18T12:03:10.853016Z ERROR main melin_server_runtime::server: shutdown complete (with pipeline failure) [INFO] [stdout] test sec04_rate_limit_replicates_to_replica ... FAILED [INFO] [stderr] [INFO] [stderr] thread 'response' (8067) panicked at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/melin-server-runtime-0.15.0/src/response.rs:300:10: [INFO] [stderr] failed to create io_uring instance for response stage: Os { code: 1, kind: PermissionDenied, message: "Operation not permitted" } [INFO] [stderr] stack backtrace: [INFO] [stderr] 0: 0x632e76a5c9a1 - <::print::DisplayBacktrace as core[a373ad73a504971]::fmt::Display>::fmt [INFO] [stderr] 1: 0x632e76a7463a - core[a373ad73a504971]::fmt::write [INFO] [stderr] 2: 0x632e76a626ec - ::write_fmt [INFO] [stderr] 3: 0x632e76a3d1d6 - std[617be2ca0a6fd877]::panicking::default_hook::{closure#0} [INFO] [stderr] 4: 0x632e76a53d41 - std[617be2ca0a6fd877]::panicking::default_hook [INFO] [stderr] 5: 0x632e76a5406b - std[617be2ca0a6fd877]::panicking::panic_with_hook [INFO] [stderr] 6: 0x632e76a3d282 - std[617be2ca0a6fd877]::panicking::panic_handler::{closure#0} [INFO] [stderr] 7: 0x632e76a35cf9 - std[617be2ca0a6fd877]::sys::backtrace::__rust_end_short_backtrace:: [INFO] [stderr] 8: 0x632e76a3e03d - __rustc[ea4f61d10b84890b]::rust_begin_unwind [INFO] [stderr] 9: 0x632e76a74e1c - core[a373ad73a504971]::panicking::panic_fmt [INFO] [stderr] 10: 0x632e76a74bf2 - core[a373ad73a504971]::result::unwrap_failed [INFO] [stderr] 11: 0x632e7668140f - melin_server_runtime[439fe4d1bbbbe025]::response::run:: [INFO] [stderr] 12: 0x632e76668e1f - std[617be2ca0a6fd877]::sys::backtrace::__rust_begin_short_backtrace::::{closure#5}, ()> [INFO] [stderr] 13: 0x632e76674a84 - ::{closure#5}, ()>::{closure#1} as core[a373ad73a504971]::ops::function::FnOnce<()>>::call_once::{shim:vtable#0} [INFO] [stderr] 14: 0x632e76a5b2ff - ::new::thread_start [INFO] [stderr] 15: 0x7f31cf36adfa - [INFO] [stderr] 16: 0x7f31cf3fe3d4 - clone [INFO] [stderr] 17: 0x0 - [INFO] [stdout] 2026-10-18T12:03:40.670021Z ERROR main melin_server_runtime::server: pipeline thread died, initiating shutdown [INFO] [stderr] [INFO] [stderr] thread 'uring-reader' (8076) panicked at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/melin-server-runtime-0.15.0/src/reader.rs:421:10: [INFO] [stderr] failed to create io_uring instance: Os { code: 1, kind: PermissionDenied, message: "Operation not permitted" } [INFO] [stderr] stack backtrace: [INFO] [stderr] 0: 0x632e76a5c9a1 - <::print::DisplayBacktrace as core[a373ad73a504971]::fmt::Display>::fmt [INFO] [stderr] 1: 0x632e76a7463a - core[a373ad73a504971]::fmt::write [INFO] [stderr] 2: 0x632e76a626ec - ::write_fmt [INFO] [stderr] 3: 0x632e76a3d1d6 - std[617be2ca0a6fd877]::panicking::default_hook::{closure#0} [INFO] [stderr] 4: 0x632e76a53d41 - std[617be2ca0a6fd877]::panicking::default_hook [INFO] [stderr] 5: 0x632e76a5406b - std[617be2ca0a6fd877]::panicking::panic_with_hook [INFO] [stderr] 6: 0x632e76a3d282 - std[617be2ca0a6fd877]::panicking::panic_handler::{closure#0} [INFO] [stderr] 7: 0x632e76a35cf9 - std[617be2ca0a6fd877]::sys::backtrace::__rust_end_short_backtrace:: [INFO] [stderr] 8: 0x632e76a3e03d - __rustc[ea4f61d10b84890b]::rust_begin_unwind [INFO] [stderr] 9: 0x632e76a74e1c - core[a373ad73a504971]::panicking::panic_fmt [INFO] [stderr] 10: 0x632e76a74bf2 - core[a373ad73a504971]::result::unwrap_failed [INFO] [stderr] 11: 0x632e76691678 - melin_server_runtime[439fe4d1bbbbe025]::reader::reader_loop:: [INFO] [stderr] 12: 0x632e76668495 - std[617be2ca0a6fd877]::sys::backtrace::__rust_begin_short_backtrace::::{closure#0}, ()> [INFO] [stderr] 13: 0x632e766742be - ::{closure#0}, ()>::{closure#1} as core[a373ad73a504971]::ops::function::FnOnce<()>>::call_once::{shim:vtable#0} [INFO] [stderr] 14: 0x632e76a5b2ff - ::new::thread_start [INFO] [stderr] 15: 0x7f31cf36adfa - [INFO] [stderr] 16: 0x7f31cf3fe3d4 - clone [INFO] [stderr] 17: 0x0 - [INFO] [stdout] 2026-10-18T12:03:40.675793Z ERROR main melin_server_runtime::reader: reader thread panicked failed to create io_uring instance: Os { code: 1, kind: PermissionDenied, message: "Operation not permitted" } [INFO] [stderr] Error: "pipeline failure" [INFO] [stdout] 2026-10-18T12:03:40.817992Z ERROR main melin_server_runtime::server: pipeline thread panicked thread="response" failed to create io_uring instance for response stage: Os { code: 1, kind: PermissionDenied, message: "Operation not permitted" } [INFO] [stdout] 2026-10-18T12:03:40.818131Z ERROR main melin_server_runtime::server: shutdown complete (with pipeline failure) [INFO] [stdout] 2026-10-18T12:03:41.157782Z ERROR replica-receiver melin_server_runtime::replication::tcp_receiver: UringTransport init failed error=io_uring init failed: Operation not permitted (os error 1) [INFO] [stdout] test same_key_request_seq_hwm_survives_failover ... FAILED [INFO] [stderr] [INFO] [stderr] thread 'response' (8087) panicked at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/melin-server-runtime-0.15.0/src/response.rs:300:10: [INFO] [stderr] failed to create io_uring instance for response stage: Os { code: 1, kind: PermissionDenied, message: "Operation not permitted" } [INFO] [stderr] stack backtrace: [INFO] [stderr] 0: 0x55b9a6bdd9a1 - <::print::DisplayBacktrace as core[a373ad73a504971]::fmt::Display>::fmt [INFO] [stderr] 1: 0x55b9a6bf563a - core[a373ad73a504971]::fmt::write [INFO] [stderr] 2: 0x55b9a6be36ec - ::write_fmt [INFO] [stderr] 3: 0x55b9a6bbe1d6 - std[617be2ca0a6fd877]::panicking::default_hook::{closure#0} [INFO] [stderr] 4: 0x55b9a6bd4d41 - std[617be2ca0a6fd877]::panicking::default_hook [INFO] [stderr] 5: 0x55b9a6bd506b - std[617be2ca0a6fd877]::panicking::panic_with_hook [INFO] [stderr] 6: 0x55b9a6bbe282 - std[617be2ca0a6fd877]::panicking::panic_handler::{closure#0} [INFO] [stderr] 7: 0x55b9a6bb6cf9 - std[617be2ca0a6fd877]::sys::backtrace::__rust_end_short_backtrace:: [INFO] [stderr] 8: 0x55b9a6bbf03d - __rustc[ea4f61d10b84890b]::rust_begin_unwind [INFO] [stderr] 9: 0x55b9a6bf5e1c - core[a373ad73a504971]::panicking::panic_fmt [INFO] [stderr] 10: 0x55b9a6bf5bf2 - core[a373ad73a504971]::result::unwrap_failed [INFO] [stderr] 11: 0x55b9a680240f - melin_server_runtime[439fe4d1bbbbe025]::response::run:: [INFO] [stderr] 12: 0x55b9a67e9e1f - std[617be2ca0a6fd877]::sys::backtrace::__rust_begin_short_backtrace::::{closure#5}, ()> [INFO] [stderr] 13: 0x55b9a67f5a84 - ::{closure#5}, ()>::{closure#1} as core[a373ad73a504971]::ops::function::FnOnce<()>>::call_once::{shim:vtable#0} [INFO] [stderr] 14: 0x55b9a6bdc2ff - ::new::thread_start [INFO] [stderr] 15: 0x73edb9d5bdfa - [INFO] [stderr] 16: 0x73edb9def3d4 - clone [INFO] [stderr] 17: 0x0 - [INFO] [stdout] 2026-10-18T12:04:12.162662Z ERROR main melin_server_runtime::server: pipeline thread died, initiating shutdown [INFO] [stderr] [INFO] [stderr] thread 'uring-reader' (8099) panicked at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/melin-server-runtime-0.15.0/src/reader.rs:421:10: [INFO] [stderr] failed to create io_uring instance: Os { code: 1, kind: PermissionDenied, message: "Operation not permitted" } [INFO] [stderr] stack backtrace: [INFO] [stderr] 0: 0x55b9a6bdd9a1 - <::print::DisplayBacktrace as core[a373ad73a504971]::fmt::Display>::fmt [INFO] [stderr] 1: 0x55b9a6bf563a - core[a373ad73a504971]::fmt::write [INFO] [stderr] 2: 0x55b9a6be36ec - ::write_fmt [INFO] [stderr] 3: 0x55b9a6bbe1d6 - std[617be2ca0a6fd877]::panicking::default_hook::{closure#0} [INFO] [stderr] 4: 0x55b9a6bd4d41 - std[617be2ca0a6fd877]::panicking::default_hook [INFO] [stderr] 5: 0x55b9a6bd506b - std[617be2ca0a6fd877]::panicking::panic_with_hook [INFO] [stderr] 6: 0x55b9a6bbe282 - std[617be2ca0a6fd877]::panicking::panic_handler::{closure#0} [INFO] [stderr] 7: 0x55b9a6bb6cf9 - std[617be2ca0a6fd877]::sys::backtrace::__rust_end_short_backtrace:: [INFO] [stderr] 8: 0x55b9a6bbf03d - __rustc[ea4f61d10b84890b]::rust_begin_unwind [INFO] [stderr] 9: 0x55b9a6bf5e1c - core[a373ad73a504971]::panicking::panic_fmt [INFO] [stderr] 10: 0x55b9a6bf5bf2 - core[a373ad73a504971]::result::unwrap_failed [INFO] [stderr] 11: 0x55b9a6812678 - melin_server_runtime[439fe4d1bbbbe025]::reader::reader_loop:: [INFO] [stderr] 12: 0x55b9a67e9495 - std[617be2ca0a6fd877]::sys::backtrace::__rust_begin_short_backtrace::::{closure#0}, ()> [INFO] [stderr] 13: 0x55b9a67f52be - ::{closure#0}, ()>::{closure#1} as core[a373ad73a504971]::ops::function::FnOnce<()>>::call_once::{shim:vtable#0} [INFO] [stderr] 14: 0x55b9a6bdc2ff - ::new::thread_start [INFO] [stderr] 15: 0x73edb9d5bdfa - [INFO] [stderr] 16: 0x73edb9def3d4 - clone [INFO] [stderr] 17: 0x0 - [INFO] [stdout] 2026-10-18T12:04:12.181901Z ERROR main melin_server_runtime::reader: reader thread panicked failed to create io_uring instance: Os { code: 1, kind: PermissionDenied, message: "Operation not permitted" } [INFO] [stdout] 2026-10-18T12:04:12.456815Z ERROR replica-receiver melin_server_runtime::replication::tcp_receiver: UringTransport init failed error=io_uring init failed: Operation not permitted (os error 1) [INFO] [stdout] 2026-10-18T12:04:13.015586Z ERROR replica-receiver melin_server_runtime::replication::tcp_receiver: UringTransport init failed error=io_uring init failed: Operation not permitted (os error 1) [INFO] [stdout] 2026-10-18T12:04:13.116745Z ERROR main melin_server_runtime::server: pipeline thread panicked thread="response" failed to create io_uring instance for response stage: Os { code: 1, kind: PermissionDenied, message: "Operation not permitted" } [INFO] [stdout] 2026-10-18T12:04:13.118701Z ERROR main melin_server_runtime::server: shutdown complete (with pipeline failure) [INFO] [stderr] Error: "pipeline failure" [INFO] [stdout] test dual_replication_survives_one_replica_failure ... FAILED [INFO] [stderr] [INFO] [stderr] thread 'response' (8117) panicked at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/melin-server-runtime-0.15.0/src/response.rs:300:10: [INFO] [stderr] failed to create io_uring instance for response stage: Os { code: 1, kind: PermissionDenied, message: "Operation not permitted" } [INFO] [stderr] stack backtrace: [INFO] [stderr] 0: 0x6094bd09d9a1 - <::print::DisplayBacktrace as core[a373ad73a504971]::fmt::Display>::fmt [INFO] [stderr] 1: 0x6094bd0b563a - core[a373ad73a504971]::fmt::write [INFO] [stderr] 2: 0x6094bd0a36ec - ::write_fmt [INFO] [stderr] 3: 0x6094bd07e1d6 - std[617be2ca0a6fd877]::panicking::default_hook::{closure#0} [INFO] [stderr] 4: 0x6094bd094d41 - std[617be2ca0a6fd877]::panicking::default_hook [INFO] [stderr] 5: 0x6094bd09506b - std[617be2ca0a6fd877]::panicking::panic_with_hook [INFO] [stderr] 6: 0x6094bd07e282 - std[617be2ca0a6fd877]::panicking::panic_handler::{closure#0} [INFO] [stderr] 7: 0x6094bd076cf9 - std[617be2ca0a6fd877]::sys::backtrace::__rust_end_short_backtrace:: [INFO] [stderr] 8: 0x6094bd07f03d - __rustc[ea4f61d10b84890b]::rust_begin_unwind [INFO] [stderr] 9: 0x6094bd0b5e1c - core[a373ad73a504971]::panicking::panic_fmt [INFO] [stderr] 10: 0x6094bd0b5bf2 - core[a373ad73a504971]::result::unwrap_failed [INFO] [stderr] 11: 0x6094bccc240f - melin_server_runtime[439fe4d1bbbbe025]::response::run:: [INFO] [stderr] 12: 0x6094bcca9e1f - std[617be2ca0a6fd877]::sys::backtrace::__rust_begin_short_backtrace::::{closure#5}, ()> [INFO] [stderr] 13: 0x6094bccb5a84 - ::{closure#5}, ()>::{closure#1} as core[a373ad73a504971]::ops::function::FnOnce<()>>::call_once::{shim:vtable#0} [INFO] [stderr] 14: 0x6094bd09c2ff - ::new::thread_start [INFO] [stderr] 15: 0x75e39003adfa - [INFO] [stderr] 16: 0x75e3900ce3d4 - clone [INFO] [stderr] 17: 0x0 - [INFO] [stderr] [INFO] [stdout] 2026-10-18T12:04:43.889718Z ERROR main melin_server_runtime::server: pipeline thread died, initiating shutdown [INFO] [stderr] thread 'uring-reader' (8126) panicked at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/melin-server-runtime-0.15.0/src/reader.rs:421:10: [INFO] [stdout] 2026-10-18T12:04:43.891874Z ERROR main melin_server_runtime::reader: reader thread panicked failed to create io_uring instance: Os { code: 1, kind: PermissionDenied, message: "Operation not permitted" } [INFO] [stderr] failed to create io_uring instance: Os { code: 1, kind: PermissionDenied, message: "Operation not permitted" } [INFO] [stderr] stack backtrace: [INFO] [stderr] 0: 0x6094bd09d9a1 - <::print::DisplayBacktrace as core[a373ad73a504971]::fmt::Display>::fmt [INFO] [stderr] 1: 0x6094bd0b563a - core[a373ad73a504971]::fmt::write [INFO] [stderr] 2: 0x6094bd0a36ec - ::write_fmt [INFO] [stderr] 3: 0x6094bd07e1d6 - std[617be2ca0a6fd877]::panicking::default_hook::{closure#0} [INFO] [stderr] 4: 0x6094bd094d41 - std[617be2ca0a6fd877]::panicking::default_hook [INFO] [stderr] 5: 0x6094bd09506b - std[617be2ca0a6fd877]::panicking::panic_with_hook [INFO] [stderr] 6: 0x6094bd07e282 - std[617be2ca0a6fd877]::panicking::panic_handler::{closure#0} [INFO] [stderr] 7: 0x6094bd076cf9 - std[617be2ca0a6fd877]::sys::backtrace::__rust_end_short_backtrace:: [INFO] [stderr] 8: 0x6094bd07f03d - __rustc[ea4f61d10b84890b]::rust_begin_unwind [INFO] [stderr] 9: 0x6094bd0b5e1c - core[a373ad73a504971]::panicking::panic_fmt [INFO] [stderr] 10: 0x6094bd0b5bf2 - core[a373ad73a504971]::result::unwrap_failed [INFO] [stderr] 11: 0x6094bccd2678 - melin_server_runtime[439fe4d1bbbbe025]::reader::reader_loop:: [INFO] [stderr] 12: 0x6094bcca9495 - std[617be2ca0a6fd877]::sys::backtrace::__rust_begin_short_backtrace::::{closure#0}, ()> [INFO] [stderr] 13: 0x6094bccb52be - ::{closure#0}, ()>::{closure#1} as core[a373ad73a504971]::ops::function::FnOnce<()>>::call_once::{shim:vtable#0} [INFO] [stderr] 14: 0x6094bd09c2ff - ::new::thread_start [INFO] [stderr] 15: 0x75e39003adfa - [INFO] [stderr] 16: 0x75e3900ce3d4 - clone [INFO] [stderr] 17: 0x0 - [INFO] [stdout] 2026-10-18T12:04:44.412804Z ERROR replica-receiver melin_server_runtime::replication::tcp_receiver: UringTransport init failed error=io_uring init failed: Operation not permitted (os error 1) [INFO] [stdout] 2026-10-18T12:04:44.864039Z ERROR main melin_server_runtime::server: pipeline thread panicked thread="response" failed to create io_uring instance for response stage: Os { code: 1, kind: PermissionDenied, message: "Operation not permitted" } [INFO] [stdout] 2026-10-18T12:04:44.866765Z ERROR main melin_server_runtime::server: shutdown complete (with pipeline failure) [INFO] [stderr] Error: "pipeline failure" [INFO] [stdout] test kill_during_active_fills ... FAILED [INFO] [stderr] [INFO] [stderr] thread 'response' (8137) panicked at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/melin-server-runtime-0.15.0/src/response.rs:300:10: [INFO] [stderr] failed to create io_uring instance for response stage: Os { code: 1, kind: PermissionDenied, message: "Operation not permitted" } [INFO] [stderr] stack backtrace: [INFO] [stderr] 0: 0x5a48df6c29a1 - <::print::DisplayBacktrace as core[a373ad73a504971]::fmt::Display>::fmt [INFO] [stderr] 1: 0x5a48df6da63a - core[a373ad73a504971]::fmt::write [INFO] [stderr] 2: 0x5a48df6c86ec - ::write_fmt [INFO] [stderr] 3: 0x5a48df6a31d6 - std[617be2ca0a6fd877]::panicking::default_hook::{closure#0} [INFO] [stderr] 4: 0x5a48df6b9d41 - std[617be2ca0a6fd877]::panicking::default_hook [INFO] [stderr] 5: 0x5a48df6ba06b - std[617be2ca0a6fd877]::panicking::panic_with_hook [INFO] [stderr] 6: 0x5a48df6a3282 - std[617be2ca0a6fd877]::panicking::panic_handler::{closure#0} [INFO] [stderr] 7: 0x5a48df69bcf9 - std[617be2ca0a6fd877]::sys::backtrace::__rust_end_short_backtrace:: [INFO] [stderr] 8: 0x5a48df6a403d - __rustc[ea4f61d10b84890b]::rust_begin_unwind [INFO] [stderr] 9: 0x5a48df6dae1c - core[a373ad73a504971]::panicking::panic_fmt [INFO] [stderr] 10: 0x5a48df6dabf2 - core[a373ad73a504971]::result::unwrap_failed [INFO] [stderr] 11: 0x5a48df2e740f - melin_server_runtime[439fe4d1bbbbe025]::response::run:: [INFO] [stderr] 12: 0x5a48df2cee1f - std[617be2ca0a6fd877]::sys::backtrace::__rust_begin_short_backtrace::::{closure#5}, ()> [INFO] [stderr] 13: 0x5a48df2daa84 - ::{closure#5}, ()>::{closure#1} as core[a373ad73a504971]::ops::function::FnOnce<()>>::call_once::{shim:vtable#0} [INFO] [stderr] 14: 0x5a48df6c12ff - ::new::thread_start [INFO] [stderr] 15: 0x7c44c9b07dfa - [INFO] [stderr] 16: 0x7c44c9b9b3d4 - clone [INFO] [stderr] 17: 0x0 - [INFO] [stdout] 2026-10-18T12:05:15.152726Z ERROR main melin_server_runtime::server: pipeline thread died, initiating shutdown [INFO] [stderr] [INFO] [stdout] 2026-10-18T12:05:15.154781Z ERROR main melin_server_runtime::reader: reader thread panicked failed to create io_uring instance: Os { code: 1, kind: PermissionDenied, message: "Operation not permitted" } [INFO] [stderr] thread 'uring-reader' (8146) panicked at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/melin-server-runtime-0.15.0/src/reader.rs:421:10: [INFO] [stderr] failed to create io_uring instance: Os { code: 1, kind: PermissionDenied, message: "Operation not permitted" } [INFO] [stderr] stack backtrace: [INFO] [stderr] 0: 0x5a48df6c29a1 - <::print::DisplayBacktrace as core[a373ad73a504971]::fmt::Display>::fmt [INFO] [stderr] 1: 0x5a48df6da63a - core[a373ad73a504971]::fmt::write [INFO] [stderr] 2: 0x5a48df6c86ec - ::write_fmt [INFO] [stderr] 3: 0x5a48df6a31d6 - std[617be2ca0a6fd877]::panicking::default_hook::{closure#0} [INFO] [stderr] 4: 0x5a48df6b9d41 - std[617be2ca0a6fd877]::panicking::default_hook [INFO] [stderr] 5: 0x5a48df6ba06b - std[617be2ca0a6fd877]::panicking::panic_with_hook [INFO] [stderr] 6: 0x5a48df6a3282 - std[617be2ca0a6fd877]::panicking::panic_handler::{closure#0} [INFO] [stderr] 7: 0x5a48df69bcf9 - std[617be2ca0a6fd877]::sys::backtrace::__rust_end_short_backtrace:: [INFO] [stderr] 8: 0x5a48df6a403d - __rustc[ea4f61d10b84890b]::rust_begin_unwind [INFO] [stderr] 9: 0x5a48df6dae1c - core[a373ad73a504971]::panicking::panic_fmt [INFO] [stderr] 10: 0x5a48df6dabf2 - core[a373ad73a504971]::result::unwrap_failed [INFO] [stderr] 11: 0x5a48df2f7678 - melin_server_runtime[439fe4d1bbbbe025]::reader::reader_loop:: [INFO] [stderr] 12: 0x5a48df2ce495 - std[617be2ca0a6fd877]::sys::backtrace::__rust_begin_short_backtrace::::{closure#0}, ()> [INFO] [stderr] 13: 0x5a48df2da2be - ::{closure#0}, ()>::{closure#1} as core[a373ad73a504971]::ops::function::FnOnce<()>>::call_once::{shim:vtable#0} [INFO] [stderr] 14: 0x5a48df6c12ff - ::new::thread_start [INFO] [stderr] 15: 0x7c44c9b07dfa - [INFO] [stderr] 16: 0x7c44c9b9b3d4 - clone [INFO] [stderr] 17: 0x0 - [INFO] [stdout] 2026-10-18T12:05:15.610786Z ERROR replica-receiver melin_server_runtime::replication::tcp_receiver: UringTransport init failed error=io_uring init failed: Operation not permitted (os error 1) [INFO] [stderr] Error: "pipeline failure" [INFO] [stdout] 2026-10-18T12:05:16.122426Z ERROR main melin_server_runtime::server: pipeline thread panicked thread="response" failed to create io_uring instance for response stage: Os { code: 1, kind: PermissionDenied, message: "Operation not permitted" } [INFO] [stdout] 2026-10-18T12:05:16.122590Z ERROR main melin_server_runtime::server: shutdown complete (with pipeline failure) [INFO] [stdout] test crashed_primary_recovers_from_journal ... FAILED [INFO] [stdout] [INFO] [stdout] failures: [INFO] [stdout] [INFO] [stdout] ---- evicted_replica_catchup_under_load_preserves_dense_lineage stdout ---- [INFO] [stdout] [INFO] [stdout] thread 'evicted_replica_catchup_under_load_preserves_dense_lineage' (7557) panicked at tests/failover.rs:275:13: [INFO] [stdout] only 0 of 2 replica(s) attached to 127.0.0.1:20001 within 30s — the test would race its own fixture from here [INFO] [stdout] stack backtrace: [INFO] [stdout] 0: 0x602b245e0c51 - <::print::DisplayBacktrace as core[a373ad73a504971]::fmt::Display>::fmt [INFO] [stdout] 1: 0x602b245f6fda - core[a373ad73a504971]::fmt::write [INFO] [stdout] 2: 0x602b245e54ac - as core[a373ad73a504971]::io::write::Write>::write_fmt [INFO] [stdout] 3: 0x602b245b8e86 - std[617be2ca0a6fd877]::panicking::default_hook::{closure#0} [INFO] [stdout] 4: 0x602b245d7079 - std[617be2ca0a6fd877]::panicking::default_hook [INFO] [stdout] 5: 0x602b2455cf40 - test[aada8e7e76ba119b]::test_main_inner::{closure#0} [INFO] [stdout] 6: 0x602b245d73a2 - std[617be2ca0a6fd877]::panicking::panic_with_hook [INFO] [stdout] 7: 0x602b245b8f32 - std[617be2ca0a6fd877]::panicking::panic_handler::{closure#0} [INFO] [stdout] 8: 0x602b245b18c9 - std[617be2ca0a6fd877]::sys::backtrace::__rust_end_short_backtrace:: [INFO] [stdout] 9: 0x602b245b9b3d - __rustc[ea4f61d10b84890b]::rust_begin_unwind [INFO] [stdout] 10: 0x602b245f773c - core[a373ad73a504971]::panicking::panic_fmt [INFO] [stdout] 11: 0x602b244f22e0 - failover[ab1c72d9133d71e1]::wait_for_replicas [INFO] [stdout] 12: 0x602b244f7009 - ::start_with_args [INFO] [stdout] 13: 0x602b2451065a - >::call_once [INFO] [stdout] 14: 0x602b24596f39 - serial_test[ddb617897473a8f9]::serial_code_lock::local_serial_core [INFO] [stdout] 15: 0x602b244f9efd - >::call_once [INFO] [stdout] 16: 0x602b2455038b - test[aada8e7e76ba119b]::__rust_begin_short_backtrace::, fn() -> core[a373ad73a504971]::result::Result<(), alloc[2e1b2a31b91974c5]::string::String>> [INFO] [stdout] 17: 0x602b2455d565 - test[aada8e7e76ba119b]::run_test::{closure#0} [INFO] [stdout] 18: 0x602b24556e54 - std[617be2ca0a6fd877]::sys::backtrace::__rust_begin_short_backtrace:: [INFO] [stdout] 19: 0x602b245605f2 - ::{closure#1} as core[a373ad73a504971]::ops::function::FnOnce<()>>::call_once::{shim:vtable#0} [INFO] [stdout] 20: 0x602b245dfc2f - ::new::thread_start [INFO] [stdout] 21: 0x76bb120b2dfa - [INFO] [stdout] 22: 0x76bb121463d4 - clone [INFO] [stdout] 23: 0x0 - [INFO] [stdout] [INFO] [stdout] ---- catchup_with_fills_during_gap stdout ---- [INFO] [stdout] [INFO] [stdout] thread 'catchup_with_fills_during_gap' (7550) panicked at tests/failover.rs:275:13: [INFO] [stdout] only 1 of 2 replica(s) attached to 127.0.0.1:20010 within 30s — the test would race its own fixture from here [INFO] [stdout] stack backtrace: [INFO] [stdout] 0: 0x602b245e0c51 - <::print::DisplayBacktrace as core[a373ad73a504971]::fmt::Display>::fmt [INFO] [stdout] 1: 0x602b245f6fda - core[a373ad73a504971]::fmt::write [INFO] [stdout] 2: 0x602b245e54ac - as core[a373ad73a504971]::io::write::Write>::write_fmt [INFO] [stdout] 3: 0x602b245b8e86 - std[617be2ca0a6fd877]::panicking::default_hook::{closure#0} [INFO] [stdout] 4: 0x602b245d7079 - std[617be2ca0a6fd877]::panicking::default_hook [INFO] [stdout] 5: 0x602b2455cf40 - test[aada8e7e76ba119b]::test_main_inner::{closure#0} [INFO] [stdout] 6: 0x602b245d73a2 - std[617be2ca0a6fd877]::panicking::panic_with_hook [INFO] [stdout] 7: 0x602b245b8f32 - std[617be2ca0a6fd877]::panicking::panic_handler::{closure#0} [INFO] [stdout] 8: 0x602b245b18c9 - std[617be2ca0a6fd877]::sys::backtrace::__rust_end_short_backtrace:: [INFO] [stdout] 9: 0x602b245b9b3d - __rustc[ea4f61d10b84890b]::rust_begin_unwind [INFO] [stdout] 10: 0x602b245f773c - core[a373ad73a504971]::panicking::panic_fmt [INFO] [stdout] 11: 0x602b244f22e0 - failover[ab1c72d9133d71e1]::wait_for_replicas [INFO] [stdout] 12: 0x602b244f7009 - ::start_with_args [INFO] [stdout] 13: 0x602b244fea21 - >::call_once [INFO] [stdout] 14: 0x602b24596f39 - serial_test[ddb617897473a8f9]::serial_code_lock::local_serial_core [INFO] [stdout] 15: 0x602b244f95fd - >::call_once [INFO] [stdout] 16: 0x602b2455038b - test[aada8e7e76ba119b]::__rust_begin_short_backtrace::, fn() -> core[a373ad73a504971]::result::Result<(), alloc[2e1b2a31b91974c5]::string::String>> [INFO] [stdout] 17: 0x602b2455d565 - test[aada8e7e76ba119b]::run_test::{closure#0} [INFO] [stdout] 18: 0x602b24556e54 - std[617be2ca0a6fd877]::sys::backtrace::__rust_begin_short_backtrace:: [INFO] [stdout] 19: 0x602b245605f2 - ::{closure#1} as core[a373ad73a504971]::ops::function::FnOnce<()>>::call_once::{shim:vtable#0} [INFO] [stdout] 20: 0x602b245dfc2f - ::new::thread_start [INFO] [stdout] 21: 0x76bb120b2dfa - [INFO] [stdout] 22: 0x76bb121463d4 - clone [INFO] [stdout] 23: 0x0 - [INFO] [stdout] [INFO] [stdout] ---- kill_without_waiting_for_replication stdout ---- [INFO] [stdout] [INFO] [stdout] thread 'kill_without_waiting_for_replication' (7564) panicked at tests/failover.rs:275:13: [INFO] [stdout] only 0 of 1 replica(s) attached to 127.0.0.1:20019 within 30s — the test would race its own fixture from here [INFO] [stdout] stack backtrace: [INFO] [stdout] 0: 0x602b245e0c51 - <::print::DisplayBacktrace as core[a373ad73a504971]::fmt::Display>::fmt [INFO] [stdout] 1: 0x602b245f6fda - core[a373ad73a504971]::fmt::write [INFO] [stdout] 2: 0x602b245e54ac - as core[a373ad73a504971]::io::write::Write>::write_fmt [INFO] [stdout] 3: 0x602b245b8e86 - std[617be2ca0a6fd877]::panicking::default_hook::{closure#0} [INFO] [stdout] 4: 0x602b245d7079 - std[617be2ca0a6fd877]::panicking::default_hook [INFO] [stdout] 5: 0x602b2455cf40 - test[aada8e7e76ba119b]::test_main_inner::{closure#0} [INFO] [stdout] 6: 0x602b245d73a2 - std[617be2ca0a6fd877]::panicking::panic_with_hook [INFO] [stdout] 7: 0x602b245b8f32 - std[617be2ca0a6fd877]::panicking::panic_handler::{closure#0} [INFO] [stdout] 8: 0x602b245b18c9 - std[617be2ca0a6fd877]::sys::backtrace::__rust_end_short_backtrace:: [INFO] [stdout] 9: 0x602b245b9b3d - __rustc[ea4f61d10b84890b]::rust_begin_unwind [INFO] [stdout] 10: 0x602b245f773c - core[a373ad73a504971]::panicking::panic_fmt [INFO] [stdout] 11: 0x602b244f22e0 - failover[ab1c72d9133d71e1]::wait_for_replicas [INFO] [stdout] 12: 0x602b244f81d4 - ::start_with_extra_args [INFO] [stdout] 13: 0x602b24503c25 - >::call_once [INFO] [stdout] 14: 0x602b24596f39 - serial_test[ddb617897473a8f9]::serial_code_lock::local_serial_core [INFO] [stdout] 15: 0x602b244f977d - >::call_once [INFO] [stdout] 16: 0x602b2455038b - test[aada8e7e76ba119b]::__rust_begin_short_backtrace::, fn() -> core[a373ad73a504971]::result::Result<(), alloc[2e1b2a31b91974c5]::string::String>> [INFO] [stdout] 17: 0x602b2455d565 - test[aada8e7e76ba119b]::run_test::{closure#0} [INFO] [stdout] 18: 0x602b24556e54 - std[617be2ca0a6fd877]::sys::backtrace::__rust_begin_short_backtrace:: [INFO] [stdout] 19: 0x602b245605f2 - ::{closure#1} as core[a373ad73a504971]::ops::function::FnOnce<()>>::call_once::{shim:vtable#0} [INFO] [stdout] 20: 0x602b245dfc2f - ::new::thread_start [INFO] [stdout] 21: 0x76bb120b2dfa - [INFO] [stdout] 22: 0x76bb121463d4 - clone [INFO] [stdout] 23: 0x0 - [INFO] [stdout] [INFO] [stdout] ---- kill_primary_promote_replica_no_data_loss stdout ---- [INFO] [stdout] [INFO] [stdout] thread 'kill_primary_promote_replica_no_data_loss' (7563) panicked at tests/failover.rs:275:13: [INFO] [stdout] only 0 of 1 replica(s) attached to 127.0.0.1:20025 within 30s — the test would race its own fixture from here [INFO] [stdout] stack backtrace: [INFO] [stdout] 0: 0x602b245e0c51 - <::print::DisplayBacktrace as core[a373ad73a504971]::fmt::Display>::fmt [INFO] [stdout] 1: 0x602b245f6fda - core[a373ad73a504971]::fmt::write [INFO] [stdout] 2: 0x602b245e54ac - as core[a373ad73a504971]::io::write::Write>::write_fmt [INFO] [stdout] 3: 0x602b245b8e86 - std[617be2ca0a6fd877]::panicking::default_hook::{closure#0} [INFO] [stdout] 4: 0x602b245d7079 - std[617be2ca0a6fd877]::panicking::default_hook [INFO] [stdout] 5: 0x602b2455cf40 - test[aada8e7e76ba119b]::test_main_inner::{closure#0} [INFO] [stdout] 6: 0x602b245d73a2 - std[617be2ca0a6fd877]::panicking::panic_with_hook [INFO] [stdout] 7: 0x602b245b8f32 - std[617be2ca0a6fd877]::panicking::panic_handler::{closure#0} [INFO] [stdout] 8: 0x602b245b18c9 - std[617be2ca0a6fd877]::sys::backtrace::__rust_end_short_backtrace:: [INFO] [stdout] 9: 0x602b245b9b3d - __rustc[ea4f61d10b84890b]::rust_begin_unwind [INFO] [stdout] 10: 0x602b245f773c - core[a373ad73a504971]::panicking::panic_fmt [INFO] [stdout] 11: 0x602b244f22e0 - failover[ab1c72d9133d71e1]::wait_for_replicas [INFO] [stdout] 12: 0x602b244f81d4 - ::start_with_extra_args [INFO] [stdout] 13: 0x602b24509815 - >::call_once [INFO] [stdout] 14: 0x602b24596f39 - serial_test[ddb617897473a8f9]::serial_code_lock::local_serial_core [INFO] [stdout] 15: 0x602b244f9afd - >::call_once [INFO] [stdout] 16: 0x602b2455038b - test[aada8e7e76ba119b]::__rust_begin_short_backtrace::, fn() -> core[a373ad73a504971]::result::Result<(), alloc[2e1b2a31b91974c5]::string::String>> [INFO] [stdout] 17: 0x602b2455d565 - test[aada8e7e76ba119b]::run_test::{closure#0} [INFO] [stdout] 18: 0x602b24556e54 - std[617be2ca0a6fd877]::sys::backtrace::__rust_begin_short_backtrace:: [INFO] [stdout] 19: 0x602b245605f2 - ::{closure#1} as core[a373ad73a504971]::ops::function::FnOnce<()>>::call_once::{shim:vtable#0} [INFO] [stdout] 20: 0x602b245dfc2f - ::new::thread_start [INFO] [stdout] 21: 0x76bb120b2dfa - [INFO] [stdout] 22: 0x76bb121463d4 - clone [INFO] [stdout] 23: 0x0 - [INFO] [stdout] [INFO] [stdout] ---- dual_replication_with_fills_then_failover stdout ---- [INFO] [stdout] [INFO] [stdout] thread 'dual_replication_with_fills_then_failover' (7556) panicked at tests/failover.rs:275:13: [INFO] [stdout] only 0 of 2 replica(s) attached to 127.0.0.1:20031 within 30s — the test would race its own fixture from here [INFO] [stdout] stack backtrace: [INFO] [stdout] 0: 0x602b245e0c51 - <::print::DisplayBacktrace as core[a373ad73a504971]::fmt::Display>::fmt [INFO] [stdout] 1: 0x602b245f6fda - core[a373ad73a504971]::fmt::write [INFO] [stdout] 2: 0x602b245e54ac - as core[a373ad73a504971]::io::write::Write>::write_fmt [INFO] [stdout] 3: 0x602b245b8e86 - std[617be2ca0a6fd877]::panicking::default_hook::{closure#0} [INFO] [stdout] 4: 0x602b245d7079 - std[617be2ca0a6fd877]::panicking::default_hook [INFO] [stdout] 5: 0x602b2455cf40 - test[aada8e7e76ba119b]::test_main_inner::{closure#0} [INFO] [stdout] 6: 0x602b245d73a2 - std[617be2ca0a6fd877]::panicking::panic_with_hook [INFO] [stdout] 7: 0x602b245b8f32 - std[617be2ca0a6fd877]::panicking::panic_handler::{closure#0} [INFO] [stdout] 8: 0x602b245b18c9 - std[617be2ca0a6fd877]::sys::backtrace::__rust_end_short_backtrace:: [INFO] [stdout] 9: 0x602b245b9b3d - __rustc[ea4f61d10b84890b]::rust_begin_unwind [INFO] [stdout] 10: 0x602b245f773c - core[a373ad73a504971]::panicking::panic_fmt [INFO] [stdout] 11: 0x602b244f22e0 - failover[ab1c72d9133d71e1]::wait_for_replicas [INFO] [stdout] 12: 0x602b244f7009 - ::start_with_args [INFO] [stdout] 13: 0x602b2450787e - >::call_once [INFO] [stdout] 14: 0x602b24596f39 - serial_test[ddb617897473a8f9]::serial_code_lock::local_serial_core [INFO] [stdout] 15: 0x602b244f9a7d - >::call_once [INFO] [stdout] 16: 0x602b2455038b - test[aada8e7e76ba119b]::__rust_begin_short_backtrace::, fn() -> core[a373ad73a504971]::result::Result<(), alloc[2e1b2a31b91974c5]::string::String>> [INFO] [stdout] 17: 0x602b2455d565 - test[aada8e7e76ba119b]::run_test::{closure#0} [INFO] [stdout] 18: 0x602b24556e54 - std[617be2ca0a6fd877]::sys::backtrace::__rust_begin_short_backtrace:: [INFO] [stdout] 19: 0x602b245605f2 - ::{closure#1} as core[a373ad73a504971]::ops::function::FnOnce<()>>::call_once::{shim:vtable#0} [INFO] [stdout] 20: 0x602b245dfc2f - ::new::thread_start [INFO] [stdout] 21: 0x76bb120b2dfa - [INFO] [stdout] 22: 0x76bb121463d4 - clone [INFO] [stdout] 23: 0x0 - [INFO] [stdout] [INFO] [stdout] ---- disk_ram_gate_stalls_while_replica_frozen stdout ---- [INFO] [stdout] [INFO] [stdout] thread 'disk_ram_gate_stalls_while_replica_frozen' (7552) panicked at tests/failover.rs:275:13: [INFO] [stdout] only 0 of 1 replica(s) attached to 127.0.0.1:20040 within 30s — the test would race its own fixture from here [INFO] [stdout] stack backtrace: [INFO] [stdout] 0: 0x602b245e0c51 - <::print::DisplayBacktrace as core[a373ad73a504971]::fmt::Display>::fmt [INFO] [stdout] 1: 0x602b245f6fda - core[a373ad73a504971]::fmt::write [INFO] [stdout] 2: 0x602b245e54ac - as core[a373ad73a504971]::io::write::Write>::write_fmt [INFO] [stdout] 3: 0x602b245b8e86 - std[617be2ca0a6fd877]::panicking::default_hook::{closure#0} [INFO] [stdout] 4: 0x602b245d7079 - std[617be2ca0a6fd877]::panicking::default_hook [INFO] [stdout] 5: 0x602b2455cf40 - test[aada8e7e76ba119b]::test_main_inner::{closure#0} [INFO] [stdout] 6: 0x602b245d73a2 - std[617be2ca0a6fd877]::panicking::panic_with_hook [INFO] [stdout] 7: 0x602b245b8f32 - std[617be2ca0a6fd877]::panicking::panic_handler::{closure#0} [INFO] [stdout] 8: 0x602b245b18c9 - std[617be2ca0a6fd877]::sys::backtrace::__rust_end_short_backtrace:: [INFO] [stdout] 9: 0x602b245b9b3d - __rustc[ea4f61d10b84890b]::rust_begin_unwind [INFO] [stdout] 10: 0x602b245f773c - core[a373ad73a504971]::panicking::panic_fmt [INFO] [stdout] 11: 0x602b244f22e0 - failover[ab1c72d9133d71e1]::wait_for_replicas [INFO] [stdout] 12: 0x602b244f81d4 - ::start_with_extra_args [INFO] [stdout] 13: 0x602b245070ac - >::call_once [INFO] [stdout] 14: 0x602b24596f39 - serial_test[ddb617897473a8f9]::serial_code_lock::local_serial_core [INFO] [stdout] 15: 0x602b244f99fd - >::call_once [INFO] [stdout] 16: 0x602b2455038b - test[aada8e7e76ba119b]::__rust_begin_short_backtrace::, fn() -> core[a373ad73a504971]::result::Result<(), alloc[2e1b2a31b91974c5]::string::String>> [INFO] [stdout] 17: 0x602b2455d565 - test[aada8e7e76ba119b]::run_test::{closure#0} [INFO] [stdout] 18: 0x602b24556e54 - std[617be2ca0a6fd877]::sys::backtrace::__rust_begin_short_backtrace:: [INFO] [stdout] 19: 0x602b245605f2 - ::{closure#1} as core[a373ad73a504971]::ops::function::FnOnce<()>>::call_once::{shim:vtable#0} [INFO] [stdout] 20: 0x602b245dfc2f - ::new::thread_start [INFO] [stdout] 21: 0x76bb120b2dfa - [INFO] [stdout] 22: 0x76bb121463d4 - clone [INFO] [stdout] 23: 0x0 - [INFO] [stdout] [INFO] [stdout] ---- recovered_primary_ack_gate_holds stdout ---- [INFO] [stdout] [INFO] [stdout] thread 'recovered_primary_ack_gate_holds' (7626) panicked at tests/failover.rs:275:13: [INFO] [stdout] only 0 of 1 replica(s) attached to 127.0.0.1:20046 within 30s — the test would race its own fixture from here [INFO] [stdout] stack backtrace: [INFO] [stdout] 0: 0x602b245e0c51 - <::print::DisplayBacktrace as core[a373ad73a504971]::fmt::Display>::fmt [INFO] [stdout] 1: 0x602b245f6fda - core[a373ad73a504971]::fmt::write [INFO] [stdout] 2: 0x602b245e54ac - as core[a373ad73a504971]::io::write::Write>::write_fmt [INFO] [stdout] 3: 0x602b245b8e86 - std[617be2ca0a6fd877]::panicking::default_hook::{closure#0} [INFO] [stdout] 4: 0x602b245d7079 - std[617be2ca0a6fd877]::panicking::default_hook [INFO] [stdout] 5: 0x602b2455cf40 - test[aada8e7e76ba119b]::test_main_inner::{closure#0} [INFO] [stdout] 6: 0x602b245d73a2 - std[617be2ca0a6fd877]::panicking::panic_with_hook [INFO] [stdout] 7: 0x602b245b8f32 - std[617be2ca0a6fd877]::panicking::panic_handler::{closure#0} [INFO] [stdout] 8: 0x602b245b18c9 - std[617be2ca0a6fd877]::sys::backtrace::__rust_end_short_backtrace:: [INFO] [stdout] 9: 0x602b245b9b3d - __rustc[ea4f61d10b84890b]::rust_begin_unwind [INFO] [stdout] 10: 0x602b245f773c - core[a373ad73a504971]::panicking::panic_fmt [INFO] [stdout] 11: 0x602b244f22e0 - failover[ab1c72d9133d71e1]::wait_for_replicas [INFO] [stdout] 12: 0x602b244f81d4 - ::start_with_extra_args [INFO] [stdout] 13: 0x602b24502c14 - >::call_once [INFO] [stdout] 14: 0x602b24596f39 - serial_test[ddb617897473a8f9]::serial_code_lock::local_serial_core [INFO] [stdout] 15: 0x602b244f96fd - >::call_once [INFO] [stdout] 16: 0x602b2455038b - test[aada8e7e76ba119b]::__rust_begin_short_backtrace::, fn() -> core[a373ad73a504971]::result::Result<(), alloc[2e1b2a31b91974c5]::string::String>> [INFO] [stdout] 17: 0x602b2455d565 - test[aada8e7e76ba119b]::run_test::{closure#0} [INFO] [stdout] 18: 0x602b24556e54 - std[617be2ca0a6fd877]::sys::backtrace::__rust_begin_short_backtrace:: [INFO] [stdout] 19: 0x602b245605f2 - ::{closure#1} as core[a373ad73a504971]::ops::function::FnOnce<()>>::call_once::{shim:vtable#0} [INFO] [stdout] 20: 0x602b245dfc2f - ::new::thread_start [INFO] [stdout] 21: 0x76bb120b2dfa - [INFO] [stdout] 22: 0x76bb121463d4 - clone [INFO] [stdout] 23: 0x0 - [INFO] [stdout] [INFO] [stdout] ---- fresh_replica_full_catchup stdout ---- [INFO] [stdout] [INFO] [stdout] thread 'fresh_replica_full_catchup' (7558) panicked at tests/failover.rs:275:13: [INFO] [stdout] only 0 of 2 replica(s) attached to 127.0.0.1:20052 within 30s — the test would race its own fixture from here [INFO] [stdout] stack backtrace: [INFO] [stdout] 0: 0x602b245e0c51 - <::print::DisplayBacktrace as core[a373ad73a504971]::fmt::Display>::fmt [INFO] [stdout] 1: 0x602b245f6fda - core[a373ad73a504971]::fmt::write [INFO] [stdout] 2: 0x602b245e54ac - as core[a373ad73a504971]::io::write::Write>::write_fmt [INFO] [stdout] 3: 0x602b245b8e86 - std[617be2ca0a6fd877]::panicking::default_hook::{closure#0} [INFO] [stdout] 4: 0x602b245d7079 - std[617be2ca0a6fd877]::panicking::default_hook [INFO] [stdout] 5: 0x602b2455cf40 - test[aada8e7e76ba119b]::test_main_inner::{closure#0} [INFO] [stdout] 6: 0x602b245d73a2 - std[617be2ca0a6fd877]::panicking::panic_with_hook [INFO] [stdout] 7: 0x602b245b8f32 - std[617be2ca0a6fd877]::panicking::panic_handler::{closure#0} [INFO] [stdout] 8: 0x602b245b18c9 - std[617be2ca0a6fd877]::sys::backtrace::__rust_end_short_backtrace:: [INFO] [stdout] 9: 0x602b245b9b3d - __rustc[ea4f61d10b84890b]::rust_begin_unwind [INFO] [stdout] 10: 0x602b245f773c - core[a373ad73a504971]::panicking::panic_fmt [INFO] [stdout] 11: 0x602b244f22e0 - failover[ab1c72d9133d71e1]::wait_for_replicas [INFO] [stdout] 12: 0x602b244f7009 - ::start_with_args [INFO] [stdout] 13: 0x602b244fca55 - >::call_once [INFO] [stdout] 14: 0x602b24596f39 - serial_test[ddb617897473a8f9]::serial_code_lock::local_serial_core [INFO] [stdout] 15: 0x602b244f957d - >::call_once [INFO] [stdout] 16: 0x602b2455038b - test[aada8e7e76ba119b]::__rust_begin_short_backtrace::, fn() -> core[a373ad73a504971]::result::Result<(), alloc[2e1b2a31b91974c5]::string::String>> [INFO] [stdout] 17: 0x602b2455d565 - test[aada8e7e76ba119b]::run_test::{closure#0} [INFO] [stdout] 18: 0x602b24556e54 - std[617be2ca0a6fd877]::sys::backtrace::__rust_begin_short_backtrace:: [INFO] [stdout] 19: 0x602b245605f2 - ::{closure#1} as core[a373ad73a504971]::ops::function::FnOnce<()>>::call_once::{shim:vtable#0} [INFO] [stdout] 20: 0x602b245dfc2f - ::new::thread_start [INFO] [stdout] 21: 0x76bb120b2dfa - [INFO] [stdout] 22: 0x76bb121463d4 - clone [INFO] [stdout] 23: 0x0 - [INFO] [stdout] [INFO] [stdout] ---- dual_replication_halts_when_both_disconnect stdout ---- [INFO] [stdout] [INFO] [stdout] thread 'dual_replication_halts_when_both_disconnect' (7553) panicked at tests/failover.rs:275:13: [INFO] [stdout] only 0 of 2 replica(s) attached to 127.0.0.1:20061 within 30s — the test would race its own fixture from here [INFO] [stdout] stack backtrace: [INFO] [stdout] 0: 0x602b245e0c51 - <::print::DisplayBacktrace as core[a373ad73a504971]::fmt::Display>::fmt [INFO] [stdout] 1: 0x602b245f6fda - core[a373ad73a504971]::fmt::write [INFO] [stdout] 2: 0x602b245e54ac - as core[a373ad73a504971]::io::write::Write>::write_fmt [INFO] [stdout] 3: 0x602b245b8e86 - std[617be2ca0a6fd877]::panicking::default_hook::{closure#0} [INFO] [stdout] 4: 0x602b245d7079 - std[617be2ca0a6fd877]::panicking::default_hook [INFO] [stdout] 5: 0x602b2455cf40 - test[aada8e7e76ba119b]::test_main_inner::{closure#0} [INFO] [stdout] 6: 0x602b245d73a2 - std[617be2ca0a6fd877]::panicking::panic_with_hook [INFO] [stdout] 7: 0x602b245b8f32 - std[617be2ca0a6fd877]::panicking::panic_handler::{closure#0} [INFO] [stdout] 8: 0x602b245b18c9 - std[617be2ca0a6fd877]::sys::backtrace::__rust_end_short_backtrace:: [INFO] [stdout] 9: 0x602b245b9b3d - __rustc[ea4f61d10b84890b]::rust_begin_unwind [INFO] [stdout] 10: 0x602b245f773c - core[a373ad73a504971]::panicking::panic_fmt [INFO] [stdout] 11: 0x602b244f22e0 - failover[ab1c72d9133d71e1]::wait_for_replicas [INFO] [stdout] 12: 0x602b244f7009 - ::start_with_args [INFO] [stdout] 13: 0x602b2450aa75 - >::call_once [INFO] [stdout] 14: 0x602b24596f39 - serial_test[ddb617897473a8f9]::serial_code_lock::local_serial_core [INFO] [stdout] 15: 0x602b244f9bfd - >::call_once [INFO] [stdout] 16: 0x602b2455038b - test[aada8e7e76ba119b]::__rust_begin_short_backtrace::, fn() -> core[a373ad73a504971]::result::Result<(), alloc[2e1b2a31b91974c5]::string::String>> [INFO] [stdout] 17: 0x602b2455d565 - test[aada8e7e76ba119b]::run_test::{closure#0} [INFO] [stdout] 18: 0x602b24556e54 - std[617be2ca0a6fd877]::sys::backtrace::__rust_begin_short_backtrace:: [INFO] [stdout] 19: 0x602b245605f2 - ::{closure#1} as core[a373ad73a504971]::ops::function::FnOnce<()>>::call_once::{shim:vtable#0} [INFO] [stdout] 20: 0x602b245dfc2f - ::new::thread_start [INFO] [stdout] 21: 0x76bb120b2dfa - [INFO] [stdout] 22: 0x76bb121463d4 - clone [INFO] [stdout] 23: 0x0 - [INFO] [stdout] [INFO] [stdout] ---- rotation_soak_under_load stdout ---- [INFO] [stdout] [INFO] [stdout] thread 'rotation_soak_under_load' (7699) panicked at tests/failover.rs:275:13: [INFO] [stdout] only 0 of 1 replica(s) attached to 127.0.0.1:20070 within 30s — the test would race its own fixture from here [INFO] [stdout] stack backtrace: [INFO] [stdout] 0: 0x602b245e0c51 - <::print::DisplayBacktrace as core[a373ad73a504971]::fmt::Display>::fmt [INFO] [stdout] 1: 0x602b245f6fda - core[a373ad73a504971]::fmt::write [INFO] [stdout] 2: 0x602b245e54ac - as core[a373ad73a504971]::io::write::Write>::write_fmt [INFO] [stdout] 3: 0x602b245b8e86 - std[617be2ca0a6fd877]::panicking::default_hook::{closure#0} [INFO] [stdout] 4: 0x602b245d7079 - std[617be2ca0a6fd877]::panicking::default_hook [INFO] [stdout] 5: 0x602b2455cf40 - test[aada8e7e76ba119b]::test_main_inner::{closure#0} [INFO] [stdout] 6: 0x602b245d73a2 - std[617be2ca0a6fd877]::panicking::panic_with_hook [INFO] [stdout] 7: 0x602b245b8f32 - std[617be2ca0a6fd877]::panicking::panic_handler::{closure#0} [INFO] [stdout] 8: 0x602b245b18c9 - std[617be2ca0a6fd877]::sys::backtrace::__rust_end_short_backtrace:: [INFO] [stdout] 9: 0x602b245b9b3d - __rustc[ea4f61d10b84890b]::rust_begin_unwind [INFO] [stdout] 10: 0x602b245f773c - core[a373ad73a504971]::panicking::panic_fmt [INFO] [stdout] 11: 0x602b244f22e0 - failover[ab1c72d9133d71e1]::wait_for_replicas [INFO] [stdout] 12: 0x602b244fadd7 - >::call_once [INFO] [stdout] 13: 0x602b24596f39 - serial_test[ddb617897473a8f9]::serial_code_lock::local_serial_core [INFO] [stdout] 14: 0x602b244f94fd - >::call_once [INFO] [stdout] 15: 0x602b2455038b - test[aada8e7e76ba119b]::__rust_begin_short_backtrace::, fn() -> core[a373ad73a504971]::result::Result<(), alloc[2e1b2a31b91974c5]::string::String>> [INFO] [stdout] 16: 0x602b2455d565 - test[aada8e7e76ba119b]::run_test::{closure#0} [INFO] [stdout] 17: 0x602b24556e54 - std[617be2ca0a6fd877]::sys::backtrace::__rust_begin_short_backtrace:: [INFO] [stdout] 18: 0x602b245605f2 - ::{closure#1} as core[a373ad73a504971]::ops::function::FnOnce<()>>::call_once::{shim:vtable#0} [INFO] [stdout] 19: 0x602b245dfc2f - ::new::thread_start [INFO] [stdout] 20: 0x76bb120b2dfa - [INFO] [stdout] 21: 0x76bb121463d4 - clone [INFO] [stdout] 22: 0x0 - [INFO] [stdout] [INFO] [stdout] ---- journals_contiguous_across_replication stdout ---- [INFO] [stdout] [INFO] [stdout] thread 'journals_contiguous_across_replication' (7561) panicked at tests/failover.rs:324:13: [INFO] [stdout] 127.0.0.1:20077 journaled only 0 of 12 seed events within 30s — the fixture's starting journal_seq is indeterminate [INFO] [stdout] stack backtrace: [INFO] [stdout] 0: 0x602b245e0c51 - <::print::DisplayBacktrace as core[a373ad73a504971]::fmt::Display>::fmt [INFO] [stdout] 1: 0x602b245f6fda - core[a373ad73a504971]::fmt::write [INFO] [stdout] 2: 0x602b245e54ac - as core[a373ad73a504971]::io::write::Write>::write_fmt [INFO] [stdout] 3: 0x602b245b8e86 - std[617be2ca0a6fd877]::panicking::default_hook::{closure#0} [INFO] [stdout] 4: 0x602b245d7079 - std[617be2ca0a6fd877]::panicking::default_hook [INFO] [stdout] 5: 0x602b2455cf40 - test[aada8e7e76ba119b]::test_main_inner::{closure#0} [INFO] [stdout] 6: 0x602b245d73a2 - std[617be2ca0a6fd877]::panicking::panic_with_hook [INFO] [stdout] 7: 0x602b245b8f32 - std[617be2ca0a6fd877]::panicking::panic_handler::{closure#0} [INFO] [stdout] 8: 0x602b245b18c9 - std[617be2ca0a6fd877]::sys::backtrace::__rust_end_short_backtrace:: [INFO] [stdout] 9: 0x602b245b9b3d - __rustc[ea4f61d10b84890b]::rust_begin_unwind [INFO] [stdout] 10: 0x602b245f773c - core[a373ad73a504971]::panicking::panic_fmt [INFO] [stdout] 11: 0x602b244f1b0c - failover[ab1c72d9133d71e1]::wait_for_seed [INFO] [stdout] 12: 0x602b244f81fa - ::start_with_extra_args [INFO] [stdout] 13: 0x602b24506505 - >::call_once [INFO] [stdout] 14: 0x602b24596f39 - serial_test[ddb617897473a8f9]::serial_code_lock::local_serial_core [INFO] [stdout] 15: 0x602b244f987d - >::call_once [INFO] [stdout] 16: 0x602b2455038b - test[aada8e7e76ba119b]::__rust_begin_short_backtrace::, fn() -> core[a373ad73a504971]::result::Result<(), alloc[2e1b2a31b91974c5]::string::String>> [INFO] [stdout] 17: 0x602b2455d565 - test[aada8e7e76ba119b]::run_test::{closure#0} [INFO] [stdout] 18: 0x602b24556e54 - std[617be2ca0a6fd877]::sys::backtrace::__rust_begin_short_backtrace:: [INFO] [stdout] 19: 0x602b245605f2 - ::{closure#1} as core[a373ad73a504971]::ops::function::FnOnce<()>>::call_once::{shim:vtable#0} [INFO] [stdout] 20: 0x602b245dfc2f - ::new::thread_start [INFO] [stdout] 21: 0x76bb120b2dfa - [INFO] [stdout] 22: 0x76bb121463d4 - clone [INFO] [stdout] 23: 0x0 - [INFO] [stdout] [INFO] [stdout] ---- in_memory_cursor_runs_ahead_of_persisted_under_sustained_traffic stdout ---- [INFO] [stdout] [INFO] [stdout] thread 'in_memory_cursor_runs_ahead_of_persisted_under_sustained_traffic' (7560) panicked at tests/failover.rs:275:13: [INFO] [stdout] only 0 of 2 replica(s) attached to 127.0.0.1:20083 within 30s — the test would race its own fixture from here [INFO] [stdout] stack backtrace: [INFO] [stdout] 0: 0x602b245e0c51 - <::print::DisplayBacktrace as core[a373ad73a504971]::fmt::Display>::fmt [INFO] [stdout] 1: 0x602b245f6fda - core[a373ad73a504971]::fmt::write [INFO] [stdout] 2: 0x602b245e54ac - as core[a373ad73a504971]::io::write::Write>::write_fmt [INFO] [stdout] 3: 0x602b245b8e86 - std[617be2ca0a6fd877]::panicking::default_hook::{closure#0} [INFO] [stdout] 4: 0x602b245d7079 - std[617be2ca0a6fd877]::panicking::default_hook [INFO] [stdout] 5: 0x602b2455cf40 - test[aada8e7e76ba119b]::test_main_inner::{closure#0} [INFO] [stdout] 6: 0x602b245d73a2 - std[617be2ca0a6fd877]::panicking::panic_with_hook [INFO] [stdout] 7: 0x602b245b8f32 - std[617be2ca0a6fd877]::panicking::panic_handler::{closure#0} [INFO] [stdout] 8: 0x602b245b18c9 - std[617be2ca0a6fd877]::sys::backtrace::__rust_end_short_backtrace:: [INFO] [stdout] 9: 0x602b245b9b3d - __rustc[ea4f61d10b84890b]::rust_begin_unwind [INFO] [stdout] 10: 0x602b245f773c - core[a373ad73a504971]::panicking::panic_fmt [INFO] [stdout] 11: 0x602b244f22e0 - failover[ab1c72d9133d71e1]::wait_for_replicas [INFO] [stdout] 12: 0x602b244f7009 - ::start_with_args [INFO] [stdout] 13: 0x602b24511f8a - >::call_once [INFO] [stdout] 14: 0x602b24596f39 - serial_test[ddb617897473a8f9]::serial_code_lock::local_serial_core [INFO] [stdout] 15: 0x602b244f9ffd - >::call_once [INFO] [stdout] 16: 0x602b2455038b - test[aada8e7e76ba119b]::__rust_begin_short_backtrace::, fn() -> core[a373ad73a504971]::result::Result<(), alloc[2e1b2a31b91974c5]::string::String>> [INFO] [stdout] 17: 0x602b2455d565 - test[aada8e7e76ba119b]::run_test::{closure#0} [INFO] [stdout] 18: 0x602b24556e54 - std[617be2ca0a6fd877]::sys::backtrace::__rust_begin_short_backtrace:: [INFO] [stdout] 19: 0x602b245605f2 - ::{closure#1} as core[a373ad73a504971]::ops::function::FnOnce<()>>::call_once::{shim:vtable#0} [INFO] [stdout] 20: 0x602b245dfc2f - ::new::thread_start [INFO] [stdout] 21: 0x76bb120b2dfa - [INFO] [stdout] 22: 0x76bb121463d4 - clone [INFO] [stdout] 23: 0x0 - [INFO] [stdout] [INFO] [stdout] ---- catchup_then_immediate_failover stdout ---- [INFO] [stdout] [INFO] [stdout] thread 'catchup_then_immediate_failover' (7549) panicked at tests/failover.rs:275:13: [INFO] [stdout] only 0 of 2 replica(s) attached to 127.0.0.1:20092 within 30s — the test would race its own fixture from here [INFO] [stdout] stack backtrace: [INFO] [stdout] 0: 0x602b245e0c51 - <::print::DisplayBacktrace as core[a373ad73a504971]::fmt::Display>::fmt [INFO] [stdout] 1: 0x602b245f6fda - core[a373ad73a504971]::fmt::write [INFO] [stdout] 2: 0x602b245e54ac - as core[a373ad73a504971]::io::write::Write>::write_fmt [INFO] [stdout] 3: 0x602b245b8e86 - std[617be2ca0a6fd877]::panicking::default_hook::{closure#0} [INFO] [stdout] 4: 0x602b245d7079 - std[617be2ca0a6fd877]::panicking::default_hook [INFO] [stdout] 5: 0x602b2455cf40 - test[aada8e7e76ba119b]::test_main_inner::{closure#0} [INFO] [stdout] 6: 0x602b245d73a2 - std[617be2ca0a6fd877]::panicking::panic_with_hook [INFO] [stdout] 7: 0x602b245b8f32 - std[617be2ca0a6fd877]::panicking::panic_handler::{closure#0} [INFO] [stdout] 8: 0x602b245b18c9 - std[617be2ca0a6fd877]::sys::backtrace::__rust_end_short_backtrace:: [INFO] [stdout] 9: 0x602b245b9b3d - __rustc[ea4f61d10b84890b]::rust_begin_unwind [INFO] [stdout] 10: 0x602b245f773c - core[a373ad73a504971]::panicking::panic_fmt [INFO] [stdout] 11: 0x602b244f22e0 - failover[ab1c72d9133d71e1]::wait_for_replicas [INFO] [stdout] 12: 0x602b244f7009 - ::start_with_args [INFO] [stdout] 13: 0x602b245012d1 - >::call_once [INFO] [stdout] 14: 0x602b24596f39 - serial_test[ddb617897473a8f9]::serial_code_lock::local_serial_core [INFO] [stdout] 15: 0x602b244f967d - >::call_once [INFO] [stdout] 16: 0x602b2455038b - test[aada8e7e76ba119b]::__rust_begin_short_backtrace::, fn() -> core[a373ad73a504971]::result::Result<(), alloc[2e1b2a31b91974c5]::string::String>> [INFO] [stdout] 17: 0x602b2455d565 - test[aada8e7e76ba119b]::run_test::{closure#0} [INFO] [stdout] 18: 0x602b24556e54 - std[617be2ca0a6fd877]::sys::backtrace::__rust_begin_short_backtrace:: [INFO] [stdout] 19: 0x602b245605f2 - ::{closure#1} as core[a373ad73a504971]::ops::function::FnOnce<()>>::call_once::{shim:vtable#0} [INFO] [stdout] 20: 0x602b245dfc2f - ::new::thread_start [INFO] [stdout] 21: 0x76bb120b2dfa - [INFO] [stdout] 22: 0x76bb121463d4 - clone [INFO] [stdout] 23: 0x0 - [INFO] [stdout] [INFO] [stdout] ---- replacement_replica_catches_up_from_journal stdout ---- [INFO] [stdout] [INFO] [stdout] thread 'replacement_replica_catches_up_from_journal' (7647) panicked at tests/failover.rs:275:13: [INFO] [stdout] only 0 of 2 replica(s) attached to 127.0.0.1:20101 within 30s — the test would race its own fixture from here [INFO] [stdout] stack backtrace: [INFO] [stdout] 0: 0x602b245e0c51 - <::print::DisplayBacktrace as core[a373ad73a504971]::fmt::Display>::fmt [INFO] [stdout] 1: 0x602b245f6fda - core[a373ad73a504971]::fmt::write [INFO] [stdout] 2: 0x602b245e54ac - as core[a373ad73a504971]::io::write::Write>::write_fmt [INFO] [stdout] 3: 0x602b245b8e86 - std[617be2ca0a6fd877]::panicking::default_hook::{closure#0} [INFO] [stdout] 4: 0x602b245d7079 - std[617be2ca0a6fd877]::panicking::default_hook [INFO] [stdout] 5: 0x602b2455cf40 - test[aada8e7e76ba119b]::test_main_inner::{closure#0} [INFO] [stdout] 6: 0x602b245d73a2 - std[617be2ca0a6fd877]::panicking::panic_with_hook [INFO] [stdout] 7: 0x602b245b8f32 - std[617be2ca0a6fd877]::panicking::panic_handler::{closure#0} [INFO] [stdout] 8: 0x602b245b18c9 - std[617be2ca0a6fd877]::sys::backtrace::__rust_end_short_backtrace:: [INFO] [stdout] 9: 0x602b245b9b3d - __rustc[ea4f61d10b84890b]::rust_begin_unwind [INFO] [stdout] 10: 0x602b245f773c - core[a373ad73a504971]::panicking::panic_fmt [INFO] [stdout] 11: 0x602b244f22e0 - failover[ab1c72d9133d71e1]::wait_for_replicas [INFO] [stdout] 12: 0x602b244f7009 - ::start_with_args [INFO] [stdout] 13: 0x602b2450ce11 - >::call_once [INFO] [stdout] 14: 0x602b24596f39 - serial_test[ddb617897473a8f9]::serial_code_lock::local_serial_core [INFO] [stdout] 15: 0x602b244f9cfd - >::call_once [INFO] [stdout] 16: 0x602b2455038b - test[aada8e7e76ba119b]::__rust_begin_short_backtrace::, fn() -> core[a373ad73a504971]::result::Result<(), alloc[2e1b2a31b91974c5]::string::String>> [INFO] [stdout] 17: 0x602b2455d565 - test[aada8e7e76ba119b]::run_test::{closure#0} [INFO] [stdout] 18: 0x602b24556e54 - std[617be2ca0a6fd877]::sys::backtrace::__rust_begin_short_backtrace:: [INFO] [stdout] 19: 0x602b245605f2 - ::{closure#1} as core[a373ad73a504971]::ops::function::FnOnce<()>>::call_once::{shim:vtable#0} [INFO] [stdout] 20: 0x602b245dfc2f - ::new::thread_start [INFO] [stdout] 21: 0x76bb120b2dfa - [INFO] [stdout] 22: 0x76bb121463d4 - clone [INFO] [stdout] 23: 0x0 - [INFO] [stdout] [INFO] [stdout] ---- snapshot_transfer_when_archives_purged stdout ---- [INFO] [stdout] [INFO] [stdout] thread 'snapshot_transfer_when_archives_purged' (7772) panicked at tests/failover.rs:167:13: [INFO] [stdout] health endpoint 127.0.0.1:20110 did not respond within 30s [INFO] [stdout] stack backtrace: [INFO] [stdout] 0: 0x602b245e0c51 - <::print::DisplayBacktrace as core[a373ad73a504971]::fmt::Display>::fmt [INFO] [stdout] 1: 0x602b245f6fda - core[a373ad73a504971]::fmt::write [INFO] [stdout] 2: 0x602b245e54ac - as core[a373ad73a504971]::io::write::Write>::write_fmt [INFO] [stdout] 3: 0x602b245b8e86 - std[617be2ca0a6fd877]::panicking::default_hook::{closure#0} [INFO] [stdout] 4: 0x602b245d7079 - std[617be2ca0a6fd877]::panicking::default_hook [INFO] [stdout] 5: 0x602b2455cf40 - test[aada8e7e76ba119b]::test_main_inner::{closure#0} [INFO] [stdout] 6: 0x602b245d73a2 - std[617be2ca0a6fd877]::panicking::panic_with_hook [INFO] [stdout] 7: 0x602b245b8f32 - std[617be2ca0a6fd877]::panicking::panic_handler::{closure#0} [INFO] [stdout] 8: 0x602b245b18c9 - std[617be2ca0a6fd877]::sys::backtrace::__rust_end_short_backtrace:: [INFO] [stdout] 9: 0x602b245b9b3d - __rustc[ea4f61d10b84890b]::rust_begin_unwind [INFO] [stdout] 10: 0x602b245f773c - core[a373ad73a504971]::panicking::panic_fmt [INFO] [stdout] 11: 0x602b244ef848 - failover[ab1c72d9133d71e1]::snapshot_transfer_when_archives_purged::{closure#0} [INFO] [stdout] 12: 0x602b24596f39 - serial_test[ddb617897473a8f9]::serial_code_lock::local_serial_core [INFO] [stdout] 13: 0x602b244f997d - >::call_once [INFO] [stdout] 14: 0x602b2455038b - test[aada8e7e76ba119b]::__rust_begin_short_backtrace::, fn() -> core[a373ad73a504971]::result::Result<(), alloc[2e1b2a31b91974c5]::string::String>> [INFO] [stdout] 15: 0x602b2455d565 - test[aada8e7e76ba119b]::run_test::{closure#0} [INFO] [stderr] error: test failed, to rerun pass `--test failover` [INFO] [stdout] 16: 0x602b24556e54 - std[617be2ca0a6fd877]::sys::backtrace::__rust_begin_short_backtrace:: [INFO] [stdout] 17: 0x602b245605f2 - ::{closure#1} as core[a373ad73a504971]::ops::function::FnOnce<()>>::call_once::{shim:vtable#0} [INFO] [stdout] 18: 0x602b245dfc2f - ::new::thread_start [INFO] [stdout] 19: 0x76bb120b2dfa - [INFO] [stdout] 20: 0x76bb121463d4 - clone [INFO] [stdout] 21: 0x0 - [INFO] [stdout] [INFO] [stdout] ---- policy_degraded_gauge_transitions_with_cluster_shape stdout ---- [INFO] [stdout] [INFO] [stdout] thread 'policy_degraded_gauge_transitions_with_cluster_shape' (7595) panicked at tests/failover.rs:275:13: [INFO] [stdout] only 0 of 2 replica(s) attached to 127.0.0.1:20112 within 30s — the test would race its own fixture from here [INFO] [stdout] stack backtrace: [INFO] [stdout] 0: 0x602b245e0c51 - <::print::DisplayBacktrace as core[a373ad73a504971]::fmt::Display>::fmt [INFO] [stdout] 1: 0x602b245f6fda - core[a373ad73a504971]::fmt::write [INFO] [stdout] 2: 0x602b245e54ac - as core[a373ad73a504971]::io::write::Write>::write_fmt [INFO] [stdout] 3: 0x602b245b8e86 - std[617be2ca0a6fd877]::panicking::default_hook::{closure#0} [INFO] [stdout] 4: 0x602b245d7079 - std[617be2ca0a6fd877]::panicking::default_hook [INFO] [stdout] 5: 0x602b2455cf40 - test[aada8e7e76ba119b]::test_main_inner::{closure#0} [INFO] [stdout] 6: 0x602b245d73a2 - std[617be2ca0a6fd877]::panicking::panic_with_hook [INFO] [stdout] 7: 0x602b245b8f32 - std[617be2ca0a6fd877]::panicking::panic_handler::{closure#0} [INFO] [stdout] 8: 0x602b245b18c9 - std[617be2ca0a6fd877]::sys::backtrace::__rust_end_short_backtrace:: [INFO] [stdout] 9: 0x602b245b9b3d - __rustc[ea4f61d10b84890b]::rust_begin_unwind [INFO] [stdout] 10: 0x602b245f773c - core[a373ad73a504971]::panicking::panic_fmt [INFO] [stdout] 11: 0x602b244f22e0 - failover[ab1c72d9133d71e1]::wait_for_replicas [INFO] [stdout] 12: 0x602b244f7009 - ::start_with_args [INFO] [stdout] 13: 0x602b2450fac9 - >::call_once [INFO] [stdout] 14: 0x602b24596f39 - serial_test[ddb617897473a8f9]::serial_code_lock::local_serial_core [INFO] [stdout] 15: 0x602b244f9dfd - >::call_once [INFO] [stdout] 16: 0x602b2455038b - test[aada8e7e76ba119b]::__rust_begin_short_backtrace::, fn() -> core[a373ad73a504971]::result::Result<(), alloc[2e1b2a31b91974c5]::string::String>> [INFO] [stdout] 17: 0x602b2455d565 - test[aada8e7e76ba119b]::run_test::{closure#0} [INFO] [stdout] 18: 0x602b24556e54 - std[617be2ca0a6fd877]::sys::backtrace::__rust_begin_short_backtrace:: [INFO] [stdout] 19: 0x602b245605f2 - ::{closure#1} as core[a373ad73a504971]::ops::function::FnOnce<()>>::call_once::{shim:vtable#0} [INFO] [stdout] 20: 0x602b245dfc2f - ::new::thread_start [INFO] [stdout] 21: 0x76bb120b2dfa - [INFO] [stdout] 22: 0x76bb121463d4 - clone [INFO] [stdout] 23: 0x0 - [INFO] [stdout] [INFO] [stdout] ---- replica_reconnects_after_primary_restart_without_journal_wipe stdout ---- [INFO] [stdout] [INFO] [stdout] thread 'replica_reconnects_after_primary_restart_without_journal_wipe' (7668) panicked at tests/failover.rs:324:13: [INFO] [stdout] 127.0.0.1:20121 journaled only 0 of 12 seed events within 30s — the fixture's starting journal_seq is indeterminate [INFO] [stdout] stack backtrace: [INFO] [stdout] 0: 0x602b245e0c51 - <::print::DisplayBacktrace as core[a373ad73a504971]::fmt::Display>::fmt [INFO] [stdout] 1: 0x602b245f6fda - core[a373ad73a504971]::fmt::write [INFO] [stdout] 2: 0x602b245e54ac - as core[a373ad73a504971]::io::write::Write>::write_fmt [INFO] [stdout] 3: 0x602b245b8e86 - std[617be2ca0a6fd877]::panicking::default_hook::{closure#0} [INFO] [stdout] 4: 0x602b245d7079 - std[617be2ca0a6fd877]::panicking::default_hook [INFO] [stdout] 5: 0x602b2455cf40 - test[aada8e7e76ba119b]::test_main_inner::{closure#0} [INFO] [stdout] 6: 0x602b245d73a2 - std[617be2ca0a6fd877]::panicking::panic_with_hook [INFO] [stdout] 7: 0x602b245b8f32 - std[617be2ca0a6fd877]::panicking::panic_handler::{closure#0} [INFO] [stdout] 8: 0x602b245b18c9 - std[617be2ca0a6fd877]::sys::backtrace::__rust_end_short_backtrace:: [INFO] [stdout] 9: 0x602b245b9b3d - __rustc[ea4f61d10b84890b]::rust_begin_unwind [INFO] [stdout] 10: 0x602b245f773c - core[a373ad73a504971]::panicking::panic_fmt [INFO] [stdout] 11: 0x602b244f1b0c - failover[ab1c72d9133d71e1]::wait_for_seed [INFO] [stdout] 12: 0x602b244f81fa - ::start_with_extra_args [INFO] [stdout] 13: 0x602b24510f24 - >::call_once [INFO] [stdout] 14: 0x602b24596f39 - serial_test[ddb617897473a8f9]::serial_code_lock::local_serial_core [INFO] [stdout] 15: 0x602b244f9f7d - >::call_once [INFO] [stdout] 16: 0x602b2455038b - test[aada8e7e76ba119b]::__rust_begin_short_backtrace::, fn() -> core[a373ad73a504971]::result::Result<(), alloc[2e1b2a31b91974c5]::string::String>> [INFO] [stdout] 17: 0x602b2455d565 - test[aada8e7e76ba119b]::run_test::{closure#0} [INFO] [stdout] 18: 0x602b24556e54 - std[617be2ca0a6fd877]::sys::backtrace::__rust_begin_short_backtrace:: [INFO] [stdout] 19: 0x602b245605f2 - ::{closure#1} as core[a373ad73a504971]::ops::function::FnOnce<()>>::call_once::{shim:vtable#0} [INFO] [stdout] 20: 0x602b245dfc2f - ::new::thread_start [INFO] [stdout] 21: 0x76bb120b2dfa - [INFO] [stdout] 22: 0x76bb121463d4 - clone [INFO] [stdout] 23: 0x0 - [INFO] [stdout] [INFO] [stdout] ---- dual_replication_promote_replica1_after_replica2_dies stdout ---- [INFO] [stdout] [INFO] [stdout] thread 'dual_replication_promote_replica1_after_replica2_dies' (7554) panicked at tests/failover.rs:275:13: [INFO] [stdout] only 1 of 2 replica(s) attached to 127.0.0.1:20127 within 30s — the test would race its own fixture from here [INFO] [stdout] stack backtrace: [INFO] [stdout] 0: 0x602b245e0c51 - <::print::DisplayBacktrace as core[a373ad73a504971]::fmt::Display>::fmt [INFO] [stdout] 1: 0x602b245f6fda - core[a373ad73a504971]::fmt::write [INFO] [stdout] 2: 0x602b245e54ac - as core[a373ad73a504971]::io::write::Write>::write_fmt [INFO] [stdout] 3: 0x602b245b8e86 - std[617be2ca0a6fd877]::panicking::default_hook::{closure#0} [INFO] [stdout] 4: 0x602b245d7079 - std[617be2ca0a6fd877]::panicking::default_hook [INFO] [stdout] 5: 0x602b2455cf40 - test[aada8e7e76ba119b]::test_main_inner::{closure#0} [INFO] [stdout] 6: 0x602b245d73a2 - std[617be2ca0a6fd877]::panicking::panic_with_hook [INFO] [stdout] 7: 0x602b245b8f32 - std[617be2ca0a6fd877]::panicking::panic_handler::{closure#0} [INFO] [stdout] 8: 0x602b245b18c9 - std[617be2ca0a6fd877]::sys::backtrace::__rust_end_short_backtrace:: [INFO] [stdout] 9: 0x602b245b9b3d - __rustc[ea4f61d10b84890b]::rust_begin_unwind [INFO] [stdout] 10: 0x602b245f773c - core[a373ad73a504971]::panicking::panic_fmt [INFO] [stdout] 11: 0x602b244f22e0 - failover[ab1c72d9133d71e1]::wait_for_replicas [INFO] [stdout] 12: 0x602b244f7009 - ::start_with_args [INFO] [stdout] 13: 0x602b2450fca5 - >::call_once [INFO] [stdout] 14: 0x602b24596f39 - serial_test[ddb617897473a8f9]::serial_code_lock::local_serial_core [INFO] [stdout] 15: 0x602b244f9e7d - >::call_once [INFO] [stdout] 16: 0x602b2455038b - test[aada8e7e76ba119b]::__rust_begin_short_backtrace::, fn() -> core[a373ad73a504971]::result::Result<(), alloc[2e1b2a31b91974c5]::string::String>> [INFO] [stdout] 17: 0x602b2455d565 - test[aada8e7e76ba119b]::run_test::{closure#0} [INFO] [stdout] 18: 0x602b24556e54 - std[617be2ca0a6fd877]::sys::backtrace::__rust_begin_short_backtrace:: [INFO] [stdout] 19: 0x602b245605f2 - ::{closure#1} as core[a373ad73a504971]::ops::function::FnOnce<()>>::call_once::{shim:vtable#0} [INFO] [stdout] 20: 0x602b245dfc2f - ::new::thread_start [INFO] [stdout] 21: 0x76bb120b2dfa - [INFO] [stdout] 22: 0x76bb121463d4 - clone [INFO] [stdout] 23: 0x0 - [INFO] [stdout] [INFO] [stdout] ---- higher_epoch_handshake_fences_stale_primary stdout ---- [INFO] [stdout] [INFO] [stdout] thread 'higher_epoch_handshake_fences_stale_primary' (7559) panicked at tests/failover.rs:275:13: [INFO] [stdout] only 0 of 2 replica(s) attached to 127.0.0.1:20136 within 30s — the test would race its own fixture from here [INFO] [stdout] stack backtrace: [INFO] [stdout] 0: 0x602b245e0c51 - <::print::DisplayBacktrace as core[a373ad73a504971]::fmt::Display>::fmt [INFO] [stdout] 1: 0x602b245f6fda - core[a373ad73a504971]::fmt::write [INFO] [stdout] 2: 0x602b245e54ac - as core[a373ad73a504971]::io::write::Write>::write_fmt [INFO] [stdout] 3: 0x602b245b8e86 - std[617be2ca0a6fd877]::panicking::default_hook::{closure#0} [INFO] [stdout] 4: 0x602b245d7079 - std[617be2ca0a6fd877]::panicking::default_hook [INFO] [stdout] 5: 0x602b2455cf40 - test[aada8e7e76ba119b]::test_main_inner::{closure#0} [INFO] [stdout] 6: 0x602b245d73a2 - std[617be2ca0a6fd877]::panicking::panic_with_hook [INFO] [stdout] 7: 0x602b245b8f32 - std[617be2ca0a6fd877]::panicking::panic_handler::{closure#0} [INFO] [stdout] 8: 0x602b245b18c9 - std[617be2ca0a6fd877]::sys::backtrace::__rust_end_short_backtrace:: [INFO] [stdout] 9: 0x602b245b9b3d - __rustc[ea4f61d10b84890b]::rust_begin_unwind [INFO] [stdout] 10: 0x602b245f773c - core[a373ad73a504971]::panicking::panic_fmt [INFO] [stdout] 11: 0x602b244f22e0 - failover[ab1c72d9133d71e1]::wait_for_replicas [INFO] [stdout] 12: 0x602b2450babd - >::call_once [INFO] [stdout] 13: 0x602b24596f39 - serial_test[ddb617897473a8f9]::serial_code_lock::local_serial_core [INFO] [stdout] 14: 0x602b244f9c7d - >::call_once [INFO] [stdout] 15: 0x602b2455038b - test[aada8e7e76ba119b]::__rust_begin_short_backtrace::, fn() -> core[a373ad73a504971]::result::Result<(), alloc[2e1b2a31b91974c5]::string::String>> [INFO] [stdout] 16: 0x602b2455d565 - test[aada8e7e76ba119b]::run_test::{closure#0} [INFO] [stdout] 17: 0x602b24556e54 - std[617be2ca0a6fd877]::sys::backtrace::__rust_begin_short_backtrace:: [INFO] [stdout] 18: 0x602b245605f2 - ::{closure#1} as core[a373ad73a504971]::ops::function::FnOnce<()>>::call_once::{shim:vtable#0} [INFO] [stdout] 19: 0x602b245dfc2f - ::new::thread_start [INFO] [stdout] 20: 0x76bb120b2dfa - [INFO] [stdout] 21: 0x76bb121463d4 - clone [INFO] [stdout] 22: 0x0 - [INFO] [stdout] [INFO] [stdout] ---- sec04_rate_limit_replicates_to_replica stdout ---- [INFO] [stdout] [INFO] [stdout] thread 'sec04_rate_limit_replicates_to_replica' (7741) panicked at tests/failover.rs:275:13: [INFO] [stdout] only 0 of 1 replica(s) attached to 127.0.0.1:20145 within 30s — the test would race its own fixture from here [INFO] [stdout] stack backtrace: [INFO] [stdout] 0: 0x602b245e0c51 - <::print::DisplayBacktrace as core[a373ad73a504971]::fmt::Display>::fmt [INFO] [stdout] 1: 0x602b245f6fda - core[a373ad73a504971]::fmt::write [INFO] [stdout] 2: 0x602b245e54ac - as core[a373ad73a504971]::io::write::Write>::write_fmt [INFO] [stdout] 3: 0x602b245b8e86 - std[617be2ca0a6fd877]::panicking::default_hook::{closure#0} [INFO] [stdout] 4: 0x602b245d7079 - std[617be2ca0a6fd877]::panicking::default_hook [INFO] [stdout] 5: 0x602b2455cf40 - test[aada8e7e76ba119b]::test_main_inner::{closure#0} [INFO] [stdout] 6: 0x602b245d73a2 - std[617be2ca0a6fd877]::panicking::panic_with_hook [INFO] [stdout] 7: 0x602b245b8f32 - std[617be2ca0a6fd877]::panicking::panic_handler::{closure#0} [INFO] [stdout] 8: 0x602b245b18c9 - std[617be2ca0a6fd877]::sys::backtrace::__rust_end_short_backtrace:: [INFO] [stdout] 9: 0x602b245b9b3d - __rustc[ea4f61d10b84890b]::rust_begin_unwind [INFO] [stdout] 10: 0x602b245f773c - core[a373ad73a504971]::panicking::panic_fmt [INFO] [stdout] 11: 0x602b244f22e0 - failover[ab1c72d9133d71e1]::wait_for_replicas [INFO] [stdout] 12: 0x602b244f81d4 - ::start_with_extra_args [INFO] [stdout] 13: 0x602b245069f3 - >::call_once [INFO] [stdout] 14: 0x602b24596f39 - serial_test[ddb617897473a8f9]::serial_code_lock::local_serial_core [INFO] [stdout] 15: 0x602b244f98fd - >::call_once [INFO] [stdout] 16: 0x602b2455038b - test[aada8e7e76ba119b]::__rust_begin_short_backtrace::, fn() -> core[a373ad73a504971]::result::Result<(), alloc[2e1b2a31b91974c5]::string::String>> [INFO] [stdout] 17: 0x602b2455d565 - test[aada8e7e76ba119b]::run_test::{closure#0} [INFO] [stdout] 18: 0x602b24556e54 - std[617be2ca0a6fd877]::sys::backtrace::__rust_begin_short_backtrace:: [INFO] [stdout] 19: 0x602b245605f2 - ::{closure#1} as core[a373ad73a504971]::ops::function::FnOnce<()>>::call_once::{shim:vtable#0} [INFO] [stdout] 20: 0x602b245dfc2f - ::new::thread_start [INFO] [stdout] 21: 0x76bb120b2dfa - [INFO] [stdout] 22: 0x76bb121463d4 - clone [INFO] [stdout] 23: 0x0 - [INFO] [stdout] [INFO] [stdout] ---- same_key_request_seq_hwm_survives_failover stdout ---- [INFO] [stdout] [INFO] [stdout] thread 'same_key_request_seq_hwm_survives_failover' (7720) panicked at tests/failover.rs:275:13: [INFO] [stdout] only unknown of 1 replica(s) attached to 127.0.0.1:20151 within 30s — the test would race its own fixture from here [INFO] [stdout] stack backtrace: [INFO] [stdout] 0: 0x602b245e0c51 - <::print::DisplayBacktrace as core[a373ad73a504971]::fmt::Display>::fmt [INFO] [stdout] 1: 0x602b245f6fda - core[a373ad73a504971]::fmt::write [INFO] [stdout] 2: 0x602b245e54ac - as core[a373ad73a504971]::io::write::Write>::write_fmt [INFO] [stdout] 3: 0x602b245b8e86 - std[617be2ca0a6fd877]::panicking::default_hook::{closure#0} [INFO] [stdout] 4: 0x602b245d7079 - std[617be2ca0a6fd877]::panicking::default_hook [INFO] [stdout] 5: 0x602b2455cf40 - test[aada8e7e76ba119b]::test_main_inner::{closure#0} [INFO] [stdout] 6: 0x602b245d73a2 - std[617be2ca0a6fd877]::panicking::panic_with_hook [INFO] [stdout] 7: 0x602b245b8f32 - std[617be2ca0a6fd877]::panicking::panic_handler::{closure#0} [INFO] [stdout] 8: 0x602b245b18c9 - std[617be2ca0a6fd877]::sys::backtrace::__rust_end_short_backtrace:: [INFO] [stdout] 9: 0x602b245b9b3d - __rustc[ea4f61d10b84890b]::rust_begin_unwind [INFO] [stdout] 10: 0x602b245f773c - core[a373ad73a504971]::panicking::panic_fmt [INFO] [stdout] 11: 0x602b244f22e0 - failover[ab1c72d9133d71e1]::wait_for_replicas [INFO] [stdout] 12: 0x602b244f81d4 - ::start_with_extra_args [INFO] [stdout] 13: 0x602b24509fe0 - >::call_once [INFO] [stdout] 14: 0x602b24596f39 - serial_test[ddb617897473a8f9]::serial_code_lock::local_serial_core [INFO] [stdout] 15: 0x602b244f9b7d - >::call_once [INFO] [stdout] 16: 0x602b2455038b - test[aada8e7e76ba119b]::__rust_begin_short_backtrace::, fn() -> core[a373ad73a504971]::result::Result<(), alloc[2e1b2a31b91974c5]::string::String>> [INFO] [stdout] 17: 0x602b2455d565 - test[aada8e7e76ba119b]::run_test::{closure#0} [INFO] [stdout] 18: 0x602b24556e54 - std[617be2ca0a6fd877]::sys::backtrace::__rust_begin_short_backtrace:: [INFO] [stdout] 19: 0x602b245605f2 - ::{closure#1} as core[a373ad73a504971]::ops::function::FnOnce<()>>::call_once::{shim:vtable#0} [INFO] [stdout] 20: 0x602b245dfc2f - ::new::thread_start [INFO] [stdout] 21: 0x76bb120b2dfa - [INFO] [stdout] 22: 0x76bb121463d4 - clone [INFO] [stdout] 23: 0x0 - [INFO] [stdout] [INFO] [stdout] ---- dual_replication_survives_one_replica_failure stdout ---- [INFO] [stdout] [INFO] [stdout] thread 'dual_replication_survives_one_replica_failure' (7555) panicked at tests/failover.rs:275:13: [INFO] [stdout] only 0 of 2 replica(s) attached to 127.0.0.1:20157 within 30s — the test would race its own fixture from here [INFO] [stdout] stack backtrace: [INFO] [stdout] 0: 0x602b245e0c51 - <::print::DisplayBacktrace as core[a373ad73a504971]::fmt::Display>::fmt [INFO] [stdout] 1: 0x602b245f6fda - core[a373ad73a504971]::fmt::write [INFO] [stdout] 2: 0x602b245e54ac - as core[a373ad73a504971]::io::write::Write>::write_fmt [INFO] [stdout] 3: 0x602b245b8e86 - std[617be2ca0a6fd877]::panicking::default_hook::{closure#0} [INFO] [stdout] 4: 0x602b245d7079 - std[617be2ca0a6fd877]::panicking::default_hook [INFO] [stdout] 5: 0x602b2455cf40 - test[aada8e7e76ba119b]::test_main_inner::{closure#0} [INFO] [stdout] 6: 0x602b245d73a2 - std[617be2ca0a6fd877]::panicking::panic_with_hook [INFO] [stdout] 7: 0x602b245b8f32 - std[617be2ca0a6fd877]::panicking::panic_handler::{closure#0} [INFO] [stdout] 8: 0x602b245b18c9 - std[617be2ca0a6fd877]::sys::backtrace::__rust_end_short_backtrace:: [INFO] [stdout] 9: 0x602b245b9b3d - __rustc[ea4f61d10b84890b]::rust_begin_unwind [INFO] [stdout] 10: 0x602b245f773c - core[a373ad73a504971]::panicking::panic_fmt [INFO] [stdout] 11: 0x602b244f22e0 - failover[ab1c72d9133d71e1]::wait_for_replicas [INFO] [stdout] 12: 0x602b244f7009 - ::start_with_args [INFO] [stdout] 13: 0x602b2450f215 - >::call_once [INFO] [stdout] 14: 0x602b24596f39 - serial_test[ddb617897473a8f9]::serial_code_lock::local_serial_core [INFO] [stdout] 15: 0x602b244f9d7d - >::call_once [INFO] [stdout] 16: 0x602b2455038b - test[aada8e7e76ba119b]::__rust_begin_short_backtrace::, fn() -> core[a373ad73a504971]::result::Result<(), alloc[2e1b2a31b91974c5]::string::String>> [INFO] [stdout] 17: 0x602b2455d565 - test[aada8e7e76ba119b]::run_test::{closure#0} [INFO] [stdout] 18: 0x602b24556e54 - std[617be2ca0a6fd877]::sys::backtrace::__rust_begin_short_backtrace:: [INFO] [stdout] 19: 0x602b245605f2 - ::{closure#1} as core[a373ad73a504971]::ops::function::FnOnce<()>>::call_once::{shim:vtable#0} [INFO] [stdout] 20: 0x602b245dfc2f - ::new::thread_start [INFO] [stdout] 21: 0x76bb120b2dfa - [INFO] [stdout] 22: 0x76bb121463d4 - clone [INFO] [stdout] 23: 0x0 - [INFO] [stdout] [INFO] [stdout] ---- kill_during_active_fills stdout ---- [INFO] [stdout] [INFO] [stdout] thread 'kill_during_active_fills' (7562) panicked at tests/failover.rs:275:13: [INFO] [stdout] only 0 of 1 replica(s) attached to 127.0.0.1:20166 within 30s — the test would race its own fixture from here [INFO] [stdout] stack backtrace: [INFO] [stdout] 0: 0x602b245e0c51 - <::print::DisplayBacktrace as core[a373ad73a504971]::fmt::Display>::fmt [INFO] [stdout] 1: 0x602b245f6fda - core[a373ad73a504971]::fmt::write [INFO] [stdout] 2: 0x602b245e54ac - as core[a373ad73a504971]::io::write::Write>::write_fmt [INFO] [stdout] 3: 0x602b245b8e86 - std[617be2ca0a6fd877]::panicking::default_hook::{closure#0} [INFO] [stdout] 4: 0x602b245d7079 - std[617be2ca0a6fd877]::panicking::default_hook [INFO] [stdout] 5: 0x602b2455cf40 - test[aada8e7e76ba119b]::test_main_inner::{closure#0} [INFO] [stdout] 6: 0x602b245d73a2 - std[617be2ca0a6fd877]::panicking::panic_with_hook [INFO] [stdout] 7: 0x602b245b8f32 - std[617be2ca0a6fd877]::panicking::panic_handler::{closure#0} [INFO] [stdout] 8: 0x602b245b18c9 - std[617be2ca0a6fd877]::sys::backtrace::__rust_end_short_backtrace:: [INFO] [stdout] 9: 0x602b245b9b3d - __rustc[ea4f61d10b84890b]::rust_begin_unwind [INFO] [stdout] 10: 0x602b245f773c - core[a373ad73a504971]::panicking::panic_fmt [INFO] [stdout] 11: 0x602b244f22e0 - failover[ab1c72d9133d71e1]::wait_for_replicas [INFO] [stdout] 12: 0x602b244f81d4 - ::start_with_extra_args [INFO] [stdout] 13: 0x602b244fa045 - >::call_once [INFO] [stdout] 14: 0x602b24596f39 - serial_test[ddb617897473a8f9]::serial_code_lock::local_serial_core [INFO] [stdout] 15: 0x602b244f947d - >::call_once [INFO] [stdout] 16: 0x602b2455038b - test[aada8e7e76ba119b]::__rust_begin_short_backtrace::, fn() -> core[a373ad73a504971]::result::Result<(), alloc[2e1b2a31b91974c5]::string::String>> [INFO] [stdout] 17: 0x602b2455d565 - test[aada8e7e76ba119b]::run_test::{closure#0} [INFO] [stdout] 18: 0x602b24556e54 - std[617be2ca0a6fd877]::sys::backtrace::__rust_begin_short_backtrace:: [INFO] [stdout] 19: 0x602b245605f2 - ::{closure#1} as core[a373ad73a504971]::ops::function::FnOnce<()>>::call_once::{shim:vtable#0} [INFO] [stdout] 20: 0x602b245dfc2f - ::new::thread_start [INFO] [stdout] 21: 0x76bb120b2dfa - [INFO] [stdout] 22: 0x76bb121463d4 - clone [INFO] [stdout] 23: 0x0 - [INFO] [stdout] [INFO] [stdout] ---- crashed_primary_recovers_from_journal stdout ---- [INFO] [stdout] [INFO] [stdout] thread 'crashed_primary_recovers_from_journal' (7551) panicked at tests/failover.rs:275:13: [INFO] [stdout] only 0 of 1 replica(s) attached to 127.0.0.1:20172 within 30s — the test would race its own fixture from here [INFO] [stdout] stack backtrace: [INFO] [stdout] 0: 0x602b245e0c51 - <::print::DisplayBacktrace as core[a373ad73a504971]::fmt::Display>::fmt [INFO] [stdout] 1: 0x602b245f6fda - core[a373ad73a504971]::fmt::write [INFO] [stdout] 2: 0x602b245e54ac - as core[a373ad73a504971]::io::write::Write>::write_fmt [INFO] [stdout] 3: 0x602b245b8e86 - std[617be2ca0a6fd877]::panicking::default_hook::{closure#0} [INFO] [stdout] 4: 0x602b245d7079 - std[617be2ca0a6fd877]::panicking::default_hook [INFO] [stdout] 5: 0x602b2455cf40 - test[aada8e7e76ba119b]::test_main_inner::{closure#0} [INFO] [stdout] 6: 0x602b245d73a2 - std[617be2ca0a6fd877]::panicking::panic_with_hook [INFO] [stdout] 7: 0x602b245b8f32 - std[617be2ca0a6fd877]::panicking::panic_handler::{closure#0} [INFO] [stdout] 8: 0x602b245b18c9 - std[617be2ca0a6fd877]::sys::backtrace::__rust_end_short_backtrace:: [INFO] [stdout] 9: 0x602b245b9b3d - __rustc[ea4f61d10b84890b]::rust_begin_unwind [INFO] [stdout] 10: 0x602b245f773c - core[a373ad73a504971]::panicking::panic_fmt [INFO] [stdout] 11: 0x602b244f22e0 - failover[ab1c72d9133d71e1]::wait_for_replicas [INFO] [stdout] 12: 0x602b244f81d4 - ::start_with_extra_args [INFO] [stdout] 13: 0x602b24504365 - >::call_once [INFO] [stdout] 14: 0x602b24596f39 - serial_test[ddb617897473a8f9]::serial_code_lock::local_serial_core [INFO] [stdout] 15: 0x602b244f97fd - >::call_once [INFO] [stdout] 16: 0x602b2455038b - test[aada8e7e76ba119b]::__rust_begin_short_backtrace::, fn() -> core[a373ad73a504971]::result::Result<(), alloc[2e1b2a31b91974c5]::string::String>> [INFO] [stdout] 17: 0x602b2455d565 - test[aada8e7e76ba119b]::run_test::{closure#0} [INFO] [stdout] 18: 0x602b24556e54 - std[617be2ca0a6fd877]::sys::backtrace::__rust_begin_short_backtrace:: [INFO] [stdout] 19: 0x602b245605f2 - ::{closure#1} as core[a373ad73a504971]::ops::function::FnOnce<()>>::call_once::{shim:vtable#0} [INFO] [stdout] 20: 0x602b245dfc2f - ::new::thread_start [INFO] [stdout] 21: 0x76bb120b2dfa - [INFO] [stdout] 22: 0x76bb121463d4 - clone [INFO] [stdout] 23: 0x0 - [INFO] [stdout] [INFO] [stdout] [INFO] [stdout] failures: [INFO] [stdout] catchup_then_immediate_failover [INFO] [stdout] catchup_with_fills_during_gap [INFO] [stdout] crashed_primary_recovers_from_journal [INFO] [stdout] disk_ram_gate_stalls_while_replica_frozen [INFO] [stdout] dual_replication_halts_when_both_disconnect [INFO] [stdout] dual_replication_promote_replica1_after_replica2_dies [INFO] [stdout] dual_replication_survives_one_replica_failure [INFO] [stdout] dual_replication_with_fills_then_failover [INFO] [stdout] evicted_replica_catchup_under_load_preserves_dense_lineage [INFO] [stdout] fresh_replica_full_catchup [INFO] [stdout] higher_epoch_handshake_fences_stale_primary [INFO] [stdout] in_memory_cursor_runs_ahead_of_persisted_under_sustained_traffic [INFO] [stdout] journals_contiguous_across_replication [INFO] [stdout] kill_during_active_fills [INFO] [stdout] kill_primary_promote_replica_no_data_loss [INFO] [stdout] kill_without_waiting_for_replication [INFO] [stdout] policy_degraded_gauge_transitions_with_cluster_shape [INFO] [stdout] recovered_primary_ack_gate_holds [INFO] [stdout] replacement_replica_catches_up_from_journal [INFO] [stdout] replica_reconnects_after_primary_restart_without_journal_wipe [INFO] [stdout] rotation_soak_under_load [INFO] [stdout] same_key_request_seq_hwm_survives_failover [INFO] [stdout] sec04_rate_limit_replicates_to_replica [INFO] [stdout] snapshot_transfer_when_archives_purged [INFO] [stdout] [INFO] [stdout] test result: FAILED. 0 passed; 24 failed; 0 ignored; 0 measured; 0 filtered out; finished in 751.72s [INFO] [stdout] [INFO] running `Command { std: "docker" "inspect" "10c669e04f2a16b15d21c73f6f2d08c97cbc473105ebd58f305f122aa46aa314", kill_on_drop: false }` [INFO] running `Command { std: "docker" "rm" "-f" "10c669e04f2a16b15d21c73f6f2d08c97cbc473105ebd58f305f122aa46aa314", kill_on_drop: false }` [INFO] [stdout] 10c669e04f2a16b15d21c73f6f2d08c97cbc473105ebd58f305f122aa46aa314