[INFO] fetching crate infinite-db 0.4.0...
[INFO] testing infinite-db-0.4.0 against 1.100.0-beta.1+cargoflags=--release for beta-release-1.100-2
[INFO] extracting crate infinite-db 0.4.0 into /workspace/builds/worker-6-tc2/source
[INFO] started tweaking crates.io crate infinite-db 0.4.0
[INFO] removed 0 missing examples
[INFO] removed 0 missing tests
[INFO] finished tweaking crates.io crate infinite-db 0.4.0
[INFO] tweaked toml for crates.io crate infinite-db 0.4.0 written to /workspace/builds/worker-6-tc2/source/Cargo.toml
[INFO] validating manifest of crates.io crate infinite-db 0.4.0 on toolchain 1.100.0-beta.1
[INFO] running `Command { std: CARGO_HOME="/workspace/cargo-home" RUSTUP_HOME="/workspace/rustup-home" "/workspace/cargo-home/bin/cargo" "+1.100.0-beta.1" "metadata" "--manifest-path" "Cargo.toml" "--no-deps", kill_on_drop: false }`
[INFO] crate crates.io crate infinite-db 0.4.0 already has a lockfile, it will not be regenerated
[INFO] running `Command { std: CARGO_HOME="/workspace/cargo-home" RUSTUP_HOME="/workspace/rustup-home" "/workspace/cargo-home/bin/cargo" "+1.100.0-beta.1" "fetch" "--manifest-path" "Cargo.toml", kill_on_drop: false }`
[INFO] running `Command { std: "docker" "create" "-v" "/var/lib/crater-agent-workspace/builds/worker-6-tc2/source:/opt/rustwide/workdir:ro,Z" "-v" "/var/lib/crater-agent-workspace/builds/worker-6-tc2/target:/opt/rustwide/target:rw,Z" "-v" "/var/lib/crater-agent-workspace/cargo-home:/opt/rustwide/cargo-home:ro,Z" "-v" "/var/lib/crater-agent-workspace/rustup-home:/opt/rustwide/rustup-home:ro,Z" "-m" "1610612736" "--network" "none" "ghcr.io/rust-lang/crates-build-env/linux@sha256:77db811e55d90add9212f6832c23229e3de5f1c1f9905e8cb07319ea15df73ac" "sleep" "infinity", kill_on_drop: false }`
[INFO] [stdout] 228988b32ea3dd5575ba77f33ac6cdbb3c9d8feac412e4e9210972231f367449
[INFO] running `Command { std: "docker" "start" "228988b32ea3dd5575ba77f33ac6cdbb3c9d8feac412e4e9210972231f367449", kill_on_drop: false }`
[INFO] running `Command { std: "docker" "inspect" "228988b32ea3dd5575ba77f33ac6cdbb3c9d8feac412e4e9210972231f367449", kill_on_drop: false }`
[INFO] running `Command { std: "docker" "exec" "-e" "SOURCE_DIR=/opt/rustwide/workdir" "-e" "CARGO_HOME=/opt/rustwide/cargo-home" "-e" "RUSTUP_HOME=/opt/rustwide/rustup-home" "-e" "CARGO_TARGET_DIR=/opt/rustwide/target" "-w" "/opt/rustwide/workdir" "--user" "0:0" "228988b32ea3dd5575ba77f33ac6cdbb3c9d8feac412e4e9210972231f367449" "/opt/rustwide/cargo-home/bin/cargo" "+1.100.0-beta.1" "metadata" "--no-deps" "--format-version=1", kill_on_drop: false }`
[INFO] running `Command { std: "docker" "inspect" "228988b32ea3dd5575ba77f33ac6cdbb3c9d8feac412e4e9210972231f367449", kill_on_drop: false }`
[INFO] running `Command { std: "docker" "exec" "-e" "SOURCE_DIR=/opt/rustwide/workdir" "-e" "CARGO_HOME=/opt/rustwide/cargo-home" "-e" "RUSTUP_HOME=/opt/rustwide/rustup-home" "-e" "CARGO_TARGET_DIR=/opt/rustwide/target" "-e" "CARGO_INCREMENTAL=0" "-e" "RUST_BACKTRACE=full" "-e" "RUSTFLAGS=--cap-lints=warn" "-e" "RUSTDOCFLAGS=--cap-lints=warn" "-w" "/opt/rustwide/workdir" "--user" "0:0" "228988b32ea3dd5575ba77f33ac6cdbb3c9d8feac412e4e9210972231f367449" "/opt/rustwide/cargo-home/bin/cargo" "+1.100.0-beta.1" "build" "--frozen" "--message-format=json" "--release", kill_on_drop: false }`
[INFO] [stderr]    Compiling proc-macro2 v1.0.106
[INFO] [stderr]    Compiling quote v1.0.45
[INFO] [stderr]    Compiling parking_lot_core v0.9.12
[INFO] [stderr]    Compiling blake3 v1.8.4
[INFO] [stderr]    Compiling getrandom v0.3.4
[INFO] [stderr]    Compiling arc-swap v1.9.1
[INFO] [stderr]    Compiling dashmap v6.2.1
[INFO] [stderr]    Compiling parking_lot v0.12.5
[INFO] [stderr]    Compiling syn v2.0.117
[INFO] [stderr]    Compiling serde_derive v1.0.228
[INFO] [stderr]    Compiling serde v1.0.228
[INFO] [stderr]    Compiling infinite-db v0.4.0 (/opt/rustwide/workdir)
[INFO] [stderr] warning: unused dependency `getrandom`
[INFO] [stderr]   --> Cargo.toml:21:15
[INFO] [stderr]    |
[INFO] [stderr] 21 | [dependencies.getrandom]
[INFO] [stderr]    |               ^^^^^^^^^^
[INFO] [stderr]    |
[INFO] [stderr]    = note: `cargo::unused_dependencies` is set to `warn` by default
[INFO] [stderr] help: consider removing the dependency on `getrandom`
[INFO] [stderr] warning: `infinite-db` (manifest) generated 1 warning
[INFO] [stderr]     Finished `release` profile [optimized] target(s) in 28.87s
[INFO] running `Command { std: "docker" "inspect" "228988b32ea3dd5575ba77f33ac6cdbb3c9d8feac412e4e9210972231f367449", kill_on_drop: false }`
[INFO] running `Command { std: "docker" "exec" "-e" "SOURCE_DIR=/opt/rustwide/workdir" "-e" "CARGO_HOME=/opt/rustwide/cargo-home" "-e" "RUSTUP_HOME=/opt/rustwide/rustup-home" "-e" "CARGO_TARGET_DIR=/opt/rustwide/target" "-e" "CARGO_INCREMENTAL=0" "-e" "RUST_BACKTRACE=full" "-e" "RUSTFLAGS=--cap-lints=warn" "-e" "RUSTDOCFLAGS=--cap-lints=warn" "-w" "/opt/rustwide/workdir" "--user" "0:0" "228988b32ea3dd5575ba77f33ac6cdbb3c9d8feac412e4e9210972231f367449" "/opt/rustwide/cargo-home/bin/cargo" "+1.100.0-beta.1" "test" "--frozen" "--no-run" "--message-format=json" "--release", kill_on_drop: false }`
[INFO] [stderr]    Compiling zerocopy v0.8.50
[INFO] [stderr]    Compiling either v1.16.0
[INFO] [stderr]    Compiling zerocopy-derive v0.8.50
[INFO] [stderr]    Compiling clap v4.6.1
[INFO] [stderr]    Compiling tokio-macros v2.7.0
[INFO] [stderr]    Compiling is-terminal v0.4.17
[INFO] [stderr]    Compiling infinite-db v0.4.0 (/opt/rustwide/workdir)
[INFO] [stderr]    Compiling itertools v0.10.5
[INFO] [stderr]    Compiling rayon v1.12.0
[INFO] [stderr]    Compiling tokio v1.52.1
[INFO] [stderr]    Compiling criterion-plot v0.5.0
[INFO] [stderr]    Compiling half v2.7.1
[INFO] [stderr]    Compiling ciborium-ll v0.2.2
[INFO] [stderr]    Compiling ciborium v0.2.2
[INFO] [stderr]    Compiling criterion v0.5.1
[INFO] [stderr] warning: unused dependency `getrandom`
[INFO] [stderr]   --> Cargo.toml:21:15
[INFO] [stderr]    |
[INFO] [stderr] 21 | [dependencies.getrandom]
[INFO] [stderr]    |               ^^^^^^^^^^
[INFO] [stderr]    |
[INFO] [stderr]    = note: `cargo::unused_dependencies` is set to `warn` by default
[INFO] [stderr] help: consider removing the dependency on `getrandom`
[INFO] [stderr] warning: `infinite-db` (manifest) generated 1 warning
[INFO] [stderr]     Finished `release` profile [optimized] target(s) in 1m 19s
[INFO] running `Command { std: "docker" "inspect" "228988b32ea3dd5575ba77f33ac6cdbb3c9d8feac412e4e9210972231f367449", kill_on_drop: false }`
[INFO] running `Command { std: "docker" "exec" "-e" "SOURCE_DIR=/opt/rustwide/workdir" "-e" "CARGO_HOME=/opt/rustwide/cargo-home" "-e" "RUSTUP_HOME=/opt/rustwide/rustup-home" "-e" "CARGO_TARGET_DIR=/opt/rustwide/target" "-e" "CARGO_INCREMENTAL=0" "-e" "RUST_BACKTRACE=full" "-e" "RUSTFLAGS=--cap-lints=warn" "-e" "RUSTDOCFLAGS=--cap-lints=warn" "-w" "/opt/rustwide/workdir" "--user" "0:0" "228988b32ea3dd5575ba77f33ac6cdbb3c9d8feac412e4e9210972231f367449" "/opt/rustwide/cargo-home/bin/cargo" "+1.100.0-beta.1" "test" "--frozen" "--release", kill_on_drop: false }`
[INFO] [stderr] warning: unused dependency `getrandom`
[INFO] [stderr]   --> Cargo.toml:21:15
[INFO] [stderr]    |
[INFO] [stderr] 21 | [dependencies.getrandom]
[INFO] [stderr]    |               ^^^^^^^^^^
[INFO] [stderr]    |
[INFO] [stderr]    = note: `cargo::unused_dependencies` is set to `warn` by default
[INFO] [stderr] help: consider removing the dependency on `getrandom`
[INFO] [stderr] warning: `infinite-db` (manifest) generated 1 warning
[INFO] [stderr]     Finished `release` profile [optimized] target(s) in 0.16s
[INFO] [stderr]      Running unittests src/lib.rs (/opt/rustwide/target/release/build/infinite-db/50de2eb0f026f6cb/out/infinite_db-50de2eb0f026f6cb)
[INFO] [stdout] 
[INFO] [stdout] running 86 tests
[INFO] [stdout] test engine::derivation::tests::hash_partition_is_stable ... ok
[INFO] [stdout] test engine::derivation::watermark::tests::out_of_order_retire_does_not_skip ... ok
[INFO] [stdout] test engine::derivation::watermark::tests::failure_blocks_advancement ... ok
[INFO] [stdout] test engine::derivation::watermark::tests::complete_waits_on_outstanding_gap ... ok
[INFO] [stdout] test engine::hlc_clock::tests::backward_physical_bumps_logical ... ok
[INFO] [stdout] test engine::hlc_clock::tests::forward_jump_trusted ... ok
[INFO] [stdout] test engine::hlc_clock::tests::logical_exhaustion_bumps_physical ... ok
[INFO] [stdout] test engine::hlc_clock::tests::receive_advances_past_remote ... ok
[INFO] [stdout] test engine::hlc_clock::tests::stamps_monotone_under_fixed_physical ... ok
[INFO] [stdout] test engine::hypergraph::tests::prepare_deletes_tombstones_all_rows ... ok
[INFO] [stdout] test engine::hypergraph::tests::prepare_writes_edge_plus_index_rows_v2 ... ok
[INFO] [stdout] test engine::intent_checkpoint::tests::all_durable_without_checkpoint_is_uncommitted ... ok
[INFO] [stdout] test engine::intent_checkpoint::tests::split_committed_group_and_uncommitted_tail ... ok
[INFO] [stdout] test engine::session_wal_store::tests::merge_recovered_order_is_hlc_not_file_order ... ok
[INFO] [stdout] test engine::watermark::tests::allocate_registers_before_stable_observes ... ok
[INFO] [stdout] test engine::derivation::watermark::tests::per_session_watermark_independent_advance ... ok
[INFO] [stdout] test engine::watermark::tests::revision_range_nth ... ok
[INFO] [stdout] test engine::watermark::tests::stable_with_non_dense_outstanding_ids ... ok
[INFO] [stdout] test infinitedb_core::address::revision_bincode_tests::legacy_revision_bincode_roundtrip ... ok
[INFO] [stdout] test infinitedb_core::hyperedge::tests::directed_missing_tail_err ... ok
[INFO] [stdout] test infinitedb_core::flow_vector::tests::distinct_directions_distinct_buckets_at_default_granularity ... ok
[INFO] [stdout] test infinitedb_core::flow_vector::tests::zero_delta_maps_to_zero_buckets ... ok
[INFO] [stdout] test infinitedb_core::hlc::tests::pack_roundtrip ... ok
[INFO] [stdout] test infinitedb_core::hyperedge::tests::directed_requires_tail_and_head ... ok
[INFO] [stdout] test infinitedb_core::hyperedge_codec::tests::random_noise_never_panics ... ok
[INFO] [stdout] test infinitedb_core::hyperedge_codec::tests::v1_decodes_as_undirected_neutral ... ok
[INFO] [stdout] test infinitedb_core::hyperedge_codec::tests::v1_reencode_produces_v2 ... ok
[INFO] [stdout] test infinitedb_core::hyperedge_codec::tests::v2_roundtrip ... ok
[INFO] [stdout] test infinitedb_core::hyperedge_codec::tests::v2_tag_not_confused_with_small_id_v1 ... ok
[INFO] [stdout] test infinitedb_core::hyperedge_codec::tests::v3_decode_yields_none_computation ... ok
[INFO] [stdout] test infinitedb_core::hyperedge::tests::undirected_all_neutral_ok ... ok
[INFO] [stdout] test infinitedb_core::endpoint_index::tests::v2_payload_tag_roundtrip ... ok
[INFO] [stdout] test infinitedb_core::frame::tests::flatten_union_scope ... ok
[INFO] [stdout] test infinitedb_core::frame::tests::merge_session_union_combines_filters ... ok
[INFO] [stdout] test infinitedb_core::hyperedge_codec::tests::v3_roundtrip_with_provenance ... ok
[INFO] [stdout] test infinitedb_core::kind_catalog::tests::oblig_directed_rejects_undirected ... ok
[INFO] [stdout] test infinitedb_core::kind_catalog::tests::reject_policy_errors_for_unknown_kind ... ok
[INFO] [stdout] test infinitedb_core::frame::tests::merge_session_with_spaces_drops_filter ... ok
[INFO] [stdout] test infinitedb_core::endpoint_index::tests::query_bounds_pin_outgoing_polarity ... ok
[INFO] [stdout] test infinitedb_core::endpoint_index::tests::v2_inserts_polarity_between_coords_and_edge_id ... ok
[INFO] [stdout] test infinitedb_core::hlc::tests::global_sequence_monotone ... ok
[INFO] [stdout] test infinitedb_core::kind_catalog::tests::unknown_kind_reject_fires_before_directionality ... ok
[INFO] [stdout] test infinitedb_core::kind_catalog::tests::unknown_kind_skips_directionality_under_allow ... ok
[INFO] [stdout] test infinitedb_core::record_identity::tests::cached_and_unset_keys_group_equally ... ok
[INFO] [stdout] test infinitedb_core::revision_codec::tests::hlc_wire_roundtrip ... ok
[INFO] [stdout] test infinitedb_core::hyperedge::tests::storage_point_splits_id ... ok
[INFO] [stdout] test infinitedb_core::hyperedge_codec::tests::v4_roundtrip_with_authoring_and_computation ... ok
[INFO] [stdout] test infinitedb_core::hyperedge_codec::tests::v4_roundtrip_with_computation ... ok
[INFO] [stdout] test infinitedb_core::staleness::tests::diagnose_condemned_by_judgment ... ok
[INFO] [stdout] test infinitedb_core::kind_catalog::tests::oblig_directed_accepts_directed ... ok
[INFO] [stdout] test infinitedb_core::adapter::tests::kind_label_converts_to_core_kind ... ok
[INFO] [stdout] test infinitedb_core::hlc::tests::legacy_orders_before_hlc_era ... ok
[INFO] [stdout] test infinitedb_core::staleness_closure::tests::fresh_when_revision_matches ... ok
[INFO] [stdout] test infinitedb_core::traversal::tests::b_connectivity_requires_all_tails ... ok
[INFO] [stdout] test infinitedb_core::staleness_closure::tests::legacy_pin_matches_hlc_observed_sequence ... ok
[INFO] [stdout] test engine::watermark::tests::compute_stable_cases ... ok
[INFO] [stdout] test infinitedb_core::traversal::tests::acyclic_dag_and_cycle ... ok
[INFO] [stdout] test infinitedb_index::ordinal::tests::i32_order_preserved ... ok
[INFO] [stdout] test infinitedb_index::ordinal::tests::f32_order_preserved ... ok
[INFO] [stdout] test infinitedb_index::temporal::tests::roundtrip_u64 ... ok
[INFO] [stdout] test infinitedb_storage::gc::tests::keeps_all_when_horizon_is_zero ... ok
[INFO] [stdout] test infinitedb_storage::gc::tests::prunes_old_tombstones ... ok
[INFO] [stdout] test infinitedb_index::temporal::tests::tick_is_monotonic ... ok
[INFO] [stdout] test infinitedb_storage::gc::tests::prunes_old_versions ... ok
[INFO] [stdout] test infinitedb_core::staleness::tests::diagnose_stale_frame ... ok
[INFO] [stdout] test infinitedb_index::range_decompose::tests::bbox_corner_keys_are_covered ... ok
[INFO] [stdout] test infinitedb_core::staleness_closure::tests::stale_when_revision_superseded ... ok
[INFO] [stdout] test infinitedb_index::composite::tests::two_points_order_is_consistent ... ok
[INFO] [stdout] test infinitedb_index::hilbert::tests::locality_correlates_for_nearby_points ... ok
[INFO] [stdout] test infinitedb_storage::compaction::tests::compacts_two_blocks_into_one ... ok
[INFO] [stdout] test infinitedb_core::hyperedge::tests::undirected_with_polarity_err ... ok
[INFO] [stdout] test infinitedb_index::hilbert::tests::encode_decode_roundtrip_2d ... ok
[INFO] [stdout] test infinitedb_index::hilbert::tests::encode_decode_roundtrip_3d ... ok
[INFO] [stdout] test infinitedb_storage::hot_segment::tests::failed_sync_group_rolls_back ... ok
[INFO] [stdout] test infinitedb_storage::session_wal::tests::failed_sync_truncates_uncommitted_tail ... ok
[INFO] [stdout] test infinitedb_storage::hot_segment::tests::uncommitted_frames_not_replayed ... ok
[INFO] [stdout] test infinitedb_storage::session_wal::tests::uncommitted_frames_not_replayed ... ok
[INFO] [stdout] test infinitedb_storage::wal::tests::roundtrip_single_entry ... ok
[INFO] [stdout] test infinitedb_storage::wal::tests::multiple_entries_roundtrip ... ok
[INFO] [stdout] test infinitedb_storage::wal::tests::buffered_syncs_periodically ... ok
[INFO] [stdout] test infinitedb_storage::hot_segment::tests::roundtrip_append_and_read ... ok
[INFO] [stdout] test infinitedb_storage::wal::tests::rewrite_uses_single_sync ... ok
[INFO] [stdout] test engine::watermark::tests::allocation_registration_atomic_under_contention ... ok
[INFO] [stdout] test infinitedb_storage::compaction::tests::dedup_keeps_latest_revision ... ok
[INFO] [stdout] test engine::frame::tests::suppress_removes_condemned ... ok
[INFO] [stderr]      Running tests/concurrent_db.rs (/opt/rustwide/target/release/build/infinite-db/5698727d145cda88/out/concurrent_db-5698727d145cda88)
[INFO] [stdout] test infinitedb_storage::session_wal::tests::roundtrip_committed_frames ... ok
[INFO] [stdout] 
[INFO] [stdout] test result: ok. 86 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.09s
[INFO] [stdout] 
[INFO] [stdout] 
[INFO] [stdout] running 5 tests
[INFO] [stdout] test v4_format_written_on_create ... ok
[INFO] [stdout] test concurrent_enqueues_from_multiple_threads ... ok
[INFO] [stdout] test fire_and_forget_insert_visible_after_sync ... ok
[INFO] [stdout] test concurrent_reads_while_writing ... ok
[INFO] [stdout] test durability_after_reopen ... ok
[INFO] [stderr]      Running tests/concurrent_db_phase_c.rs (/opt/rustwide/target/release/build/infinite-db/2790f6242a2654a2/out/concurrent_db_phase_c-2790f6242a2654a2)
[INFO] [stdout] 
[INFO] [stdout] test result: ok. 5 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.06s
[INFO] [stdout] 
[INFO] [stdout] 
[INFO] [stdout] running 7 tests
[INFO] [stdout] test query_bbox_on_branch_sees_overlay_not_main ... ok
[INFO] [stdout] test merge_prefer_higher_revision ... ok
[INFO] [stdout] test merge_fast_forward_non_conflicting ... ok
[INFO] [stdout] test merge_conflict_detection ... ok
[INFO] [stdout] test lazy_hilbert_shards_spawn_on_first_write ... ok
[INFO] [stdout] test shard_parallel_same_branch ... ok
[INFO] [stdout] test parallel_branch_writes_no_overlap ... ok
[INFO] [stdout] 
[INFO] [stdout] test result: ok. 7 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.20s
[INFO] [stdout] 
[INFO] [stderr]      Running tests/crcw_correctness.rs (/opt/rustwide/target/release/build/infinite-db/3517276a9dc8c5ad/out/crcw_correctness-3517276a9dc8c5ad)
[INFO] [stdout] 
[INFO] [stdout] running 23 tests
[INFO] [stdout] test as_of_survives_auto_compaction_with_keep_all ... FAILED
[INFO] [stdout] test failed_group_not_resurrected_on_reopen ... FAILED
[INFO] [stdout] test endpoint_space_multi_shard_writes_isolated ... FAILED
[INFO] [stdout] test insert_many_round_trip ... ok
[INFO] [stdout] test branch_overlay_keys_do_not_collide ... ok
[INFO] [stdout] test read_txn_repeatable_under_concurrent_writes ... FAILED
[INFO] [stdout] test branch_overlay_survives_reopen ... ok
[INFO] [stdout] test include_tombstones_regression ... ok
[INFO] [stdout] test lazy_shard_provisioning ... ok
[INFO] [stdout] test query_bbox_respects_space_hilbert_precision_in_shard_prune ... ok
[INFO] [stdout] test large_space_id_register_and_write ... ok
[INFO] [stdout] test origin_point_hilbert_key_cached_on_insert ... ok
[INFO] [stdout] test stable_advances_past_failed_group_commit ... FAILED
[INFO] [stdout] test sync_barrier_publishes_enqueued_writes ... ok
[INFO] [stdout] test stable_revision_lag_and_catch_up ... ok
[INFO] [stdout] test latest_wins ... ok
[INFO] [stdout] test allocation_registration_not_visible_to_stable ... ok
[INFO] [stdout] test tombstone_resurrection ... ok
[INFO] [stdout] test tombstone_resurrection_across_seal_boundary ... ok
[INFO] [stdout] test latest_only_compaction_drops_history ... ok
[INFO] [stdout] test seal_window_no_duplicate_records_under_load ... ok
[INFO] [stdout] test compaction_removes_superseded_blocks_without_branches ... ok
[INFO] [stderr] error: test failed, to rerun pass `--test crcw_correctness`
[INFO] [stdout] test branch_base_blocks_survive_main_compaction ... ok
[INFO] [stdout] 
[INFO] [stdout] failures:
[INFO] [stdout] 
[INFO] [stdout] ---- as_of_survives_auto_compaction_with_keep_all stdout ----
[INFO] [stdout] 
[INFO] [stdout] thread 'as_of_survives_auto_compaction_with_keep_all' (2126) panicked at tests/crcw_correctness.rs:512:49:
[INFO] [stdout] called `Result::unwrap()` on an `Err` value: PoisonError { .. }
[INFO] [stdout] stack backtrace:
[INFO] [stdout]    0:     0x5dd550345501 - <<std[617be2ca0a6fd877]::sys::backtrace::BacktraceLock>::print::DisplayBacktrace as core[a373ad73a504971]::fmt::Display>::fmt
[INFO] [stdout]    1:     0x5dd55035a29a - core[a373ad73a504971]::fmt::write
[INFO] [stdout]    2:     0x5dd550349c3c - <alloc[2e1b2a31b91974c5]::vec::Vec<u8> as core[a373ad73a504971]::io::write::Write>::write_fmt
[INFO] [stdout]    3:     0x5dd55031f3f6 - std[617be2ca0a6fd877]::panicking::default_hook::{closure#0}
[INFO] [stdout]    4:     0x5dd55033c7c9 - std[617be2ca0a6fd877]::panicking::default_hook
[INFO] [stdout]    5:     0x5dd5501fcd60 - test[aada8e7e76ba119b]::test_main_inner::{closure#0}
[INFO] [stdout]    6:     0x5dd55033c982 - std[617be2ca0a6fd877]::panicking::panic_with_hook
[INFO] [stdout]    7:     0x5dd55031f4a2 - std[617be2ca0a6fd877]::panicking::panic_handler::{closure#0}
[INFO] [stdout]    8:     0x5dd550317e09 - std[617be2ca0a6fd877]::sys::backtrace::__rust_end_short_backtrace::<std[617be2ca0a6fd877]::panicking::panic_handler::{closure#0}, !>
[INFO] [stdout]    9:     0x5dd55032019d - __rustc[ea4f61d10b84890b]::rust_begin_unwind
[INFO] [stdout]   10:     0x5dd55035a9cc - core[a373ad73a504971]::panicking::panic_fmt
[INFO] [stdout]   11:     0x5dd55035a7a2 - core[a373ad73a504971]::result::unwrap_failed
[INFO] [stdout]   12:     0x5dd5501d23bb - <crcw_correctness[f4890ce5ee6cb67]::as_of_survives_auto_compaction_with_keep_all::{closure#0} as core[a373ad73a504971]::ops::function::FnOnce<()>>::call_once
[INFO] [stdout]   13:     0x5dd5501f01ab - test[aada8e7e76ba119b]::__rust_begin_short_backtrace::<core[a373ad73a504971]::result::Result<(), alloc[2e1b2a31b91974c5]::string::String>, fn() -> core[a373ad73a504971]::result::Result<(), alloc[2e1b2a31b91974c5]::string::String>>
[INFO] [stdout]   14:     0x5dd5501fd385 - test[aada8e7e76ba119b]::run_test::{closure#0}
[INFO] [stdout]   15:     0x5dd5501f6c74 - std[617be2ca0a6fd877]::sys::backtrace::__rust_begin_short_backtrace::<test[aada8e7e76ba119b]::run_test::{closure#1}, ()>
[INFO] [stdout]   16:     0x5dd550200412 - <std[617be2ca0a6fd877]::thread::lifecycle::spawn_unchecked<test[aada8e7e76ba119b]::run_test::{closure#1}, ()>::{closure#1} as core[a373ad73a504971]::ops::function::FnOnce<()>>::call_once::{shim:vtable#0}
[INFO] [stdout]   17:     0x5dd55034483f - <std[617be2ca0a6fd877]::sys::thread::unix::Thread>::new::thread_start
[INFO] [stdout]   18:     0x7f91bebe7dfa - <unknown>
[INFO] [stdout]   19:     0x7f91bec7b3d4 - clone
[INFO] [stdout]   20:                0x0 - <unknown>
[INFO] [stdout] 
[INFO] [stdout] ---- failed_group_not_resurrected_on_reopen stdout ----
[INFO] [stdout] 
[INFO] [stdout] thread 'failed_group_not_resurrected_on_reopen' (2132) panicked at tests/crcw_correctness.rs:292:19:
[INFO] [stdout] called `Result::unwrap_err()` on an `Ok` value: ()
[INFO] [stdout] stack backtrace:
[INFO] [stdout]    0:     0x5dd550345501 - <<std[617be2ca0a6fd877]::sys::backtrace::BacktraceLock>::print::DisplayBacktrace as core[a373ad73a504971]::fmt::Display>::fmt
[INFO] [stdout]    1:     0x5dd55035a29a - core[a373ad73a504971]::fmt::write
[INFO] [stdout]    2:     0x5dd550349c3c - <alloc[2e1b2a31b91974c5]::vec::Vec<u8> as core[a373ad73a504971]::io::write::Write>::write_fmt
[INFO] [stdout]    3:     0x5dd55031f3f6 - std[617be2ca0a6fd877]::panicking::default_hook::{closure#0}
[INFO] [stdout]    4:     0x5dd55033c7c9 - std[617be2ca0a6fd877]::panicking::default_hook
[INFO] [stdout]    5:     0x5dd5501fcd60 - test[aada8e7e76ba119b]::test_main_inner::{closure#0}
[INFO] [stdout]    6:     0x5dd55033c982 - std[617be2ca0a6fd877]::panicking::panic_with_hook
[INFO] [stdout]    7:     0x5dd55031f4a2 - std[617be2ca0a6fd877]::panicking::panic_handler::{closure#0}
[INFO] [stdout]    8:     0x5dd550317e09 - std[617be2ca0a6fd877]::sys::backtrace::__rust_end_short_backtrace::<std[617be2ca0a6fd877]::panicking::panic_handler::{closure#0}, !>
[INFO] [stdout]    9:     0x5dd55032019d - __rustc[ea4f61d10b84890b]::rust_begin_unwind
[INFO] [stdout]   10:     0x5dd55035a9cc - core[a373ad73a504971]::panicking::panic_fmt
[INFO] [stdout]   11:     0x5dd55035a7a2 - core[a373ad73a504971]::result::unwrap_failed
[INFO] [stdout]   12:     0x5dd5501c9ba5 - <crcw_correctness[f4890ce5ee6cb67]::failed_group_not_resurrected_on_reopen::{closure#0} as core[a373ad73a504971]::ops::function::FnOnce<()>>::call_once
[INFO] [stdout]   13:     0x5dd5501f01ab - test[aada8e7e76ba119b]::__rust_begin_short_backtrace::<core[a373ad73a504971]::result::Result<(), alloc[2e1b2a31b91974c5]::string::String>, fn() -> core[a373ad73a504971]::result::Result<(), alloc[2e1b2a31b91974c5]::string::String>>
[INFO] [stdout]   14:     0x5dd5501fd385 - test[aada8e7e76ba119b]::run_test::{closure#0}
[INFO] [stdout]   15:     0x5dd5501f6c74 - std[617be2ca0a6fd877]::sys::backtrace::__rust_begin_short_backtrace::<test[aada8e7e76ba119b]::run_test::{closure#1}, ()>
[INFO] [stdout]   16:     0x5dd550200412 - <std[617be2ca0a6fd877]::thread::lifecycle::spawn_unchecked<test[aada8e7e76ba119b]::run_test::{closure#1}, ()>::{closure#1} as core[a373ad73a504971]::ops::function::FnOnce<()>>::call_once::{shim:vtable#0}
[INFO] [stdout]   17:     0x5dd55034483f - <std[617be2ca0a6fd877]::sys::thread::unix::Thread>::new::thread_start
[INFO] [stdout]   18:     0x7f91bebe7dfa - <unknown>
[INFO] [stdout]   19:     0x7f91bec7b3d4 - clone
[INFO] [stdout]   20:                0x0 - <unknown>
[INFO] [stdout] 
[INFO] [stdout] ---- endpoint_space_multi_shard_writes_isolated stdout ----
[INFO] [stdout] 
[INFO] [stdout] thread 'endpoint_space_multi_shard_writes_isolated' (2131) panicked at tests/crcw_correctness.rs:406:15:
[INFO] [stdout] called `Result::unwrap()` on an `Err` value: Storage(Io { kind: Other, message: "injected hot segment fsync failure", path: None })
[INFO] [stdout] stack backtrace:
[INFO] [stdout]    0:     0x5dd550345501 - <<std[617be2ca0a6fd877]::sys::backtrace::BacktraceLock>::print::DisplayBacktrace as core[a373ad73a504971]::fmt::Display>::fmt
[INFO] [stdout]    1:     0x5dd55035a29a - core[a373ad73a504971]::fmt::write
[INFO] [stdout]    2:     0x5dd550349c3c - <alloc[2e1b2a31b91974c5]::vec::Vec<u8> as core[a373ad73a504971]::io::write::Write>::write_fmt
[INFO] [stdout]    3:     0x5dd55031f3f6 - std[617be2ca0a6fd877]::panicking::default_hook::{closure#0}
[INFO] [stdout]    4:     0x5dd55033c7c9 - std[617be2ca0a6fd877]::panicking::default_hook
[INFO] [stdout]    5:     0x5dd5501fcd60 - test[aada8e7e76ba119b]::test_main_inner::{closure#0}
[INFO] [stdout]    6:     0x5dd55033c982 - std[617be2ca0a6fd877]::panicking::panic_with_hook
[INFO] [stdout]    7:     0x5dd55031f4a2 - std[617be2ca0a6fd877]::panicking::panic_handler::{closure#0}
[INFO] [stdout]    8:     0x5dd550317e09 - std[617be2ca0a6fd877]::sys::backtrace::__rust_end_short_backtrace::<std[617be2ca0a6fd877]::panicking::panic_handler::{closure#0}, !>
[INFO] [stdout]    9:     0x5dd55032019d - __rustc[ea4f61d10b84890b]::rust_begin_unwind
[INFO] [stdout]   10:     0x5dd55035a9cc - core[a373ad73a504971]::panicking::panic_fmt
[INFO] [stdout]   11:     0x5dd55035a7a2 - core[a373ad73a504971]::result::unwrap_failed
[INFO] [stdout]   12:     0x5dd5501cef22 - <crcw_correctness[f4890ce5ee6cb67]::endpoint_space_multi_shard_writes_isolated::{closure#0} as core[a373ad73a504971]::ops::function::FnOnce<()>>::call_once
[INFO] [stdout]   13:     0x5dd5501f01ab - test[aada8e7e76ba119b]::__rust_begin_short_backtrace::<core[a373ad73a504971]::result::Result<(), alloc[2e1b2a31b91974c5]::string::String>, fn() -> core[a373ad73a504971]::result::Result<(), alloc[2e1b2a31b91974c5]::string::String>>
[INFO] [stdout]   14:     0x5dd5501fd385 - test[aada8e7e76ba119b]::run_test::{closure#0}
[INFO] [stdout]   15:     0x5dd5501f6c74 - std[617be2ca0a6fd877]::sys::backtrace::__rust_begin_short_backtrace::<test[aada8e7e76ba119b]::run_test::{closure#1}, ()>
[INFO] [stdout]   16:     0x5dd550200412 - <std[617be2ca0a6fd877]::thread::lifecycle::spawn_unchecked<test[aada8e7e76ba119b]::run_test::{closure#1}, ()>::{closure#1} as core[a373ad73a504971]::ops::function::FnOnce<()>>::call_once::{shim:vtable#0}
[INFO] [stdout]   17:     0x5dd55034483f - <std[617be2ca0a6fd877]::sys::thread::unix::Thread>::new::thread_start
[INFO] [stdout]   18:     0x7f91bebe7dfa - <unknown>
[INFO] [stdout]   19:     0x7f91bec7b3d4 - clone
[INFO] [stdout]   20:                0x0 - <unknown>
[INFO] [stdout] 
[INFO] [stdout] ---- read_txn_repeatable_under_concurrent_writes stdout ----
[INFO] [stdout] 
[INFO] [stdout] thread 'read_txn_repeatable_under_concurrent_writes' (2283) panicked at tests/crcw_correctness.rs:209:15:
[INFO] [stdout] called `Result::unwrap()` on an `Err` value: Storage(Io { kind: Other, message: "injected hot segment fsync failure", path: None })
[INFO] [stdout] stack backtrace:
[INFO] [stdout]    0:     0x5dd550345501 - <<std[617be2ca0a6fd877]::sys::backtrace::BacktraceLock>::print::DisplayBacktrace as core[a373ad73a504971]::fmt::Display>::fmt
[INFO] [stdout]    1:     0x5dd55035a29a - core[a373ad73a504971]::fmt::write
[INFO] [stdout]    2:     0x5dd550349c3c - <alloc[2e1b2a31b91974c5]::vec::Vec<u8> as core[a373ad73a504971]::io::write::Write>::write_fmt
[INFO] [stdout]    3:     0x5dd55031f3f6 - std[617be2ca0a6fd877]::panicking::default_hook::{closure#0}
[INFO] [stdout]    4:     0x5dd55033c7c9 - std[617be2ca0a6fd877]::panicking::default_hook
[INFO] [stdout]    5:     0x5dd5501fcd60 - test[aada8e7e76ba119b]::test_main_inner::{closure#0}
[INFO] [stdout]    6:     0x5dd55033c982 - std[617be2ca0a6fd877]::panicking::panic_with_hook
[INFO] [stdout]    7:     0x5dd55031f4a2 - std[617be2ca0a6fd877]::panicking::panic_handler::{closure#0}
[INFO] [stdout]    8:     0x5dd550317e09 - std[617be2ca0a6fd877]::sys::backtrace::__rust_end_short_backtrace::<std[617be2ca0a6fd877]::panicking::panic_handler::{closure#0}, !>
[INFO] [stdout]    9:     0x5dd55032019d - __rustc[ea4f61d10b84890b]::rust_begin_unwind
[INFO] [stdout]   10:     0x5dd55035a9cc - core[a373ad73a504971]::panicking::panic_fmt
[INFO] [stdout]   11:     0x5dd55035a7a2 - core[a373ad73a504971]::result::unwrap_failed
[INFO] [stdout]   12:     0x5dd5501cfbec - <crcw_correctness[f4890ce5ee6cb67]::read_txn_repeatable_under_concurrent_writes::{closure#0} as core[a373ad73a504971]::ops::function::FnOnce<()>>::call_once
[INFO] [stdout]   13:     0x5dd5501f01ab - test[aada8e7e76ba119b]::__rust_begin_short_backtrace::<core[a373ad73a504971]::result::Result<(), alloc[2e1b2a31b91974c5]::string::String>, fn() -> core[a373ad73a504971]::result::Result<(), alloc[2e1b2a31b91974c5]::string::String>>
[INFO] [stdout]   14:     0x5dd5501fd385 - test[aada8e7e76ba119b]::run_test::{closure#0}
[INFO] [stdout]   15:     0x5dd5501f6c74 - std[617be2ca0a6fd877]::sys::backtrace::__rust_begin_short_backtrace::<test[aada8e7e76ba119b]::run_test::{closure#1}, ()>
[INFO] [stdout]   16:     0x5dd550200412 - <std[617be2ca0a6fd877]::thread::lifecycle::spawn_unchecked<test[aada8e7e76ba119b]::run_test::{closure#1}, ()>::{closure#1} as core[a373ad73a504971]::ops::function::FnOnce<()>>::call_once::{shim:vtable#0}
[INFO] [stdout]   17:     0x5dd55034483f - <std[617be2ca0a6fd877]::sys::thread::unix::Thread>::new::thread_start
[INFO] [stdout]   18:     0x7f91bebe7dfa - <unknown>
[INFO] [stdout]   19:     0x7f91bec7b3d4 - clone
[INFO] [stdout]   20:                0x0 - <unknown>
[INFO] [stdout] 
[INFO] [stdout] ---- stable_advances_past_failed_group_commit stdout ----
[INFO] [stdout] 
[INFO] [stdout] thread 'stable_advances_past_failed_group_commit' (2295) panicked at tests/crcw_correctness.rs:253:15:
[INFO] [stdout] called `Result::unwrap_err()` on an `Ok` value: ()
[INFO] [stdout] stack backtrace:
[INFO] [stdout]    0:     0x5dd550345501 - <<std[617be2ca0a6fd877]::sys::backtrace::BacktraceLock>::print::DisplayBacktrace as core[a373ad73a504971]::fmt::Display>::fmt
[INFO] [stdout]    1:     0x5dd55035a29a - core[a373ad73a504971]::fmt::write
[INFO] [stdout]    2:     0x5dd550349c3c - <alloc[2e1b2a31b91974c5]::vec::Vec<u8> as core[a373ad73a504971]::io::write::Write>::write_fmt
[INFO] [stdout]    3:     0x5dd55031f3f6 - std[617be2ca0a6fd877]::panicking::default_hook::{closure#0}
[INFO] [stdout]    4:     0x5dd55033c7c9 - std[617be2ca0a6fd877]::panicking::default_hook
[INFO] [stdout]    5:     0x5dd5501fcd60 - test[aada8e7e76ba119b]::test_main_inner::{closure#0}
[INFO] [stdout]    6:     0x5dd55033c982 - std[617be2ca0a6fd877]::panicking::panic_with_hook
[INFO] [stdout]    7:     0x5dd55031f4a2 - std[617be2ca0a6fd877]::panicking::panic_handler::{closure#0}
[INFO] [stdout]    8:     0x5dd550317e09 - std[617be2ca0a6fd877]::sys::backtrace::__rust_end_short_backtrace::<std[617be2ca0a6fd877]::panicking::panic_handler::{closure#0}, !>
[INFO] [stdout]    9:     0x5dd55032019d - __rustc[ea4f61d10b84890b]::rust_begin_unwind
[INFO] [stdout]   10:     0x5dd55035a9cc - core[a373ad73a504971]::panicking::panic_fmt
[INFO] [stdout]   11:     0x5dd55035a7a2 - core[a373ad73a504971]::result::unwrap_failed
[INFO] [stdout]   12:     0x5dd5501cb052 - <crcw_correctness[f4890ce5ee6cb67]::stable_advances_past_failed_group_commit::{closure#0} as core[a373ad73a504971]::ops::function::FnOnce<()>>::call_once
[INFO] [stdout]   13:     0x5dd5501f01ab - test[aada8e7e76ba119b]::__rust_begin_short_backtrace::<core[a373ad73a504971]::result::Result<(), alloc[2e1b2a31b91974c5]::string::String>, fn() -> core[a373ad73a504971]::result::Result<(), alloc[2e1b2a31b91974c5]::string::String>>
[INFO] [stdout]   14:     0x5dd5501fd385 - test[aada8e7e76ba119b]::run_test::{closure#0}
[INFO] [stdout]   15:     0x5dd5501f6c74 - std[617be2ca0a6fd877]::sys::backtrace::__rust_begin_short_backtrace::<test[aada8e7e76ba119b]::run_test::{closure#1}, ()>
[INFO] [stdout]   16:     0x5dd550200412 - <std[617be2ca0a6fd877]::thread::lifecycle::spawn_unchecked<test[aada8e7e76ba119b]::run_test::{closure#1}, ()>::{closure#1} as core[a373ad73a504971]::ops::function::FnOnce<()>>::call_once::{shim:vtable#0}
[INFO] [stdout]   17:     0x5dd55034483f - <std[617be2ca0a6fd877]::sys::thread::unix::Thread>::new::thread_start
[INFO] [stdout]   18:     0x7f91bebe7dfa - <unknown>
[INFO] [stdout]   19:     0x7f91bec7b3d4 - clone
[INFO] [stdout]   20:                0x0 - <unknown>
[INFO] [stdout] 
[INFO] [stdout] 
[INFO] [stdout] failures:
[INFO] [stdout]     as_of_survives_auto_compaction_with_keep_all
[INFO] [stdout]     endpoint_space_multi_shard_writes_isolated
[INFO] [stdout]     failed_group_not_resurrected_on_reopen
[INFO] [stdout]     read_txn_repeatable_under_concurrent_writes
[INFO] [stdout]     stable_advances_past_failed_group_commit
[INFO] [stdout] 
[INFO] [stdout] test result: FAILED. 18 passed; 5 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.47s
[INFO] [stdout] 
[INFO] running `Command { std: "docker" "inspect" "228988b32ea3dd5575ba77f33ac6cdbb3c9d8feac412e4e9210972231f367449", kill_on_drop: false }`
[INFO] running `Command { std: "docker" "rm" "-f" "228988b32ea3dd5575ba77f33ac6cdbb3c9d8feac412e4e9210972231f367449", kill_on_drop: false }`
[INFO] [stdout] 228988b32ea3dd5575ba77f33ac6cdbb3c9d8feac412e4e9210972231f367449
