[INFO] cloning repository https://github.com/rmax-ai/shardlake
[INFO] running `Command { std: "git" "-c" "credential.helper=" "-c" "credential.helper=/workspace/cargo-home/bin/git-credential-null" "clone" "--bare" "https://github.com/rmax-ai/shardlake" "/workspace/cache/git-repos/https%3A%2F%2Fgithub.com%2Frmax-ai%2Fshardlake", kill_on_drop: false }`
[INFO] [stderr] Cloning into bare repository '/workspace/cache/git-repos/https%3A%2F%2Fgithub.com%2Frmax-ai%2Fshardlake'...
[INFO] running `Command { std: "git" "rev-parse" "HEAD", kill_on_drop: false }`
[INFO] [stdout] 78f588ba5510b4d0af73f462c68892f10605a216
[INFO] testing rmax-ai/shardlake against try#33835004928d3bf65db4d4712e1330766263b0bd for pr-155739
[INFO] running `Command { std: "git" "clone" "/workspace/cache/git-repos/https%3A%2F%2Fgithub.com%2Frmax-ai%2Fshardlake" "/workspace/builds/worker-5-tc2/source", kill_on_drop: false }`
[INFO] [stderr] Cloning into '/workspace/builds/worker-5-tc2/source'...
[INFO] [stderr] done.
[INFO] started tweaking git repo https://github.com/rmax-ai/shardlake
[INFO] finished tweaking git repo https://github.com/rmax-ai/shardlake
[INFO] tweaked toml for git repo https://github.com/rmax-ai/shardlake written to /workspace/builds/worker-5-tc2/source/Cargo.toml
[INFO] validating manifest of git repo https://github.com/rmax-ai/shardlake on toolchain 33835004928d3bf65db4d4712e1330766263b0bd
[INFO] running `Command { std: CARGO_HOME="/workspace/cargo-home" RUSTUP_HOME="/workspace/rustup-home" "/workspace/cargo-home/bin/cargo" "+33835004928d3bf65db4d4712e1330766263b0bd" "metadata" "--manifest-path" "Cargo.toml" "--no-deps", kill_on_drop: false }`
[INFO] crate git repo https://github.com/rmax-ai/shardlake already has a lockfile, it will not be regenerated
[INFO] running `Command { std: CARGO_HOME="/workspace/cargo-home" RUSTUP_HOME="/workspace/rustup-home" "/workspace/cargo-home/bin/cargo" "+33835004928d3bf65db4d4712e1330766263b0bd" "fetch" "--manifest-path" "Cargo.toml", kill_on_drop: false }`
[INFO] running `Command { std: "docker" "create" "-v" "/var/lib/crater-agent-workspace/builds/worker-5-tc2/target:/opt/rustwide/target:rw,Z" "-v" "/var/lib/crater-agent-workspace/builds/worker-5-tc2/source:/opt/rustwide/workdir:ro,Z" "-v" "/var/lib/crater-agent-workspace/cargo-home:/opt/rustwide/cargo-home:ro,Z" "-v" "/var/lib/crater-agent-workspace/rustup-home:/opt/rustwide/rustup-home:ro,Z" "-e" "SOURCE_DIR=/opt/rustwide/workdir" "-e" "CARGO_TARGET_DIR=/opt/rustwide/target" "-e" "CARGO_HOME=/opt/rustwide/cargo-home" "-e" "RUSTUP_HOME=/opt/rustwide/rustup-home" "-w" "/opt/rustwide/workdir" "-m" "1610612736" "--user" "0:0" "--network" "none" "ghcr.io/rust-lang/crates-build-env/linux@sha256:d429b63d4308055ea97f60fb1d3dfca48854a00942f1bd2ad806beaf015945ec" "/opt/rustwide/cargo-home/bin/cargo" "+33835004928d3bf65db4d4712e1330766263b0bd" "metadata" "--no-deps" "--format-version=1", kill_on_drop: false }`
[INFO] [stdout] ec71df5f28493e11b013eca6d69c58ab5f23483b6943b8d890ffebb331697a6f
[INFO] running `Command { std: "docker" "start" "-a" "ec71df5f28493e11b013eca6d69c58ab5f23483b6943b8d890ffebb331697a6f", kill_on_drop: false }`
[INFO] running `Command { std: "docker" "inspect" "ec71df5f28493e11b013eca6d69c58ab5f23483b6943b8d890ffebb331697a6f", kill_on_drop: false }`
[INFO] running `Command { std: "docker" "rm" "-f" "ec71df5f28493e11b013eca6d69c58ab5f23483b6943b8d890ffebb331697a6f", kill_on_drop: false }`
[INFO] [stdout] ec71df5f28493e11b013eca6d69c58ab5f23483b6943b8d890ffebb331697a6f
[INFO] running `Command { std: "docker" "create" "-v" "/var/lib/crater-agent-workspace/builds/worker-5-tc2/target:/opt/rustwide/target:rw,Z" "-v" "/var/lib/crater-agent-workspace/builds/worker-5-tc2/source:/opt/rustwide/workdir:ro,Z" "-v" "/var/lib/crater-agent-workspace/cargo-home:/opt/rustwide/cargo-home:ro,Z" "-v" "/var/lib/crater-agent-workspace/rustup-home:/opt/rustwide/rustup-home:ro,Z" "-e" "SOURCE_DIR=/opt/rustwide/workdir" "-e" "CARGO_TARGET_DIR=/opt/rustwide/target" "-e" "CARGO_INCREMENTAL=0" "-e" "RUST_BACKTRACE=full" "-e" "RUSTFLAGS=--cap-lints=forbid" "-e" "RUSTDOCFLAGS=--cap-lints=forbid" "-e" "CARGO_HOME=/opt/rustwide/cargo-home" "-e" "RUSTUP_HOME=/opt/rustwide/rustup-home" "-w" "/opt/rustwide/workdir" "-m" "1610612736" "--user" "0:0" "--network" "none" "ghcr.io/rust-lang/crates-build-env/linux@sha256:d429b63d4308055ea97f60fb1d3dfca48854a00942f1bd2ad806beaf015945ec" "/opt/rustwide/cargo-home/bin/cargo" "+33835004928d3bf65db4d4712e1330766263b0bd" "build" "--frozen" "--message-format=json", kill_on_drop: false }`
[INFO] [stdout] b8600fff0a99154833593057224a9d31cf8170e7f94660435b4155df58b0fc26
[INFO] running `Command { std: "docker" "start" "-a" "b8600fff0a99154833593057224a9d31cf8170e7f94660435b4155df58b0fc26", kill_on_drop: false }`
[INFO] [stderr]    Compiling libc v0.2.183
[INFO] [stderr]    Compiling pin-project-lite v0.2.17
[INFO] [stderr]    Compiling tracing-core v0.1.36
[INFO] [stderr]    Compiling memchr v2.8.0
[INFO] [stderr]    Compiling getrandom v0.4.2
[INFO] [stderr]    Compiling syn v2.0.117
[INFO] [stderr]    Compiling anstream v0.6.21
[INFO] [stderr]    Compiling zerocopy v0.8.42
[INFO] [stderr]    Compiling thiserror v2.0.18
[INFO] [stderr]    Compiling either v1.15.0
[INFO] [stderr]    Compiling futures-task v0.3.32
[INFO] [stderr]    Compiling slab v0.4.12
[INFO] [stderr]    Compiling prometheus v0.13.4
[INFO] [stderr]    Compiling regex-syntax v0.8.10
[INFO] [stderr]    Compiling anyhow v1.0.102
[INFO] [stderr]    Compiling http-body-util v0.1.3
[INFO] [stderr]    Compiling futures-util v0.3.32
[INFO] [stderr]    Compiling rayon v1.11.0
[INFO] [stderr]    Compiling clap_builder v4.5.60
[INFO] [stderr]    Compiling serde_json v1.0.149
[INFO] [stderr]    Compiling tracing-log v0.2.0
[INFO] [stderr]    Compiling regex-automata v0.4.14
[INFO] [stderr]    Compiling parking_lot_core v0.9.12
[INFO] [stderr]    Compiling errno v0.3.14
[INFO] [stderr]    Compiling mio v1.1.1
[INFO] [stderr]    Compiling getrandom v0.2.17
[INFO] [stderr]    Compiling socket2 v0.6.3
[INFO] [stderr]    Compiling memmap2 v0.9.10
[INFO] [stderr]    Compiling signal-hook-registry v1.4.8
[INFO] [stderr]    Compiling uuid v1.22.0
[INFO] [stderr]    Compiling rand_core v0.6.4
[INFO] [stderr]    Compiling parking_lot v0.12.5
[INFO] [stderr]    Compiling matchers v0.2.0
[INFO] [stderr]    Compiling tracing-attributes v0.1.31
[INFO] [stderr]    Compiling serde_derive v1.0.228
[INFO] [stderr]    Compiling thiserror-impl v2.0.18
[INFO] [stderr]    Compiling clap_derive v4.5.55
[INFO] [stderr]    Compiling tokio-macros v2.6.1
[INFO] [stderr]    Compiling thiserror-impl v1.0.69
[INFO] [stderr]    Compiling tokio v1.50.0
[INFO] [stderr]    Compiling tracing v0.1.44
[INFO] [stderr]    Compiling ppv-lite86 v0.2.21
[INFO] [stderr]    Compiling axum-core v0.5.6
[INFO] [stderr]    Compiling tracing-subscriber v0.3.22
[INFO] [stderr]    Compiling thiserror v1.0.69
[INFO] [stderr]    Compiling rand_chacha v0.3.1
[INFO] [stderr]    Compiling rand v0.8.5
[INFO] [stderr]    Compiling clap v4.5.60
[INFO] [stderr]    Compiling serde v1.0.228
[INFO] [stderr]    Compiling chrono v0.4.44
[INFO] [stderr]    Compiling serde_urlencoded v0.7.1
[INFO] [stderr]    Compiling hyper v1.8.1
[INFO] [stderr]    Compiling tower v0.5.3
[INFO] [stderr]    Compiling shardlake-core v0.1.0 (/opt/rustwide/workdir/crates/shardlake-core)
[INFO] [stderr]    Compiling hyper-util v0.1.20
[INFO] [stderr]    Compiling axum v0.8.8
[INFO] [stderr]    Compiling shardlake-storage v0.1.0 (/opt/rustwide/workdir/crates/shardlake-storage)
[INFO] [stderr]    Compiling shardlake-manifest v0.1.0 (/opt/rustwide/workdir/crates/shardlake-manifest)
[INFO] [stderr]    Compiling shardlake-index v0.1.0 (/opt/rustwide/workdir/crates/shardlake-index)
[INFO] [stderr]    Compiling shardlake-bench v0.1.0 (/opt/rustwide/workdir/crates/shardlake-bench)
[INFO] [stderr]    Compiling shardlake-serve v0.1.0 (/opt/rustwide/workdir/crates/shardlake-serve)
[INFO] [stderr]    Compiling shardlake-cli v0.1.0 (/opt/rustwide/workdir/crates/shardlake-cli)
[INFO] [stderr]     Finished `dev` profile [unoptimized + debuginfo] target(s) in 57.42s
[INFO] running `Command { std: "docker" "inspect" "b8600fff0a99154833593057224a9d31cf8170e7f94660435b4155df58b0fc26", kill_on_drop: false }`
[INFO] running `Command { std: "docker" "rm" "-f" "b8600fff0a99154833593057224a9d31cf8170e7f94660435b4155df58b0fc26", kill_on_drop: false }`
[INFO] [stdout] b8600fff0a99154833593057224a9d31cf8170e7f94660435b4155df58b0fc26
[INFO] running `Command { std: "docker" "create" "-v" "/var/lib/crater-agent-workspace/builds/worker-5-tc2/target:/opt/rustwide/target:rw,Z" "-v" "/var/lib/crater-agent-workspace/builds/worker-5-tc2/source:/opt/rustwide/workdir:ro,Z" "-v" "/var/lib/crater-agent-workspace/cargo-home:/opt/rustwide/cargo-home:ro,Z" "-v" "/var/lib/crater-agent-workspace/rustup-home:/opt/rustwide/rustup-home:ro,Z" "-e" "SOURCE_DIR=/opt/rustwide/workdir" "-e" "CARGO_TARGET_DIR=/opt/rustwide/target" "-e" "CARGO_INCREMENTAL=0" "-e" "RUST_BACKTRACE=full" "-e" "RUSTFLAGS=--cap-lints=forbid" "-e" "RUSTDOCFLAGS=--cap-lints=forbid" "-e" "CARGO_HOME=/opt/rustwide/cargo-home" "-e" "RUSTUP_HOME=/opt/rustwide/rustup-home" "-w" "/opt/rustwide/workdir" "-m" "1610612736" "--user" "0:0" "--network" "none" "ghcr.io/rust-lang/crates-build-env/linux@sha256:d429b63d4308055ea97f60fb1d3dfca48854a00942f1bd2ad806beaf015945ec" "/opt/rustwide/cargo-home/bin/cargo" "+33835004928d3bf65db4d4712e1330766263b0bd" "test" "--frozen" "--no-run" "--message-format=json", kill_on_drop: false }`
[INFO] [stdout] b676bdacfe2f55355fce2ecf3b06053332cda1e4ef992e3e192f324c23aa365b
[INFO] running `Command { std: "docker" "start" "-a" "b676bdacfe2f55355fce2ecf3b06053332cda1e4ef992e3e192f324c23aa365b", kill_on_drop: false }`
[INFO] [stderr]    Compiling tempfile v3.26.0
[INFO] [stderr]    Compiling shardlake-core v0.1.0 (/opt/rustwide/workdir/crates/shardlake-core)
[INFO] [stderr]    Compiling shardlake-cli v0.1.0 (/opt/rustwide/workdir/crates/shardlake-cli)
[INFO] [stderr]    Compiling shardlake-index v0.1.0 (/opt/rustwide/workdir/crates/shardlake-index)
[INFO] [stderr]    Compiling shardlake-serve v0.1.0 (/opt/rustwide/workdir/crates/shardlake-serve)
[INFO] [stderr]    Compiling shardlake-manifest v0.1.0 (/opt/rustwide/workdir/crates/shardlake-manifest)
[INFO] [stderr]    Compiling shardlake-storage v0.1.0 (/opt/rustwide/workdir/crates/shardlake-storage)
[INFO] [stderr]    Compiling shardlake-bench v0.1.0 (/opt/rustwide/workdir/crates/shardlake-bench)
[INFO] [stderr]     Finished `test` profile [unoptimized + debuginfo] target(s) in 13.99s
[INFO] running `Command { std: "docker" "inspect" "b676bdacfe2f55355fce2ecf3b06053332cda1e4ef992e3e192f324c23aa365b", kill_on_drop: false }`
[INFO] running `Command { std: "docker" "rm" "-f" "b676bdacfe2f55355fce2ecf3b06053332cda1e4ef992e3e192f324c23aa365b", kill_on_drop: false }`
[INFO] [stdout] b676bdacfe2f55355fce2ecf3b06053332cda1e4ef992e3e192f324c23aa365b
[INFO] running `Command { std: "docker" "create" "-v" "/var/lib/crater-agent-workspace/builds/worker-5-tc2/target:/opt/rustwide/target:rw,Z" "-v" "/var/lib/crater-agent-workspace/builds/worker-5-tc2/source:/opt/rustwide/workdir:ro,Z" "-v" "/var/lib/crater-agent-workspace/cargo-home:/opt/rustwide/cargo-home:ro,Z" "-v" "/var/lib/crater-agent-workspace/rustup-home:/opt/rustwide/rustup-home:ro,Z" "-e" "SOURCE_DIR=/opt/rustwide/workdir" "-e" "CARGO_TARGET_DIR=/opt/rustwide/target" "-e" "CARGO_INCREMENTAL=0" "-e" "RUST_BACKTRACE=full" "-e" "RUSTFLAGS=--cap-lints=forbid" "-e" "RUSTDOCFLAGS=--cap-lints=forbid" "-e" "CARGO_HOME=/opt/rustwide/cargo-home" "-e" "RUSTUP_HOME=/opt/rustwide/rustup-home" "-w" "/opt/rustwide/workdir" "-m" "1610612736" "--user" "0:0" "--network" "none" "ghcr.io/rust-lang/crates-build-env/linux@sha256:d429b63d4308055ea97f60fb1d3dfca48854a00942f1bd2ad806beaf015945ec" "/opt/rustwide/cargo-home/bin/cargo" "+33835004928d3bf65db4d4712e1330766263b0bd" "test" "--frozen", kill_on_drop: false }`
[INFO] [stdout] a5b9d8b5cd89097028aa3ff54be05b0279b75fb59a78c6749b676c6f613f47c5
[INFO] running `Command { std: "docker" "start" "-a" "a5b9d8b5cd89097028aa3ff54be05b0279b75fb59a78c6749b676c6f613f47c5", kill_on_drop: false }`
[INFO] [stderr]     Finished `test` profile [unoptimized + debuginfo] target(s) in 0.27s
[INFO] [stderr]      Running unittests src/lib.rs (/opt/rustwide/target/debug/deps/shardlake_bench-ea1d9d3e250e6b3f)
[INFO] [stdout] 
[INFO] [stdout] running 50 tests
[INFO] [stdout] test tests::compare_ann_report_empty_entries_round_trips ... ok
[INFO] [stdout] test tests::ann_family_report_flattens_eval_fields_in_json ... ok
[INFO] [stdout] test generate::tests::metadata_is_none ... ok
[INFO] [stdout] test generate::tests::ids_are_sequential_from_one ... ok
[INFO] [stdout] test tests::compare_ann_report_serialises_all_entries ... ok
[INFO] [stdout] test generate::tests::reproducible_with_same_seed ... ok
[INFO] [stdout] test generate::tests::different_seeds_produce_different_data ... ok
[INFO] [stdout] test tests::cost_metrics_pq_index_has_compression_ratio_gt_one ... ok
[INFO] [stdout] test tests::cost_metrics_zero_vector_index_has_zero_memory ... ok
[INFO] [stdout] test tests::cost_metrics_saturate_on_overflow ... ok
[INFO] [stdout] test tests::cost_metrics_uncompressed_index_has_ratio_one ... ok
[INFO] [stdout] test generate::tests::generates_correct_shape ... ok
[INFO] [stdout] test tests::cost_metrics_disk_footprint_matches_sum_of_artifacts ... ok
[INFO] [stdout] test tests::evaluate_partitioning_keeps_recall_for_legacy_manifests ... ok
[INFO] [stdout] test tests::cost_metrics_use_local_paths_without_reading_artifacts ... ok
[INFO] [stdout] test generate::tests::vectors_are_close_to_centroids ... ok
[INFO] [stdout] test tests::evaluate_partitioning_propagates_search_errors ... ok
[INFO] [stdout] test tests::evaluate_partitioning_reports_routing_and_recall_with_queries ... ok
[INFO] [stdout] test tests::cost_metrics_disk_footprint_ignores_other_indexes_in_storage ... ok
[INFO] [stdout] test tests::run_benchmark_empty_queries_return_zero_metrics ... ok
[INFO] [stdout] test tests::evaluate_partitioning_reports_shard_size_distribution ... ok
[INFO] [stdout] test tests::run_benchmark_reports_correct_field_counts ... ok
[INFO] [stdout] test tests::run_benchmark_throughput_is_positive ... ok
[INFO] [stdout] test tests::run_workload_benchmark_empty_queries_returns_zero_metrics_for_all_modes ... ok
[INFO] [stdout] test tests::run_eval_hybrid_returns_error_for_mismatched_query_text_count ... ok
[INFO] [stdout] test tests::run_eval_hybrid_returns_error_for_empty_queries ... ok
[INFO] [stdout] test tests::evaluate_partitioning_std_dev_zero_for_equal_shards ... ok
[INFO] [stdout] test tests::run_workload_benchmark_cold_mode_starts_with_a_fresh_cache_per_query ... ok
[INFO] [stdout] test tests::run_eval_ann_with_ground_truth_matches_direct_eval ... ok
[INFO] [stdout] test tests::run_eval_ann_with_ground_truth_rejects_mismatched_lengths ... ok
[INFO] [stdout] test tests::run_workload_benchmark_mixed_mode_runs_without_errors ... ok
[INFO] [stdout] test tests::test_nearest_rank_percentile_uses_explicit_rank ... ok
[INFO] [stdout] test tests::test_precision_partial ... ok
[INFO] [stdout] test tests::test_precision_perfect ... ok
[INFO] [stdout] test tests::test_recall_perfect ... ok
[INFO] [stdout] test tests::test_recall_zero ... ok
[INFO] [stdout] test tests::workload_mode_display_labels_are_correct ... ok
[INFO] [stdout] test tests::workload_mode_serialises_to_snake_case ... ok
[INFO] [stdout] test tests::run_eval_hybrid_vector_recall_perfect_with_full_probe ... ok
[INFO] [stdout] test tests::run_workload_benchmark_report_field_workload_matches_requested_mode ... ok
[INFO] [stdout] test tests::run_benchmark_recall_is_perfect_with_full_probe ... ok
[INFO] [stdout] test tests::run_workload_benchmark_cold_mode_counts_intra_query_hits_for_legacy_manifests ... ok
[INFO] [stdout] test tests::run_eval_hybrid_reports_correct_field_counts ... ok
[INFO] [stdout] test tests::test_nearest_rank_percentile_handles_single_value ... ok
[INFO] [stdout] test generate::tests::rejects_zero_num_clusters - should panic ... ok
[INFO] [stdout] test generate::tests::rejects_zero_dims - should panic ... ok
[INFO] [stdout] test tests::run_workload_benchmark_warm_mode_yields_high_cache_hit_rate ... ok
[INFO] [stdout] test generate::tests::rejects_non_finite_cluster_spread - should panic ... ok
[INFO] [stdout] test generate::tests::rejects_zero_num_vectors - should panic ... ok
[INFO] [stdout] test generate::tests::rejects_negative_cluster_spread - should panic ... ok
[INFO] [stdout] 
[INFO] [stdout] test result: ok. 50 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.16s
[INFO] [stdout] 
[INFO] [stderr]      Running unittests src/main.rs (/opt/rustwide/target/debug/deps/shardlake-04820fb5cb8fa9a0)
[INFO] [stdout] 
[INFO] [stdout] running 48 tests
[INFO] [stdout] test commands::build_index::tests::validate_kmeans_sample_size_rejects_zero ... ok
[INFO] [stdout] test commands::build_index::tests::run_rejects_zero_kmeans_sample_size_before_loading_dataset ... ok
[INFO] [stdout] test commands::build_index_worker::tests::plan_mode_rejects_missing_dataset ... ok
[INFO] [stdout] test commands::build_index::tests::run_rejects_num_workers_without_parallel_before_loading_dataset ... ok
[INFO] [stdout] test commands::build_index::tests::validate_num_shards_rejects_zero ... ok
[INFO] [stdout] test commands::build_index_worker::tests::execute_mode_rejects_missing_plan ... ok
[INFO] [stdout] test commands::build_index::tests::run_rejects_zero_num_shards_before_loading_dataset ... ok
[INFO] [stdout] test commands::build_index_worker::tests::merge_mode_rejects_missing_index_version_arg ... ok
[INFO] [stdout] test commands::compare_ann::tests::canonical_ann_family_distinguishes_ivf_flat_and_pq ... ok
[INFO] [stdout] test commands::compare_ann::tests::canonical_ann_family_normalizes_hyphenated_names ... ok
[INFO] [stdout] test commands::build_index::tests::run_preserves_manifest_parse_errors ... ok
[INFO] [stdout] test commands::build_index_worker::tests::merge_mode_rejects_missing_plan ... ok
[INFO] [stdout] test commands::compare_ann::tests::comparable_aliases_pass_validation ... ok
[INFO] [stdout] test commands::build_index::tests::run_defaults_embedding_version_from_dataset_manifest ... ok
[INFO] [stdout] test commands::build_index_worker::tests::execute_mode_builds_shards ... ok
[INFO] [stdout] test commands::build_index_worker::tests::execute_mode_rejects_out_of_range_worker_id ... ok
[INFO] [stdout] test commands::build_index::tests::parallel_build_with_single_worker_matches_sequential ... ok
[INFO] [stdout] test commands::evaluate_partitioning::tests::run_validates_nprobe_zero ... ok
[INFO] [stdout] test commands::build_index::tests::parallel_build_produces_valid_manifest ... ok
[INFO] [stdout] test commands::compare_ann::tests::mismatched_metric_is_rejected ... ok
[INFO] [stdout] test commands::evaluate_partitioning::tests::run_errors_on_missing_index ... ok
[INFO] [stdout] test commands::generate::tests::validate_generate_args_rejects_negative_cluster_spread ... ok
[INFO] [stdout] test commands::generate::tests::validate_generate_args_rejects_zero_num_vectors ... ok
[INFO] [stdout] test commands::ingest::tests::dataset_manifest_dims_rejects_overflow ... ok
[INFO] [stdout] test commands::evaluate_partitioning::tests::run_validates_k_zero ... ok
[INFO] [stdout] test commands::build_index_worker::tests::plan_mode_writes_worker_plan_to_storage ... ok
[INFO] [stdout] test commands::ingest::tests::parse_records_rejects_dimension_mismatch ... ok
[INFO] [stdout] test commands::ingest::tests::run_rejects_dimension_mismatch ... ok
[INFO] [stdout] test commands::ingest::tests::run_rejects_empty_vector ... ok
[INFO] [stdout] test commands::generate::tests::run_writes_expected_artifacts ... ok
[INFO] [stdout] test commands::serve::tests::serve_args_accept_enable_debug_routes_flag ... ok
[INFO] [stdout] test commands::serve::tests::serve_args_reject_zero_shard_cache_capacity ... ok
[INFO] [stdout] test commands::serve::tests::serve_args_debug_routes_disabled_by_default ... ok
[INFO] [stdout] test commands::generate::tests::run_writes_reproducible_vectors_jsonl ... ok
[INFO] [stdout] test commands::build_index_worker::tests::merge_mode_writes_manifest ... ok
[INFO] [stdout] test commands::validate_manifest::tests::continues_after_index_load_error_when_dataset_requested_too ... ok
[INFO] [stdout] test commands::evaluate_partitioning::tests::run_succeeds_via_alias ... ok
[INFO] [stdout] test commands::validate_manifest::tests::validates_dataset_manifest_missing_artifacts ... ok
[INFO] [stdout] test commands::validate_manifest::tests::validates_dataset_manifest_success ... ok
[INFO] [stdout] test commands::evaluate_partitioning::tests::run_succeeds_against_built_index_by_version ... ok
[INFO] [stdout] test commands::build_index_worker::tests::merge_mode_preserves_plan_build_metadata ... ok
[INFO] [stdout] test commands::validate_manifest::tests::validates_index_manifest_missing_returns_error ... ok
[INFO] [stdout] test commands::validate_manifest::tests::run_errors_when_no_versions_provided ... ok
[INFO] [stdout] test commands::validate_manifest::tests::validates_invalid_dataset_manifest_as_failure_report ... ok
[INFO] [stdout] test commands::validate_manifest::tests::validates_both_manifests_together ... ok
[INFO] [stdout] test commands::compare_ann::tests::mismatched_dataset_is_rejected ... ok
[INFO] [stdout] test commands::ingest::tests::parse_records_rejects_empty_vector ... ok
[INFO] [stdout] test commands::serve::tests::load_bm25_index_returns_index_when_manifest_includes_lexical_artifact ... ok
[INFO] [stdout] 
[INFO] [stdout] test result: ok. 48 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.07s
[INFO] [stdout] 
[INFO] [stderr]      Running unittests src/lib.rs (/opt/rustwide/target/debug/deps/shardlake_core-9c2a063b23e7ed40)
[INFO] [stdout] 
[INFO] [stdout] running 6 tests
[INFO] [stdout] test config::tests::prefetch_policy_rejects_zero_threshold_when_enabled ... ok
[INFO] [stdout] test config::tests::query_config_rejects_zero_top_k ... ok
[INFO] [stdout] test config::tests::query_config_valid_with_all_fields ... ok
[INFO] [stdout] test config::tests::system_config_rejects_zero_shard_cache_capacity ... ok
[INFO] [stdout] test config::tests::query_config_rejects_invalid_fan_out ... ok
[INFO] [stderr]      Running unittests src/lib.rs (/opt/rustwide/target/debug/deps/shardlake_index-85a29acf37f87409)
[INFO] [stdout] test config::tests::query_config_rejects_zero_rerank_limit ... ok
[INFO] [stdout] 
[INFO] [stdout] test result: ok. 6 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.05s
[INFO] [stdout] 
[INFO] [stdout] 
[INFO] [stdout] running 203 tests
[INFO] [stdout] test bm25::tests::bm25_params_default_values ... ok
[INFO] [stdout] test bm25::tests::build_empty_documents_gives_empty_index ... ok
[INFO] [stdout] test bm25::tests::build_coalesces_duplicate_vector_ids_into_one_document ... ok
[INFO] [stdout] test bm25::tests::doc_with_more_matching_terms_scores_higher ... ok
[INFO] [stdout] test bm25::tests::empty_index_round_trip ... ok
[INFO] [stdout] test bm25::tests::from_bytes_rejects_bad_magic ... ok
[INFO] [stdout] test bm25::tests::build_records_correct_doc_count_and_term_count ... ok
[INFO] [stdout] test bm25::tests::from_bytes_rejects_bad_version ... ok
[INFO] [stdout] test bm25::tests::from_bytes_rejects_unknown_posting_doc_ids ... ok
[INFO] [stdout] test bm25::tests::search_is_case_insensitive ... ok
[INFO] [stdout] test bm25::tests::search_with_k_zero_returns_empty ... ok
[INFO] [stdout] test bm25::tests::search_scores_are_sorted_ascending_lower_is_more_relevant ... ok
[INFO] [stdout] test bm25::tests::search_returns_at_most_k_results ... ok
[INFO] [stdout] test bm25::tests::tokenize_empty_string ... ok
[INFO] [stdout] test bm25::tests::search_returns_empty_for_unknown_term ... ok
[INFO] [stdout] test builder::tests::build_rejects_record_dimension_mismatch ... ok
[INFO] [stdout] test bm25::tests::search_scores_are_negative ... ok
[INFO] [stdout] test bm25::tests::custom_params_affect_scores ... ok
[INFO] [stdout] test bm25::tests::save_and_load_via_object_store ... ok
[INFO] [stdout] test bm25::tests::tokenize_only_punctuation ... ok
[INFO] [stdout] test bm25::tests::term_present_in_all_docs_has_low_idf ... ok
[INFO] [stdout] test builder::tests::build_rejects_zero_kmeans_sample_size ... ok
[INFO] [stdout] test builder::tests::build_rejects_zero_num_shards ... ok
[INFO] [stdout] test bm25::tests::to_bytes_is_deterministic_across_rebuilds ... ok
[INFO] [stdout] test bm25::tests::to_bytes_from_bytes_round_trip ... ok
[INFO] [stdout] test bm25::tests::tokenize_lowercases ... ok
[INFO] [stdout] test builder::tests::build_rejects_pq_for_non_euclidean_metric ... ok
[INFO] [stdout] test bm25::tests::tokenize_splits_on_non_alphanumeric ... ok
[INFO] [stdout] test builder::tests::build_without_recall_sample_size_omits_recall_estimate ... ok
[INFO] [stdout] test cache::tests::lru_cache_capacity_one ... ok
[INFO] [stdout] test cache::tests::lru_cache_evicts_lru_when_full ... ok
[INFO] [stdout] test cache::tests::lru_cache_get_miss_on_empty ... ok
[INFO] [stdout] test cache::tests::lru_cache_insert_and_get ... ok
[INFO] [stdout] test builder::tests::build_with_sample_size_assigns_all_vectors ... ok
[INFO] [stdout] test cache::tests::lru_cache_update_promotes_existing_to_mru ... ok
[INFO] [stdout] test cache::tests::shard_cache_evicts_lru_entry ... ok
[INFO] [stdout] test cache::tests::shard_cache_hit_returns_cached_value_without_reload ... ok
[INFO] [stdout] test builder::tests::build_with_recall_sample_size_populates_recall_estimate ... ok
[INFO] [stdout] test builder::tests::recall_estimate_works_with_pq_compression ... ok
[INFO] [stdout] test cache::tests::shard_cache_load_error_is_forwarded ... ok
[INFO] [stdout] test cache::tests::shard_cache_reuses_existing_value_after_raced_load ... ok
[INFO] [stdout] test builder::tests::build_omits_sample_size_when_sampling_is_not_needed ... ok
[INFO] [stdout] test exact::tests::test_distance_cosine_identical_direction ... ok
[INFO] [stdout] test cache::tests::shard_cache_miss_invokes_load ... ok
[INFO] [stdout] test exact::tests::test_distance_cosine_zero_vector_returns_one ... ok
[INFO] [stdout] test exact::tests::test_distance_inner_product_negated_dot ... ok
[INFO] [stdout] test exact::tests::test_distance_euclidean ... ok
[INFO] [stdout] test exact::tests::test_exact_search_cosine ... ok
[INFO] [stdout] test exact::tests::test_exact_search_cosine_vs_euclidean_differ ... ok
[INFO] [stdout] test exact::tests::test_exact_search_inner_product ... ok
[INFO] [stdout] test exact::tests::test_exact_search_euclidean ... ok
[INFO] [stdout] test exact::tests::test_precision_at_k_partial ... ok
[INFO] [stdout] test exact::tests::test_distance_cosine_opposite_direction ... ok
[INFO] [stdout] test exact::tests::test_merge_top_k_deduplicates_ids_by_best_score ... ok
[INFO] [stdout] test exact::tests::test_recall_at_k ... ok
[INFO] [stdout] test ivf::tests::from_bytes_rejects_wrong_magic ... ok
[INFO] [stdout] test cache::tests::shard_cache_is_empty_after_construction ... ok
[INFO] [stdout] test exact::tests::test_distance_cosine_orthogonal ... ok
[INFO] [stdout] test ivf::tests::assign_routes_to_correct_cluster ... ok
[INFO] [stdout] test ivf::tests::from_bytes_rejects_payload_length_mismatch ... ok
[INFO] [stdout] test ivf::tests::from_bytes_rejects_zero_clusters ... ok
[INFO] [stdout] test ivf::tests::from_bytes_rejects_zero_dims ... ok
[INFO] [stdout] test ivf::tests::roundtrip_serialisation ... ok
[INFO] [stdout] test exact::tests::test_precision_at_k_perfect ... ok
[INFO] [stdout] test exact::tests::test_precision_at_k_empty_retrieved ... ok
[INFO] [stdout] test ivf::tests::top_probes_clamped_by_num_clusters ... ok
[INFO] [stdout] test ivf::tests::top_probes_returns_nprobe_clusters_sorted_by_distance ... ok
[INFO] [stdout] test builder::tests::build_with_sample_size_is_deterministic ... ok
[INFO] [stdout] test kmeans::tests::test_kmeans_basic ... ok
[INFO] [stdout] test metrics::tests::test_hit_rate_all_hits ... ok
[INFO] [stdout] test kmeans::tests::test_nearest_centroid ... ok
[INFO] [stdout] test metrics::tests::test_default_creates_zeroed_metrics ... ok
[INFO] [stdout] test metrics::tests::test_initial_snapshot_is_zeroed ... ok
[INFO] [stdout] test metrics::tests::test_mean_load_latency_no_loads ... ok
[INFO] [stdout] test metrics::tests::test_mean_load_latency_single_load ... ok
[INFO] [stdout] test metrics::tests::test_hit_rate_all_misses ... ok
[INFO] [stdout] test metrics::tests::test_hit_rate_mixed ... ok
[INFO] [stdout] test ivf::tests::train_clamps_clusters_to_vector_count ... ok
[INFO] [stdout] test metrics::tests::test_record_load_attempt_and_retained_bytes_accumulate ... ok
[INFO] [stdout] test ivf::tests::train_produces_correct_num_clusters ... ok
[INFO] [stdout] test kmeans::tests::test_kmeans_is_deterministic ... ok
[INFO] [stdout] test pipeline::tests::centroid_router_deduplicates_shards ... ok
[INFO] [stdout] test pipeline::tests::identity_embedder_returns_query_unchanged ... ok
[INFO] [stdout] test pipeline::tests::exact_candidate_search_returns_nearest ... ok
[INFO] [stdout] test pipeline::tests::noop_reranker_returns_results_unchanged ... ok
[INFO] [stdout] test pipeline::tests::search_alias_matches_run ... ok
[INFO] [stdout] test plugin::tests::ann_family_as_str_round_trips ... ok
[INFO] [stdout] test builder::tests::recall_estimate_is_deterministic ... ok
[INFO] [stdout] test plugin::tests::ann_family_unknown_name_returns_error ... ok
[INFO] [stdout] test pipeline::tests::centroid_router_preserves_probe_order_when_deduplicating ... ok
[INFO] [stdout] test kmeans::tests::test_kmeans_on_sample_finds_correct_clusters ... ok
[INFO] [stdout] test metrics::tests::test_hit_rate_no_requests ... ok
[INFO] [stdout] test plugin::tests::hnsw_plugin_family_name ... ok
[INFO] [stdout] test plugin::tests::hnsw_plugin_validate_accepts_all_metrics ... ok
[INFO] [stdout] test plugin::tests::hnsw_plugin_validate_rejects_zero_ef_search ... ok
[INFO] [stdout] test plugin::tests::diskann_plugin_beam_width_accessor ... ok
[INFO] [stdout] test plugin::tests::diskann_plugin_candidate_stage_is_sendable ... ok
[INFO] [stdout] test plugin::tests::diskann_plugin_family_name ... ok
[INFO] [stdout] test plugin::tests::diskann_plugin_validate_accepts_euclidean ... ok
[INFO] [stdout] test plugin::tests::diskann_plugin_validate_rejects_cosine ... ok
[INFO] [stdout] test plugin::tests::diskann_plugin_validate_rejects_inner_product ... ok
[INFO] [stdout] test pipeline::tests::run_rejects_invalid_query_config ... ok
[INFO] [stdout] test plugin::tests::hnsw_plugin_validate_rejects_ef_construction_less_than_m ... ok
[INFO] [stdout] test plugin::tests::hnsw_config_default_values_are_valid ... ok
[INFO] [stdout] test plugin::tests::hnsw_plugin_validate_rejects_zero_m ... ok
[INFO] [stdout] test plugin::tests::ivf_flat_plugin_family_name ... ok
[INFO] [stdout] test plugin::tests::ivf_pq_plugin_validate_rejects_cosine ... ok
[INFO] [stdout] test plugin::tests::hnsw_plugin_candidate_stage_is_sendable ... ok
[INFO] [stdout] test plugin::tests::ivf_flat_plugin_candidate_stage_is_sendable ... ok
[INFO] [stdout] test plugin::tests::ivf_pq_plugin_validate_rejects_inner_product ... ok
[INFO] [stdout] test plugin::tests::ivf_pq_plugin_candidate_stage_is_sendable ... ok
[INFO] [stdout] test plugin::tests::ivf_pq_plugin_family_name ... ok
[INFO] [stdout] test plugin::tests::ivf_pq_plugin_validate_accepts_euclidean ... ok
[INFO] [stdout] test plugin::tests::plugin_dispatch_based_on_family_name ... ok
[INFO] [stdout] test plugin::tests::registry_exists_returns_false_for_unknown_name ... ok
[INFO] [stdout] test plugin::tests::registry_exists_returns_true_for_known_names ... ok
[INFO] [stdout] test plugin::tests::registry_families_contains_all_builtins ... ok
[INFO] [stdout] test plugin::tests::registry_get_flat_rejects_ivf_pq_with_helpful_message ... ok
[INFO] [stdout] test plugin::tests::registry_get_flat_returns_diskann_plugin ... ok
[INFO] [stdout] test plugin::tests::registry_get_flat_returns_hnsw_plugin ... ok
[INFO] [stdout] test pq::tests::from_bytes_rejects_inconsistent_sub_dims ... ok
[INFO] [stdout] test plugin::tests::registry_get_flat_returns_ivf_flat_plugin ... ok
[INFO] [stdout] test pq::tests::encode_rejects_dimension_mismatch ... ok
[INFO] [stdout] test pq::tests::train_clamps_codebook_size_to_available_vectors ... ok
[INFO] [stdout] test plugin::tests::registry_get_flat_rejects_unknown_family ... ok
[INFO] [stdout] test pq::tests::from_bytes_rejects_wrong_magic ... ok
[INFO] [stdout] test pq::tests::distance_table_rejects_dimension_mismatch ... ok
[INFO] [stdout] test pq::tests::adc_distance_is_non_negative ... ok
[INFO] [stdout] test pq::tests::train_rejects_invalid_params ... ok
[INFO] [stdout] test ranking::tests::normalize_all_nan_scores_returns_zeroes ... ok
[INFO] [stdout] test ranking::tests::normalize_empty_list ... ok
[INFO] [stdout] test ranking::tests::normalize_distinct_scores ... ok
[INFO] [stdout] test ranking::tests::normalize_negative_scores_bm25_convention ... ok
[INFO] [stdout] test ranking::tests::normalize_nan_scores_with_finite_scores_penalizes_nan_entries ... ok
[INFO] [stdout] test pq::tests::train_rejects_inconsistent_vector_dimensions ... ok
[INFO] [stdout] test pq::tests::encoding_is_deterministic ... ok
[INFO] [stdout] test pipeline::tests::pipeline_run_with_reranker_emits_rerank_span ... ok
[INFO] [stdout] test pq::tests::codebook_roundtrip ... ok
[INFO] [stdout] test pq::tests::train_produces_correct_shape ... ok
[INFO] [stdout] test pq::tests::from_bytes_rejects_invalid_params ... ok
[INFO] [stdout] test plugin::tests::ivf_flat_plugin_validate_accepts_all_metrics ... ok
[INFO] [stdout] test pipeline::tests::rerank_oversample_is_clamped_to_one ... ok
[INFO] [stdout] test pq::tests::encode_returns_valid_codes ... ok
[INFO] [stdout] test ranking::tests::normalize_all_equal_scores ... ok
[INFO] [stdout] test ranking::tests::normalize_single_element ... ok
[INFO] [stdout] test ranking::tests::policy_accepts_pure_bm25 ... ok
[INFO] [stdout] test ranking::tests::policy_accepts_pure_vector ... ok
[INFO] [stdout] test ranking::tests::policy_default_is_valid ... ok
[INFO] [stdout] test ranking::tests::policy_rejects_both_zero_weights ... ok
[INFO] [stdout] test ranking::tests::policy_rejects_nan_bm25_weight ... ok
[INFO] [stdout] test ranking::tests::policy_rejects_nan_vector_weight ... ok
[INFO] [stdout] test ranking::tests::policy_rejects_negative_bm25_weight ... ok
[INFO] [stdout] test ranking::tests::policy_rejects_negative_vector_weight ... ok
[INFO] [stdout] test ranking::tests::rank_hybrid_k_zero_returns_empty ... ok
[INFO] [stdout] test ranking::tests::rank_hybrid_both_empty_returns_empty ... ok
[INFO] [stdout] test ranking::tests::rank_hybrid_k_larger_than_candidates ... ok
[INFO] [stdout] test ranking::tests::rank_hybrid_metadata_from_vector_takes_precedence ... ok
[INFO] [stdout] test ranking::tests::rank_hybrid_non_unit_weights_still_return_zero_one_scores ... ok
[INFO] [stdout] test ranking::tests::rank_hybrid_pure_vector_ignores_bm25 ... ok
[INFO] [stdout] test ranking::tests::rank_hybrid_pure_bm25_ignores_vector ... ok
[INFO] [stdout] test ranking::tests::rank_hybrid_truncates_to_k ... ok
[INFO] [stdout] test ranking::tests::rank_hybrid_preserves_bm25_only_metadata ... ok
[INFO] [stdout] test searcher::tests::load_pq_shard_records_load_attempt_on_storage_error ... ok
[INFO] [stdout] test ranking::tests::rank_hybrid_scores_are_in_zero_one_range ... ok
[INFO] [stdout] test ranking::tests::rank_hybrid_stable_tie_breaking_by_id ... ok
[INFO] [stdout] test ranking::tests::rank_hybrid_blends_both_signals ... ok
[INFO] [stdout] test ranking::tests::rank_hybrid_missing_bm25_score_penalized ... ok
[INFO] [stdout] test ranking::tests::rank_hybrid_missing_vector_score_penalized ... ok
[INFO] [stdout] test shard::tests::pq_shard_rejects_code_out_of_range ... ok
[INFO] [stdout] test shard::tests::pq_shard_rejects_version_1_bytes ... ok
[INFO] [stdout] test searcher::tests::load_shard_records_load_attempt_on_storage_error ... ok
[INFO] [stdout] test shard::tests::encoded_len_matches_serialized_size_for_raw_shards ... ok
[INFO] [stdout] test searcher::tests::search_rejects_query_dimension_mismatch ... ok
[INFO] [stdout] test shard::tests::encoded_len_matches_serialized_size_for_pq_shards ... ok
[INFO] [stdout] test searcher::tests::candidate_shards_cap_limits_probed_shards ... ok
[INFO] [stdout] test searcher::tests::max_vectors_per_shard_limits_candidates ... ok
[INFO] [stdout] test searcher::tests::from_config_rejects_invalid_prefetch_policy ... ok
[INFO] [stdout] test searcher::tests::searcher_uses_mmap_for_local_shards_when_path_is_available ... ok
[INFO] [stdout] test worker::tests::plan_workers_rejects_dimension_mismatch ... ok
[INFO] [stdout] test worker::tests::plan_workers_clamps_num_workers_to_shard_count ... ok
[INFO] [stdout] test worker::tests::plan_workers_records_build_metadata ... ok
[INFO] [stdout] test worker::tests::plan_workers_produces_correct_structure ... ok
[INFO] [stdout] test worker::tests::plan_workers_is_reproducible ... ok
[INFO] [stdout] test shard::tests::pq_shard_roundtrip ... ok
[INFO] [stdout] test searcher::tests::search_with_plan_captures_routing_details ... ok
[INFO] [stdout] test worker::tests::plan_workers_rejects_empty_records ... ok
[INFO] [stdout] test worker::tests::plan_workers_rejects_zero_num_shards ... ok
[INFO] [stdout] test worker::tests::plan_workers_rejects_zero_num_workers ... ok
[INFO] [stdout] test searcher::tests::search_with_plan_rejects_dimension_mismatch ... ok
[INFO] [stdout] test worker::tests::plan_workers_writes_coarse_quantizer_to_storage ... ok
[INFO] [stdout] test searcher::tests::searcher_falls_back_to_get_without_local_path ... ok
[INFO] [stdout] test shard::tests::roundtrip ... ok
[INFO] [stdout] test worker::tests::worker_execute_rejects_invalid_assignment_and_record_dims ... ok
[INFO] [stdout] test searcher::tests::search_with_plan_matches_search_results ... ok
[INFO] [stdout] test worker::tests::worker_execute_rejects_malformed_plan_inputs ... ok
[INFO] [stdout] test searcher::tests::prefetch_warms_hot_evicted_shards_on_runtime_search_path ... ok
[INFO] [stdout] test worker::tests::worker_execute_writes_output_metadata_to_storage ... ok
[INFO] [stdout] test worker::tests::worker_execute_builds_shards_and_writes_output ... ok
[INFO] [stdout] test worker::tests::worker_execute_output_covers_all_assigned_shards ... ok
[INFO] [stdout] test worker::tests::worker_execute_is_reproducible ... ok
[INFO] [stdout] test worker::tests::worker_plan_roundtrip_serialisation ... ok
[INFO] [stdout] test pipeline::tests::pipeline_run_emits_shard_load_and_ann_search_spans ... FAILED
[INFO] [stdout] test pq::tests::adc_self_distance_is_near_zero ... ok
[INFO] [stdout] 
[INFO] [stdout] failures:
[INFO] [stdout] 
[INFO] [stdout] ---- pipeline::tests::pipeline_run_emits_shard_load_and_ann_search_spans stdout ----
[INFO] [stdout] 
[INFO] [stdout] thread 'pipeline::tests::pipeline_run_emits_shard_load_and_ann_search_spans' (279) panicked at crates/shardlake-index/src/pipeline.rs:1238:9:
[INFO] [stdout] expected ann_search span; got: ["shard_load"]
[INFO] [stdout] stack backtrace:
[INFO] [stdout]    0:     0x65090de8444a - std[ec419fa8110287d5]::backtrace_rs::backtrace::libunwind::trace
[INFO] [stdout]                                at /rustc/33835004928d3bf65db4d4712e1330766263b0bd/library/std/src/../../backtrace/src/backtrace/libunwind.rs:117:9
[INFO] [stdout]    1:     0x65090de8444a - std[ec419fa8110287d5]::backtrace_rs::backtrace::trace_unsynchronized::<std[ec419fa8110287d5]::sys::backtrace::_print_fmt::{closure#1}>
[INFO] [stdout]                                at /rustc/33835004928d3bf65db4d4712e1330766263b0bd/library/std/src/../../backtrace/src/backtrace/mod.rs:66:14
[INFO] [stdout]    2:     0x65090de8444a - std[ec419fa8110287d5]::sys::backtrace::_print_fmt
[INFO] [stdout]                                at /rustc/33835004928d3bf65db4d4712e1330766263b0bd/library/std/src/sys/backtrace.rs:74:9
[INFO] [stdout]    3:     0x65090de8444a - <<std[ec419fa8110287d5]::sys::backtrace::BacktraceLock>::print::DisplayBacktrace as core[35159d6ffb30e017]::fmt::Display>::fmt
[INFO] [stdout]                                at /rustc/33835004928d3bf65db4d4712e1330766263b0bd/library/std/src/sys/backtrace.rs:44:26
[INFO] [stdout]    4:     0x65090de9b6aa - <core[35159d6ffb30e017]::fmt::rt::Argument>::fmt
[INFO] [stdout]                                at /rustc/33835004928d3bf65db4d4712e1330766263b0bd/library/core/src/fmt/rt.rs:152:76
[INFO] [stdout]    5:     0x65090de9b6aa - core[35159d6ffb30e017]::fmt::write
[INFO] [stdout]    6:     0x65090de89042 - std[ec419fa8110287d5]::io::default_write_fmt::<alloc[a8a7aba520129d98]::vec::Vec<u8>>
[INFO] [stdout]                                at /rustc/33835004928d3bf65db4d4712e1330766263b0bd/library/std/src/io/mod.rs:621:11
[INFO] [stdout]    7:     0x65090de89042 - <alloc[a8a7aba520129d98]::vec::Vec<u8> as std[ec419fa8110287d5]::io::Write>::write_fmt
[INFO] [stdout]                                at /rustc/33835004928d3bf65db4d4712e1330766263b0bd/library/std/src/io/mod.rs:1976:13
[INFO] [stdout]    8:     0x65090de6101f - <std[ec419fa8110287d5]::sys::backtrace::BacktraceLock>::print
[INFO] [stdout]                                at /rustc/33835004928d3bf65db4d4712e1330766263b0bd/library/std/src/sys/backtrace.rs:47:9
[INFO] [stdout]    9:     0x65090de6101f - std[ec419fa8110287d5]::panicking::default_hook::{closure#0}
[INFO] [stdout]                                at /rustc/33835004928d3bf65db4d4712e1330766263b0bd/library/std/src/panicking.rs:292:27
[INFO] [stdout]   10:     0x65090de7b959 - std[ec419fa8110287d5]::panicking::default_hook
[INFO] [stdout]                                at /rustc/33835004928d3bf65db4d4712e1330766263b0bd/library/std/src/panicking.rs:316:9
[INFO] [stdout]   11:     0x65090dcf9c2c - <alloc[a8a7aba520129d98]::boxed::Box<dyn for<'a, 'b> core[35159d6ffb30e017]::ops::function::Fn<(&'a std[ec419fa8110287d5]::panic::PanicHookInfo<'b>,), Output = ()> + core[35159d6ffb30e017]::marker::Sync + core[35159d6ffb30e017]::marker::Send> as core[35159d6ffb30e017]::ops::function::Fn<(&std[ec419fa8110287d5]::panic::PanicHookInfo,)>>::call
[INFO] [stdout]                                at /rustc/33835004928d3bf65db4d4712e1330766263b0bd/library/alloc/src/boxed.rs:2285:9
[INFO] [stdout]   12:     0x65090dcf9c2c - test[a8007afcd3effadc]::test_main_inner::<test[a8007afcd3effadc]::test_main_static::{closure#0}>::{closure#0}
[INFO] [stdout]                                at /rustc/33835004928d3bf65db4d4712e1330766263b0bd/library/test/src/lib.rs:155:21
[INFO] [stdout]   13:     0x65090de7bbd2 - <alloc[a8a7aba520129d98]::boxed::Box<dyn for<'a, 'b> core[35159d6ffb30e017]::ops::function::Fn<(&'a std[ec419fa8110287d5]::panic::PanicHookInfo<'b>,), Output = ()> + core[35159d6ffb30e017]::marker::Sync + core[35159d6ffb30e017]::marker::Send> as core[35159d6ffb30e017]::ops::function::Fn<(&std[ec419fa8110287d5]::panic::PanicHookInfo,)>>::call
[INFO] [stdout]                                at /rustc/33835004928d3bf65db4d4712e1330766263b0bd/library/alloc/src/boxed.rs:2285:9
[INFO] [stdout]   14:     0x65090de7bbd2 - std[ec419fa8110287d5]::panicking::panic_with_hook
[INFO] [stdout]                                at /rustc/33835004928d3bf65db4d4712e1330766263b0bd/library/std/src/panicking.rs:833:13
[INFO] [stdout]   15:     0x65090de610d8 - std[ec419fa8110287d5]::panicking::panic_handler::{closure#0}
[INFO] [stdout]                                at /rustc/33835004928d3bf65db4d4712e1330766263b0bd/library/std/src/panicking.rs:698:13
[INFO] [stdout]   16:     0x65090de58789 - std[ec419fa8110287d5]::sys::backtrace::__rust_end_short_backtrace::<std[ec419fa8110287d5]::panicking::panic_handler::{closure#0}, !>
[INFO] [stdout]                                at /rustc/33835004928d3bf65db4d4712e1330766263b0bd/library/std/src/sys/backtrace.rs:182:18
[INFO] [stdout]   17:     0x65090de61ded - __rustc[cfb3622dd870db93]::rust_begin_unwind
[INFO] [stdout]                                at /rustc/33835004928d3bf65db4d4712e1330766263b0bd/library/std/src/panicking.rs:689:5
[INFO] [stdout]   18:     0x65090de9be9c - core[35159d6ffb30e017]::panicking::panic_fmt
[INFO] [stdout]                                at /rustc/33835004928d3bf65db4d4712e1330766263b0bd/library/core/src/panicking.rs:80:14
[INFO] [stdout]   19:     0x65090dc97884 - shardlake_index[ab56ae5bd8802d42]::pipeline::tests::pipeline_run_emits_shard_load_and_ann_search_spans
[INFO] [stdout]                                at /opt/rustwide/workdir/crates/shardlake-index/src/pipeline.rs:1238:9
[INFO] [stdout]   20:     0x65090dc80df7 - shardlake_index[ab56ae5bd8802d42]::pipeline::tests::pipeline_run_emits_shard_load_and_ann_search_spans::{closure#0}
[INFO] [stdout]                                at /opt/rustwide/workdir/crates/shardlake-index/src/pipeline.rs:1217:60
[INFO] [stdout]   21:     0x65090dc14396 - <shardlake_index[ab56ae5bd8802d42]::pipeline::tests::pipeline_run_emits_shard_load_and_ann_search_spans::{closure#0} as core[35159d6ffb30e017]::ops::function::FnOnce<()>>::call_once
[INFO] [stdout]                                at /rustc/33835004928d3bf65db4d4712e1330766263b0bd/library/core/src/ops/function.rs:250:5
[INFO] [stdout]   22:     0x65090dcecf0b - <fn() -> core[35159d6ffb30e017]::result::Result<(), alloc[a8a7aba520129d98]::string::String> as core[35159d6ffb30e017]::ops::function::FnOnce<()>>::call_once
[INFO] [stdout]                                at /rustc/33835004928d3bf65db4d4712e1330766263b0bd/library/core/src/ops/function.rs:250:5
[INFO] [stdout]   23:     0x65090dcecf0b - test[a8007afcd3effadc]::__rust_begin_short_backtrace::<core[35159d6ffb30e017]::result::Result<(), alloc[a8a7aba520129d98]::string::String>, fn() -> core[35159d6ffb30e017]::result::Result<(), alloc[a8a7aba520129d98]::string::String>>
[INFO] [stdout]                                at /rustc/33835004928d3bf65db4d4712e1330766263b0bd/library/test/src/lib.rs:724:18
[INFO] [stdout]   24:     0x65090dcfa6fb - test[a8007afcd3effadc]::run_test_in_process::{closure#0}
[INFO] [stdout]                                at /rustc/33835004928d3bf65db4d4712e1330766263b0bd/library/test/src/lib.rs:747:74
[INFO] [stdout]   25:     0x65090dcfa6fb - <core[35159d6ffb30e017]::panic::unwind_safe::AssertUnwindSafe<test[a8007afcd3effadc]::run_test_in_process::{closure#0}> as core[35159d6ffb30e017]::ops::function::FnOnce<()>>::call_once
[INFO] [stdout]                                at /rustc/33835004928d3bf65db4d4712e1330766263b0bd/library/core/src/panic/unwind_safe.rs:275:9
[INFO] [stdout]   26:     0x65090dcfa6fb - std[ec419fa8110287d5]::panicking::catch_unwind::do_call::<core[35159d6ffb30e017]::panic::unwind_safe::AssertUnwindSafe<test[a8007afcd3effadc]::run_test_in_process::{closure#0}>, core[35159d6ffb30e017]::result::Result<(), alloc[a8a7aba520129d98]::string::String>>
[INFO] [stdout]                                at /rustc/33835004928d3bf65db4d4712e1330766263b0bd/library/std/src/panicking.rs:581:40
[INFO] [stdout]   27:     0x65090dcfa6fb - std[ec419fa8110287d5]::panicking::catch_unwind::<core[35159d6ffb30e017]::result::Result<(), alloc[a8a7aba520129d98]::string::String>, core[35159d6ffb30e017]::panic::unwind_safe::AssertUnwindSafe<test[a8007afcd3effadc]::run_test_in_process::{closure#0}>>
[INFO] [stdout]                                at /rustc/33835004928d3bf65db4d4712e1330766263b0bd/library/std/src/panicking.rs:544:19
[INFO] [stdout]   28:     0x65090dcfa6fb - std[ec419fa8110287d5]::panic::catch_unwind::<core[35159d6ffb30e017]::panic::unwind_safe::AssertUnwindSafe<test[a8007afcd3effadc]::run_test_in_process::{closure#0}>, core[35159d6ffb30e017]::result::Result<(), alloc[a8a7aba520129d98]::string::String>>
[INFO] [stdout]                                at /rustc/33835004928d3bf65db4d4712e1330766263b0bd/library/std/src/panic.rs:359:14
[INFO] [stdout]   29:     0x65090dcfa6fb - test[a8007afcd3effadc]::run_test_in_process
[INFO] [stdout]                                at /rustc/33835004928d3bf65db4d4712e1330766263b0bd/library/test/src/lib.rs:747:27
[INFO] [stdout]   30:     0x65090dcfa6fb - test[a8007afcd3effadc]::run_test::{closure#0}
[INFO] [stdout]                                at /rustc/33835004928d3bf65db4d4712e1330766263b0bd/library/test/src/lib.rs:668:43
[INFO] [stdout]   31:     0x65090dcf50d4 - test[a8007afcd3effadc]::run_test::{closure#1}
[INFO] [stdout]                                at /rustc/33835004928d3bf65db4d4712e1330766263b0bd/library/test/src/lib.rs:698:41
[INFO] [stdout]   32:     0x65090dcf50d4 - std[ec419fa8110287d5]::sys::backtrace::__rust_begin_short_backtrace::<test[a8007afcd3effadc]::run_test::{closure#1}, ()>
[INFO] [stdout]                                at /rustc/33835004928d3bf65db4d4712e1330766263b0bd/library/std/src/sys/backtrace.rs:166:18
[INFO] [stdout]   33:     0x65090dcfd302 - std[ec419fa8110287d5]::thread::lifecycle::spawn_unchecked::<test[a8007afcd3effadc]::run_test::{closure#1}, ()>::{closure#1}::{closure#0}
[INFO] [stdout]                                at /rustc/33835004928d3bf65db4d4712e1330766263b0bd/library/std/src/thread/lifecycle.rs:70:13
[INFO] [stdout]   34:     0x65090dcfd302 - <core[35159d6ffb30e017]::panic::unwind_safe::AssertUnwindSafe<std[ec419fa8110287d5]::thread::lifecycle::spawn_unchecked<test[a8007afcd3effadc]::run_test::{closure#1}, ()>::{closure#1}::{closure#0}> as core[35159d6ffb30e017]::ops::function::FnOnce<()>>::call_once
[INFO] [stdout]                                at /rustc/33835004928d3bf65db4d4712e1330766263b0bd/library/core/src/panic/unwind_safe.rs:275:9
[INFO] [stdout]   35:     0x65090dcfd302 - std[ec419fa8110287d5]::panicking::catch_unwind::do_call::<core[35159d6ffb30e017]::panic::unwind_safe::AssertUnwindSafe<std[ec419fa8110287d5]::thread::lifecycle::spawn_unchecked<test[a8007afcd3effadc]::run_test::{closure#1}, ()>::{closure#1}::{closure#0}>, ()>
[INFO] [stdout]                                at /rustc/33835004928d3bf65db4d4712e1330766263b0bd/library/std/src/panicking.rs:581:40
[INFO] [stdout]   36:     0x65090dcfd302 - std[ec419fa8110287d5]::panicking::catch_unwind::<(), core[35159d6ffb30e017]::panic::unwind_safe::AssertUnwindSafe<std[ec419fa8110287d5]::thread::lifecycle::spawn_unchecked<test[a8007afcd3effadc]::run_test::{closure#1}, ()>::{closure#1}::{closure#0}>>
[INFO] [stdout]                                at /rustc/33835004928d3bf65db4d4712e1330766263b0bd/library/std/src/panicking.rs:544:19
[INFO] [stdout]   37:     0x65090dcfd302 - std[ec419fa8110287d5]::panic::catch_unwind::<core[35159d6ffb30e017]::panic::unwind_safe::AssertUnwindSafe<std[ec419fa8110287d5]::thread::lifecycle::spawn_unchecked<test[a8007afcd3effadc]::run_test::{closure#1}, ()>::{closure#1}::{closure#0}>, ()>
[INFO] [stdout]                                at /rustc/33835004928d3bf65db4d4712e1330766263b0bd/library/std/src/panic.rs:359:14
[INFO] [stdout]   38:     0x65090dcfd302 - std[ec419fa8110287d5]::thread::lifecycle::spawn_unchecked::<test[a8007afcd3effadc]::run_test::{closure#1}, ()>::{closure#1}
[INFO] [stdout]                                at /rustc/33835004928d3bf65db4d4712e1330766263b0bd/library/std/src/thread/lifecycle.rs:68:26
[INFO] [stdout]   39:     0x65090dcfd302 - <std[ec419fa8110287d5]::thread::lifecycle::spawn_unchecked<test[a8007afcd3effadc]::run_test::{closure#1}, ()>::{closure#1} as core[35159d6ffb30e017]::ops::function::FnOnce<()>>::call_once::{shim:vtable#0}
[INFO] [stdout]                                at /rustc/33835004928d3bf65db4d4712e1330766263b0bd/library/core/src/ops/function.rs:250:5
[INFO] [stdout]   40:     0x65090de8384f - <alloc[a8a7aba520129d98]::boxed::Box<dyn core[35159d6ffb30e017]::ops::function::FnOnce<(), Output = ()> + core[35159d6ffb30e017]::marker::Send> as core[35159d6ffb30e017]::ops::function::FnOnce<()>>::call_once
[INFO] [stdout]                                at /rustc/33835004928d3bf65db4d4712e1330766263b0bd/library/alloc/src/boxed.rs:2271:9
[INFO] [stdout]   41:     0x65090de8384f - <std[ec419fa8110287d5]::sys::thread::unix::Thread>::new::thread_start
[INFO] [stderr] error: test failed, to rerun pass `-p shardlake-index --lib`
[INFO] [stdout]                                at /rustc/33835004928d3bf65db4d4712e1330766263b0bd/library/std/src/sys/thread/unix.rs:118:17
[INFO] [stdout]   42:     0x74fe27bb3aa4 - <unknown>
[INFO] [stdout]   43:     0x74fe27c40a64 - clone
[INFO] [stdout]   44:                0x0 - <unknown>
[INFO] [stdout] 
[INFO] [stdout] 
[INFO] [stdout] failures:
[INFO] [stdout]     pipeline::tests::pipeline_run_emits_shard_load_and_ann_search_spans
[INFO] [stdout] 
[INFO] [stdout] test result: FAILED. 202 passed; 1 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.21s
[INFO] [stdout] 
[INFO] running `Command { std: "docker" "inspect" "a5b9d8b5cd89097028aa3ff54be05b0279b75fb59a78c6749b676c6f613f47c5", kill_on_drop: false }`
[INFO] running `Command { std: "docker" "rm" "-f" "a5b9d8b5cd89097028aa3ff54be05b0279b75fb59a78c6749b676c6f613f47c5", kill_on_drop: false }`
[INFO] [stdout] a5b9d8b5cd89097028aa3ff54be05b0279b75fb59a78c6749b676c6f613f47c5
