[INFO] fetching crate quixotic-plecostomus-load-balancing 0.11.0...
[INFO] testing quixotic-plecostomus-load-balancing-0.11.0 against 1.99.0-beta.8+cargoflags=--release for beta-release-1.100-2
[INFO] extracting crate quixotic-plecostomus-load-balancing 0.11.0 into /workspace/builds/worker-4-tc1/source
[INFO] started tweaking crates.io crate quixotic-plecostomus-load-balancing 0.11.0
[INFO] finished tweaking crates.io crate quixotic-plecostomus-load-balancing 0.11.0
[INFO] tweaked toml for crates.io crate quixotic-plecostomus-load-balancing 0.11.0 written to /workspace/builds/worker-4-tc1/source/Cargo.toml
[INFO] validating manifest of crates.io crate quixotic-plecostomus-load-balancing 0.11.0 on toolchain 1.99.0-beta.8
[INFO] running `Command { std: CARGO_HOME="/workspace/cargo-home" RUSTUP_HOME="/workspace/rustup-home" "/workspace/cargo-home/bin/cargo" "+1.99.0-beta.8" "metadata" "--manifest-path" "Cargo.toml" "--no-deps", kill_on_drop: false }`
[INFO] crate crates.io crate quixotic-plecostomus-load-balancing 0.11.0 already has a lockfile, it will not be regenerated
[INFO] running `Command { std: CARGO_HOME="/workspace/cargo-home" RUSTUP_HOME="/workspace/rustup-home" "/workspace/cargo-home/bin/cargo" "+1.99.0-beta.8" "fetch" "--manifest-path" "Cargo.toml", kill_on_drop: false }`
[INFO] [stderr]     Updating crates.io index
[INFO] [stderr]  Downloading crates ...
[INFO] [stderr]   Downloaded quixotic-plecostomus-ketama v0.11.0
[INFO] [stderr]   Downloaded quixotic-plecostomus-http v0.11.0
[INFO] [stderr]   Downloaded quixotic-plecostomus-error v0.11.0
[INFO] [stderr]   Downloaded quixotic-plecostomus-runtime v0.11.0
[INFO] [stderr]   Downloaded i_key_sort v0.10.3
[INFO] [stderr]   Downloaded libyaml-rs v0.3.0
[INFO] [stderr]   Downloaded quixotic-plecostomus-timeout v0.11.0
[INFO] [stderr]   Downloaded quixotic-plecostomus-rustls v0.11.0
[INFO] [stderr]   Downloaded quixotic-plecostomus-s2n v0.11.0
[INFO] [stderr]   Downloaded yaml_serde v0.10.7
[INFO] [stderr]   Downloaded sfv v0.10.4
[INFO] [stderr]   Downloaded quixotic-plecostomus-openssl v0.11.0
[INFO] [stderr]   Downloaded s2n-tls-tokio v0.3.38
[INFO] [stderr]   Downloaded cc v1.4.6
[INFO] [stderr]   Downloaded no_debug v3.1.0
[INFO] [stderr]   Downloaded tokio-openssl v0.6.5
[INFO] [stderr]   Downloaded fslock v0.2.1
[INFO] [stderr]   Downloaded quixotic-plecostomus-pool v0.11.0
[INFO] [stderr]   Downloaded quixotic-plecostomus-boringssl v0.11.0
[INFO] [stderr]   Downloaded s2n-tls v0.3.38
[INFO] [stderr]   Downloaded seize v0.3.3
[INFO] [stderr]   Downloaded daemonix v0.1.0
[INFO] [stderr]   Downloaded flurry v0.5.2
[INFO] [stderr]   Downloaded boring v5.2.0
[INFO] [stderr]   Downloaded quixotic-plecostomus-core v0.11.0
[INFO] [stderr]   Downloaded s2n-tls-sys v0.3.38
[INFO] [stderr]   Downloaded libz-ng-sys v1.1.29
[INFO] [stderr]   Downloaded boring-sys v5.2.0
[INFO] running `Command { std: "docker" "create" "-v" "/var/lib/crater-agent-workspace/builds/worker-4-tc1/source:/opt/rustwide/workdir:ro,Z" "-v" "/var/lib/crater-agent-workspace/builds/worker-4-tc1/target:/opt/rustwide/target:rw,Z" "-v" "/var/lib/crater-agent-workspace/cargo-home:/opt/rustwide/cargo-home:ro,Z" "-v" "/var/lib/crater-agent-workspace/rustup-home:/opt/rustwide/rustup-home:ro,Z" "-m" "1610612736" "--network" "none" "ghcr.io/rust-lang/crates-build-env/linux@sha256:77db811e55d90add9212f6832c23229e3de5f1c1f9905e8cb07319ea15df73ac" "sleep" "infinity", kill_on_drop: false }`
[INFO] [stdout] 4197699ac10e1e81658b74c592bcadc9d17357847a8272068c2a4d30a4411de5
[INFO] running `Command { std: "docker" "start" "4197699ac10e1e81658b74c592bcadc9d17357847a8272068c2a4d30a4411de5", kill_on_drop: false }`
[INFO] running `Command { std: "docker" "inspect" "4197699ac10e1e81658b74c592bcadc9d17357847a8272068c2a4d30a4411de5", kill_on_drop: false }`
[INFO] running `Command { std: "docker" "exec" "-e" "SOURCE_DIR=/opt/rustwide/workdir" "-e" "CARGO_HOME=/opt/rustwide/cargo-home" "-e" "RUSTUP_HOME=/opt/rustwide/rustup-home" "-e" "CARGO_TARGET_DIR=/opt/rustwide/target" "-w" "/opt/rustwide/workdir" "--user" "0:0" "4197699ac10e1e81658b74c592bcadc9d17357847a8272068c2a4d30a4411de5" "/opt/rustwide/cargo-home/bin/cargo" "+1.99.0-beta.8" "metadata" "--no-deps" "--format-version=1", kill_on_drop: false }`
[INFO] running `Command { std: "docker" "inspect" "4197699ac10e1e81658b74c592bcadc9d17357847a8272068c2a4d30a4411de5", kill_on_drop: false }`
[INFO] running `Command { std: "docker" "exec" "-e" "SOURCE_DIR=/opt/rustwide/workdir" "-e" "CARGO_HOME=/opt/rustwide/cargo-home" "-e" "RUSTUP_HOME=/opt/rustwide/rustup-home" "-e" "CARGO_TARGET_DIR=/opt/rustwide/target" "-e" "CARGO_INCREMENTAL=0" "-e" "RUST_BACKTRACE=full" "-e" "RUSTFLAGS=--cap-lints=warn" "-e" "RUSTDOCFLAGS=--cap-lints=warn" "-w" "/opt/rustwide/workdir" "--user" "0:0" "4197699ac10e1e81658b74c592bcadc9d17357847a8272068c2a4d30a4411de5" "/opt/rustwide/cargo-home/bin/cargo" "+1.99.0-beta.8" "build" "--frozen" "--message-format=json" "--release", kill_on_drop: false }`
[INFO] [stderr]    Compiling errno v0.3.14
[INFO] [stderr]    Compiling mio v1.2.3
[INFO] [stderr]    Compiling hashbrown v0.17.1
[INFO] [stderr]    Compiling syn v3.0.5
[INFO] [stderr]    Compiling cc v1.4.6
[INFO] [stderr]    Compiling smallvec v1.16.1
[INFO] [stderr]    Compiling getrandom v0.2.17
[INFO] [stderr]    Compiling once_cell v1.21.4
[INFO] [stderr]    Compiling signal-hook-registry v1.4.8
[INFO] [stderr]    Compiling const-random-macro v0.1.16
[INFO] [stderr]    Compiling anstyle-parse v1.0.0
[INFO] [stderr]    Compiling parking_lot_core v0.9.12
[INFO] [stderr]    Compiling ahash v0.8.12
[INFO] [stderr]    Compiling colorchoice v1.0.5
[INFO] [stderr]    Compiling rust_decimal v1.43.0
[INFO] [stderr]    Compiling nix v0.31.3
[INFO] [stderr]    Compiling anstream v1.0.0
[INFO] [stderr]    Compiling regex-automata v0.4.18
[INFO] [stderr]    Compiling parking_lot v0.12.5
[INFO] [stderr]    Compiling getrandom v0.3.4
[INFO] [stderr]    Compiling const-random v0.1.18
[INFO] [stderr]    Compiling crc32fast v1.5.2
[INFO] [stderr]    Compiling indexmap v2.14.2
[INFO] [stderr]    Compiling dashmap v6.2.1
[INFO] [stderr]    Compiling clap_builder v4.6.6
[INFO] [stderr]    Compiling cmake v0.1.58
[INFO] [stderr]    Compiling zstd-sys v2.1.0+zstd.1.5.7
[INFO] [stderr]    Compiling libz-ng-sys v1.1.29
[INFO] [stderr]    Compiling lru v0.18.4
[INFO] [stderr]    Compiling petgraph v0.7.1
[INFO] [stderr]    Compiling brotli-decompressor v2.5.1
[INFO] [stderr]    Compiling crossbeam-queue v0.3.14
[INFO] [stderr]    Compiling syn v2.0.119
[INFO] [stderr]    Compiling num_cpus v1.17.0
[INFO] [stderr]    Compiling tokio-macros v2.7.2
[INFO] [stderr]    Compiling tokio v1.53.1
[INFO] [stderr]    Compiling futures-macro v0.3.34
[INFO] [stderr]    Compiling futures-util v0.3.34
[INFO] [stderr]    Compiling serde_derive v1.0.229
[INFO] [stderr]    Compiling clap_derive v4.6.4
[INFO] [stderr]    Compiling futures-executor v0.3.34
[INFO] [stderr]    Compiling futures v0.3.34
[INFO] [stderr]    Compiling seize v0.3.3
[INFO] [stderr]    Compiling serde v1.0.229
[INFO] [stderr]    Compiling quixotic-plecostomus-timeout v0.11.0
[INFO] [stderr]    Compiling tokio-util v0.7.19
[INFO] [stderr]    Compiling tokio-stream v0.1.19
[INFO] [stderr]    Compiling quixotic-plecostomus-error v0.11.0
[INFO] [stderr]    Compiling libyaml-rs v0.3.0
[INFO] [stderr]    Compiling h2 v0.4.19
[INFO] [stderr]    Compiling sfv v0.10.4
[INFO] [stderr]    Compiling yaml_serde v0.10.7
[INFO] [stderr]    Compiling quixotic-plecostomus-http v0.11.0
[INFO] [stderr]    Compiling tokio-test v0.4.5
[INFO] [stderr]    Compiling clap v4.6.6
[INFO] [stderr]    Compiling quixotic-plecostomus-runtime v0.11.0
[INFO] [stderr]    Compiling quixotic-plecostomus-pool v0.11.0
[INFO] [stderr]    Compiling flurry v0.5.2
[INFO] [stderr]    Compiling strum_macros v0.26.4
[INFO] [stderr]    Compiling async-trait v0.1.92
[INFO] [stderr]    Compiling regex v1.13.1
[INFO] [stderr]    Compiling bstr v1.13.1
[INFO] [stderr]    Compiling daggy v0.8.1
[INFO] [stderr]    Compiling brotli v3.5.0
[INFO] [stderr]    Compiling chrono v0.4.45
[INFO] [stderr]    Compiling socket2 v0.5.10
[INFO] [stderr]    Compiling daemonix v0.1.0
[INFO] [stderr]    Compiling quixotic-plecostomus-ketama v0.11.0
[INFO] [stderr]    Compiling arc-swap v1.9.2
[INFO] [stderr]    Compiling flate2 v1.1.10
[INFO] [stderr]    Compiling zstd-safe v7.3.0
[INFO] [stderr]    Compiling zstd v0.13.3
[INFO] [stderr]    Compiling quixotic-plecostomus-core v0.11.0
[INFO] [stderr]    Compiling quixotic-plecostomus-load-balancing v0.11.0 (/opt/rustwide/workdir)
[INFO] [stderr]     Finished `release` profile [optimized] target(s) in 4m 27s
[INFO] running `Command { std: "docker" "inspect" "4197699ac10e1e81658b74c592bcadc9d17357847a8272068c2a4d30a4411de5", kill_on_drop: false }`
[INFO] running `Command { std: "docker" "exec" "-e" "SOURCE_DIR=/opt/rustwide/workdir" "-e" "CARGO_HOME=/opt/rustwide/cargo-home" "-e" "RUSTUP_HOME=/opt/rustwide/rustup-home" "-e" "CARGO_TARGET_DIR=/opt/rustwide/target" "-e" "CARGO_INCREMENTAL=0" "-e" "RUST_BACKTRACE=full" "-e" "RUSTFLAGS=--cap-lints=warn" "-e" "RUSTDOCFLAGS=--cap-lints=warn" "-w" "/opt/rustwide/workdir" "--user" "0:0" "4197699ac10e1e81658b74c592bcadc9d17357847a8272068c2a4d30a4411de5" "/opt/rustwide/cargo-home/bin/cargo" "+1.99.0-beta.8" "test" "--frozen" "--no-run" "--message-format=json" "--release", kill_on_drop: false }`
[INFO] [stderr]    Compiling quixotic-plecostomus-load-balancing v0.11.0 (/opt/rustwide/workdir)
[INFO] [stderr]     Finished `release` profile [optimized] target(s) in 48.15s
[INFO] running `Command { std: "docker" "inspect" "4197699ac10e1e81658b74c592bcadc9d17357847a8272068c2a4d30a4411de5", kill_on_drop: false }`
[INFO] running `Command { std: "docker" "exec" "-e" "SOURCE_DIR=/opt/rustwide/workdir" "-e" "CARGO_HOME=/opt/rustwide/cargo-home" "-e" "RUSTUP_HOME=/opt/rustwide/rustup-home" "-e" "CARGO_TARGET_DIR=/opt/rustwide/target" "-e" "CARGO_INCREMENTAL=0" "-e" "RUST_BACKTRACE=full" "-e" "RUSTFLAGS=--cap-lints=warn" "-e" "RUSTDOCFLAGS=--cap-lints=warn" "-w" "/opt/rustwide/workdir" "--user" "0:0" "4197699ac10e1e81658b74c592bcadc9d17357847a8272068c2a4d30a4411de5" "/opt/rustwide/cargo-home/bin/cargo" "+1.99.0-beta.8" "test" "--frozen" "--release", kill_on_drop: false }`
[INFO] [stderr]     Finished `release` profile [optimized] target(s) in 0.22s
[INFO] [stderr]      Running unittests src/lib.rs (/opt/rustwide/target/release/deps/pingora_load_balancing-100acbaa7b5273ba)
[INFO] [stdout] 
[INFO] [stdout] running 78 tests
[INFO] [stdout] test background::tests::group_run_returns_when_shutdown_while_updating ... ok
[INFO] [stdout] test background::tests::one_time_update_schedule_finishes_after_first_run ... ok
[INFO] [stdout] test background::tests::health_check_service_run_returns_when_shutdown_while_health_checking ... ok
[INFO] [stdout] test background::tests::run_returns_when_shutdown_while_sleeping ... ok
[INFO] [stdout] test background::tests::one_time_health_check_schedule_finishes_after_first_run ... ok
[INFO] [stdout] test background::tests::run_aborts_parallel_health_check_on_shutdown ... ok
[INFO] [stdout] test background::tests::periodic_health_check_schedule_tracks_next_pass ... ok
[INFO] [stdout] test background::tests::run_returns_when_shutdown_while_health_checking ... ok
[INFO] [stdout] test background::tests::group_run_returns_when_shutdown_while_health_checking ... ok
[INFO] [stdout] test background::tests::run_returns_when_shutdown_while_updating ... ok
[INFO] [stdout] test background::tests::run_updates_when_no_shutdown_sender_remains ... ok
[INFO] [stdout] test background::tests::schedule_uses_earliest_task_deadline ... ok
[INFO] [stdout] test background::tests::schedule_without_tasks_is_idle ... ok
[INFO] [stdout] test health_check::test::test_health_observe ... ok
[INFO] [stdout] test health_check::test::test_http_custom_check ... FAILED
[INFO] [stdout] test selection::tests::unique_iter_duplicate_items_are_filtered ... ok
[INFO] [stdout] test selection::tests::unique_iter_max_iterations_is_correct ... ok
[INFO] [stdout] test health_check::test::test_tcp_check ... FAILED
[INFO] [stdout] test selection::consistent::test::test_ketama ... ok
[INFO] [stdout] test selection::weighted::test::test_fnv ... ok
[INFO] [stdout] test selection::weighted::test::test_round_robin ... ok
[INFO] [stdout] test selection::weighted::test::test_random ... ok
[INFO] [stdout] test test::test_backend_view_enablement_is_not_shared ... ok
[INFO] [stdout] test test::test_backends_with_ext ... ok
[INFO] [stdout] test test::test_current_readiness_uses_exact_backend_identity ... ok
[INFO] [stdout] test test::test_different_health_registries_probe_independently ... ok
[INFO] [stdout] test test::test_backends ... FAILED
[INFO] [stdout] test test::test_discovery_readiness ... ok
[INFO] [stdout] test test::test_cancelled_selector_destruction_holds_rebuild_gate ... ok
[INFO] [stdout] test test::test_group_selection_available_across_disjoint_membership_swap ... ok
[INFO] [stdout] test test::test_discovery_override_updates_retained_snapshot_on_readd ... ok
[INFO] [stdout] test test::test_health_check_service_reconciles_dropped_view ... ok
[INFO] [stdout] test test::test_group_old_selector_serves_disappeared_address_via_snapshot ... ok
[INFO] [stdout] test test::test_health_check_service_shuts_down_while_waiting_for_targets ... ok
[INFO] [stdout] test test::test_health_check_service_runs_shared_registry_once ... ok
[INFO] [stdout] test test::test_health_check_service_wakes_when_first_view_publishes_targets ... ok
[INFO] [stdout] test test::test_health_check_service_without_check_does_not_signal_ready ... ok
[INFO] [stdout] test test::test_group_weight_churn_preserves_selector_identity ... ok
[INFO] [stdout] test test::test_health_registry_shares_probes_across_backend_views ... ok
[INFO] [stdout] test test::test_health_registry_deduplicates_metadata_variants_by_address ... ok
[INFO] [stdout] test test::test_lb_update_stores_timing ... ok
[INFO] [stdout] test test::test_health_registry_rejects_replacing_health_check - should panic ... ok
[INFO] [stdout] test test::test_load_balancer_group_rejects_updated_backends - should panic ... ok
[INFO] [stdout] test test::test_load_balancer_group_shares_health_checks ... ok
[INFO] [stdout] test test::test_load_balancer_group_rebuilds_all_selectors ... ok
[INFO] [stdout] test test::test_group_selector_retries_after_failed_build ... ok
[INFO] [stdout] test test::test_manual_enablement_survives_discovery_updates_without_override ... ok
[INFO] [stdout] test test::test_load_balancer_group_bounds_and_coalesces_rebuilds ... ok
[INFO] [stdout] test test::test_load_balancer_group_waits_for_initial_selectors_before_ready ... ok
[INFO] [stdout] test test::test_new_selector_is_ready_before_update_callback_returns ... ok
[INFO] [stdout] test test::test_manual_enablement_survives_transition_publication ... ok
[INFO] [stdout] test test::test_parallel_health_check ... FAILED
[INFO] [stdout] test test::test_parallel_health_check_isolates_a_panicking_check ... ok
[INFO] [stdout] test test::test_old_published_snapshot_selects_disjoint_backend_after_update ... ok
[INFO] [stdout] test test::test_private_registry_preserves_backend_identity ... ok
[INFO] [stdout] test test::test_load_balancer_group_waits_for_successful_update_before_ready ... ok
[INFO] [stdout] test test::test_rebuild_gate_cancellation_preserves_retired_selector ... ok
[INFO] [stdout] test test::test_same_address_variants_have_independent_readiness ... ok
[INFO] [stdout] test test::test_select_during_blocking_update_keeps_old_backend_ready ... ok
[INFO] [stdout] test test::test_retired_selector_destruction_does_not_block_runtime ... ok
[INFO] [stdout] test test::test_selector_rebuild_task_guard_cleans_up_after_panic ... ok
[INFO] [stdout] test test::test_health_check_service_one_shot_waits_for_first_target ... ok
[INFO] [stdout] test test::test_selectors_at_different_generations_use_matching_snapshots ... ok
[INFO] [stdout] test test::test_shared_health_update_visible_after_other_view_reconcile ... ok
[INFO] [stdout] test test::test_group_readiness_tracks_latest_generation_during_burst ... ok
[INFO] [stdout] test test::test_selector_rebuild_task_abort_restores_in_flight_request ... ok
[INFO] [stdout] test test::test_static_backends ... ok
[INFO] [stdout] test test::test_transition_snapshot_covers_outgoing_and_incoming_readiness ... ok
[INFO] [stdout] test test::test_stale_group_selector_serves_removed_backend_until_converged ... ok
[INFO] [stdout] test test::test_view_drop_defers_removal_without_locking_views ... ok
[INFO] [stdout] test background::tests::run_continues_after_a_non_shutdown_watch_change ... ok
[INFO] [stdout] test test::test_unchanged_backends_do_not_rebuild_view_or_selector ... ok
[INFO] [stdout] test test::test_registry_notifies_only_when_first_target_is_added ... ok
[INFO] [stdout] test test::test_shared_rebuild_gate_never_overlaps_retired_and_replacement_selectors ... ok
[INFO] [stdout] test test::test_selector_release_waits_for_iterator_and_destructor ... ok
[INFO] [stdout] test test::test_group_drop_cancels_failed_rebuild_retries ... ok
[INFO] [stdout] test test::thread_safety::test_consistency ... ok
[INFO] [stdout] test test::test_new_generation_interrupts_rebuild_backoff ... ok
[INFO] [stdout] 
[INFO] [stdout] failures:
[INFO] [stdout] 
[INFO] [stdout] ---- health_check::test::test_http_custom_check stdout ----
[INFO] [stdout] 
[INFO] [stdout] thread 'health_check::test::test_http_custom_check' (2306) panicked at src/health_check.rs:486:42:
[INFO] [stdout] called `Result::unwrap()` on an `Err` value: Error { etype: ConnectNoRoute, esource: Unset, retry: Decided(false), cause: Some(Error { etype: ConnectNoRoute, esource: Unset, retry: Decided(false), cause: Some(ConnectErrorDetails { source: Some(Os { code: 101, kind: NetworkUnreachable, message: "Network is unreachable" }), local_addr: Some(0.0.0.0:0) }), context: Some(Owned("Fail to connect to 1.1.1.1:80")) }), context: Some(Owned("Fail to connect to addr: 1.1.1.1:80, scheme: HTTP")) }
[INFO] [stdout] stack backtrace:
[INFO] [stdout]    0:     0x615a2d345dd1 - <<std[70759c8f55707aa4]::sys::backtrace::BacktraceLock>::print::DisplayBacktrace as core[df12db4294e9bfd3]::fmt::Display>::fmt
[INFO] [stdout]    1:     0x615a2d35ca8a - core[df12db4294e9bfd3]::fmt::write
[INFO] [stdout]    2:     0x615a2d34adfc - <alloc[2182bb758b4b3781]::vec::Vec<u8> as core[df12db4294e9bfd3]::io::write::Write>::write_fmt
[INFO] [stdout]    3:     0x615a2d320466 - std[70759c8f55707aa4]::panicking::default_hook::{closure#0}
[INFO] [stdout]    4:     0x615a2d33d819 - std[70759c8f55707aa4]::panicking::default_hook
[INFO] [stdout]    5:     0x615a2d231c40 - test[9d35eded1c95d3be]::test_main_inner::<test[9d35eded1c95d3be]::test_main_static::{closure#0}>::{closure#0}
[INFO] [stdout]    6:     0x615a2d33db42 - std[70759c8f55707aa4]::panicking::panic_with_hook
[INFO] [stdout]    7:     0x615a2d320512 - std[70759c8f55707aa4]::panicking::panic_handler::{closure#0}
[INFO] [stdout]    8:     0x615a2d3191e9 - std[70759c8f55707aa4]::sys::backtrace::__rust_end_short_backtrace::<std[70759c8f55707aa4]::panicking::panic_handler::{closure#0}, !>
[INFO] [stdout]    9:     0x615a2d3212fd - __rustc[8fa7c3cbc660c2b3]::rust_begin_unwind
[INFO] [stdout]   10:     0x615a2d35d1ec - core[df12db4294e9bfd3]::panicking::panic_fmt
[INFO] [stdout]   11:     0x615a2d35cfc2 - core[df12db4294e9bfd3]::result::unwrap_failed
[INFO] [stdout]   12:     0x615a2d1cefcc - pingora_load_balancing[b161961b48d899d3]::health_check::test::test_http_custom_check::{closure#0}
[INFO] [stdout]   13:     0x615a2d1f70d3 - <tokio[a86046c109545240]::runtime::scheduler::current_thread::Context>::enter::<core[df12db4294e9bfd3]::task::poll::Poll<()>, <tokio[a86046c109545240]::runtime::scheduler::current_thread::CoreGuard>::block_on<core[df12db4294e9bfd3]::pin::Pin<&mut core[df12db4294e9bfd3]::pin::Pin<&mut dyn core[df12db4294e9bfd3]::future::future::Future<Output = ()>>>>::{closure#0}::{closure#0}>
[INFO] [stdout]   14:     0x615a2d1c4cfc - <tokio[a86046c109545240]::runtime::context::scoped::Scoped<tokio[a86046c109545240]::runtime::scheduler::Context>>::set::<<tokio[a86046c109545240]::runtime::scheduler::current_thread::CoreGuard>::enter<<tokio[a86046c109545240]::runtime::scheduler::current_thread::CoreGuard>::block_on<core[df12db4294e9bfd3]::pin::Pin<&mut core[df12db4294e9bfd3]::pin::Pin<&mut dyn core[df12db4294e9bfd3]::future::future::Future<Output = ()>>>>::{closure#0}, core[df12db4294e9bfd3]::option::Option<()>>::{closure#0}, (alloc[2182bb758b4b3781]::boxed::Box<tokio[a86046c109545240]::runtime::scheduler::current_thread::Core>, core[df12db4294e9bfd3]::option::Option<()>)>
[INFO] [stdout]   15:     0x615a2d1f696d - <tokio[a86046c109545240]::runtime::scheduler::current_thread::CurrentThread>::block_on::<core[df12db4294e9bfd3]::pin::Pin<&mut dyn core[df12db4294e9bfd3]::future::future::Future<Output = ()>>>
[INFO] [stdout]   16:     0x615a2d1e4a42 - <tokio[a86046c109545240]::runtime::runtime::Runtime>::block_on::<core[df12db4294e9bfd3]::pin::Pin<&mut dyn core[df12db4294e9bfd3]::future::future::Future<Output = ()>>>
[INFO] [stdout]   17:     0x615a2d1d496b - pingora_load_balancing[b161961b48d899d3]::health_check::test::test_http_custom_check
[INFO] [stdout]   18:     0x615a2d166059 - <pingora_load_balancing[b161961b48d899d3]::health_check::test::test_http_custom_check::{closure#0} as core[df12db4294e9bfd3]::ops::function::FnOnce<()>>::call_once
[INFO] [stdout]   19:     0x615a2d224f2b - test[9d35eded1c95d3be]::__rust_begin_short_backtrace::<core[df12db4294e9bfd3]::result::Result<(), alloc[2182bb758b4b3781]::string::String>, fn() -> core[df12db4294e9bfd3]::result::Result<(), alloc[2182bb758b4b3781]::string::String>>
[INFO] [stdout]   20:     0x615a2d232595 - test[9d35eded1c95d3be]::run_test::{closure#0}
[INFO] [stdout]   21:     0x615a2d22be54 - std[70759c8f55707aa4]::sys::backtrace::__rust_begin_short_backtrace::<test[9d35eded1c95d3be]::run_test::{closure#1}, ()>
[INFO] [stdout]   22:     0x615a2d2356f2 - <std[70759c8f55707aa4]::thread::lifecycle::spawn_unchecked<test[9d35eded1c95d3be]::run_test::{closure#1}, ()>::{closure#1} as core[df12db4294e9bfd3]::ops::function::FnOnce<()>>::call_once::{shim:vtable#0}
[INFO] [stdout]   23:     0x615a2d344ec9 - <std[70759c8f55707aa4]::sys::thread::unix::Thread>::new::thread_start
[INFO] [stdout]   24:     0x770fc8243dfa - <unknown>
[INFO] [stdout]   25:     0x770fc82d73d4 - clone
[INFO] [stdout]   26:                0x0 - <unknown>
[INFO] [stdout] 
[INFO] [stdout] ---- health_check::test::test_tcp_check stdout ----
[INFO] [stdout] 
[INFO] [stdout] thread 'health_check::test::test_tcp_check' (2308) panicked at src/health_check.rs:428:9:
[INFO] [stdout] assertion failed: tcp_check.check(&backend).await.is_ok()
[INFO] [stdout] stack backtrace:
[INFO] [stdout]    0:     0x615a2d345dd1 - <<std[70759c8f55707aa4]::sys::backtrace::BacktraceLock>::print::DisplayBacktrace as core[df12db4294e9bfd3]::fmt::Display>::fmt
[INFO] [stdout]    1:     0x615a2d35ca8a - core[df12db4294e9bfd3]::fmt::write
[INFO] [stdout]    2:     0x615a2d34adfc - <alloc[2182bb758b4b3781]::vec::Vec<u8> as core[df12db4294e9bfd3]::io::write::Write>::write_fmt
[INFO] [stdout]    3:     0x615a2d320466 - std[70759c8f55707aa4]::panicking::default_hook::{closure#0}
[INFO] [stdout]    4:     0x615a2d33d819 - std[70759c8f55707aa4]::panicking::default_hook
[INFO] [stdout]    5:     0x615a2d231c40 - test[9d35eded1c95d3be]::test_main_inner::<test[9d35eded1c95d3be]::test_main_static::{closure#0}>::{closure#0}
[INFO] [stdout]    6:     0x615a2d33db42 - std[70759c8f55707aa4]::panicking::panic_with_hook
[INFO] [stdout]    7:     0x615a2d320544 - std[70759c8f55707aa4]::panicking::panic_handler::{closure#0}
[INFO] [stdout]    8:     0x615a2d3191e9 - std[70759c8f55707aa4]::sys::backtrace::__rust_end_short_backtrace::<std[70759c8f55707aa4]::panicking::panic_handler::{closure#0}, !>
[INFO] [stdout]    9:     0x615a2d3212fd - __rustc[8fa7c3cbc660c2b3]::rust_begin_unwind
[INFO] [stdout]   10:     0x615a2d35d1ec - core[df12db4294e9bfd3]::panicking::panic_fmt
[INFO] [stdout]   11:     0x615a2d35d1b2 - core[df12db4294e9bfd3]::panicking::panic
[INFO] [stdout]   12:     0x615a2d1cdaea - pingora_load_balancing[b161961b48d899d3]::health_check::test::test_tcp_check::{closure#0}
[INFO] [stdout]   13:     0x615a2d1f70d3 - <tokio[a86046c109545240]::runtime::scheduler::current_thread::Context>::enter::<core[df12db4294e9bfd3]::task::poll::Poll<()>, <tokio[a86046c109545240]::runtime::scheduler::current_thread::CoreGuard>::block_on<core[df12db4294e9bfd3]::pin::Pin<&mut core[df12db4294e9bfd3]::pin::Pin<&mut dyn core[df12db4294e9bfd3]::future::future::Future<Output = ()>>>>::{closure#0}::{closure#0}>
[INFO] [stdout]   14:     0x615a2d1c4cfc - <tokio[a86046c109545240]::runtime::context::scoped::Scoped<tokio[a86046c109545240]::runtime::scheduler::Context>>::set::<<tokio[a86046c109545240]::runtime::scheduler::current_thread::CoreGuard>::enter<<tokio[a86046c109545240]::runtime::scheduler::current_thread::CoreGuard>::block_on<core[df12db4294e9bfd3]::pin::Pin<&mut core[df12db4294e9bfd3]::pin::Pin<&mut dyn core[df12db4294e9bfd3]::future::future::Future<Output = ()>>>>::{closure#0}, core[df12db4294e9bfd3]::option::Option<()>>::{closure#0}, (alloc[2182bb758b4b3781]::boxed::Box<tokio[a86046c109545240]::runtime::scheduler::current_thread::Core>, core[df12db4294e9bfd3]::option::Option<()>)>
[INFO] [stdout]   15:     0x615a2d1f696d - <tokio[a86046c109545240]::runtime::scheduler::current_thread::CurrentThread>::block_on::<core[df12db4294e9bfd3]::pin::Pin<&mut dyn core[df12db4294e9bfd3]::future::future::Future<Output = ()>>>
[INFO] [stdout]   16:     0x615a2d1e4a42 - <tokio[a86046c109545240]::runtime::runtime::Runtime>::block_on::<core[df12db4294e9bfd3]::pin::Pin<&mut dyn core[df12db4294e9bfd3]::future::future::Future<Output = ()>>>
[INFO] [stdout]   17:     0x615a2d1d46cb - pingora_load_balancing[b161961b48d899d3]::health_check::test::test_tcp_check
[INFO] [stdout]   18:     0x615a2d166019 - <pingora_load_balancing[b161961b48d899d3]::health_check::test::test_tcp_check::{closure#0} as core[df12db4294e9bfd3]::ops::function::FnOnce<()>>::call_once
[INFO] [stdout]   19:     0x615a2d224f2b - test[9d35eded1c95d3be]::__rust_begin_short_backtrace::<core[df12db4294e9bfd3]::result::Result<(), alloc[2182bb758b4b3781]::string::String>, fn() -> core[df12db4294e9bfd3]::result::Result<(), alloc[2182bb758b4b3781]::string::String>>
[INFO] [stdout]   20:     0x615a2d232595 - test[9d35eded1c95d3be]::run_test::{closure#0}
[INFO] [stdout]   21:     0x615a2d22be54 - std[70759c8f55707aa4]::sys::backtrace::__rust_begin_short_backtrace::<test[9d35eded1c95d3be]::run_test::{closure#1}, ()>
[INFO] [stdout]   22:     0x615a2d2356f2 - <std[70759c8f55707aa4]::thread::lifecycle::spawn_unchecked<test[9d35eded1c95d3be]::run_test::{closure#1}, ()>::{closure#1} as core[df12db4294e9bfd3]::ops::function::FnOnce<()>>::call_once::{shim:vtable#0}
[INFO] [stdout]   23:     0x615a2d344ec9 - <std[70759c8f55707aa4]::sys::thread::unix::Thread>::new::thread_start
[INFO] [stdout]   24:     0x770fc8243dfa - <unknown>
[INFO] [stdout]   25:     0x770fc82d73d4 - clone
[INFO] [stdout]   26:                0x0 - <unknown>
[INFO] [stdout] 
[INFO] [stdout] ---- test::test_backends stdout ----
[INFO] [stdout] 
[INFO] [stdout] thread 'test::test_backends' (2316) panicked at src/lib.rs:2363:9:
[INFO] [stdout] assertion failed: backends.ready(&good1)
[INFO] [stdout] stack backtrace:
[INFO] [stdout]    0:     0x615a2d345dd1 - <<std[70759c8f55707aa4]::sys::backtrace::BacktraceLock>::print::DisplayBacktrace as core[df12db4294e9bfd3]::fmt::Display>::fmt
[INFO] [stdout]    1:     0x615a2d35ca8a - core[df12db4294e9bfd3]::fmt::write
[INFO] [stdout]    2:     0x615a2d34adfc - <alloc[2182bb758b4b3781]::vec::Vec<u8> as core[df12db4294e9bfd3]::io::write::Write>::write_fmt
[INFO] [stdout]    3:     0x615a2d320466 - std[70759c8f55707aa4]::panicking::default_hook::{closure#0}
[INFO] [stdout]    4:     0x615a2d33d819 - std[70759c8f55707aa4]::panicking::default_hook
[INFO] [stdout]    5:     0x615a2d231c40 - test[9d35eded1c95d3be]::test_main_inner::<test[9d35eded1c95d3be]::test_main_static::{closure#0}>::{closure#0}
[INFO] [stdout]    6:     0x615a2d33db42 - std[70759c8f55707aa4]::panicking::panic_with_hook
[INFO] [stdout]    7:     0x615a2d320544 - std[70759c8f55707aa4]::panicking::panic_handler::{closure#0}
[INFO] [stdout]    8:     0x615a2d3191e9 - std[70759c8f55707aa4]::sys::backtrace::__rust_end_short_backtrace::<std[70759c8f55707aa4]::panicking::panic_handler::{closure#0}, !>
[INFO] [stdout]    9:     0x615a2d3212fd - __rustc[8fa7c3cbc660c2b3]::rust_begin_unwind
[INFO] [stdout]   10:     0x615a2d35d1ec - core[df12db4294e9bfd3]::panicking::panic_fmt
[INFO] [stdout]   11:     0x615a2d35d1b2 - core[df12db4294e9bfd3]::panicking::panic
[INFO] [stdout]   12:     0x615a2d0d498d - pingora_load_balancing[b161961b48d899d3]::test::test_backends::{closure#0}
[INFO] [stdout]   13:     0x615a2d1f70d3 - <tokio[a86046c109545240]::runtime::scheduler::current_thread::Context>::enter::<core[df12db4294e9bfd3]::task::poll::Poll<()>, <tokio[a86046c109545240]::runtime::scheduler::current_thread::CoreGuard>::block_on<core[df12db4294e9bfd3]::pin::Pin<&mut core[df12db4294e9bfd3]::pin::Pin<&mut dyn core[df12db4294e9bfd3]::future::future::Future<Output = ()>>>>::{closure#0}::{closure#0}>
[INFO] [stdout]   14:     0x615a2d1c4cfc - <tokio[a86046c109545240]::runtime::context::scoped::Scoped<tokio[a86046c109545240]::runtime::scheduler::Context>>::set::<<tokio[a86046c109545240]::runtime::scheduler::current_thread::CoreGuard>::enter<<tokio[a86046c109545240]::runtime::scheduler::current_thread::CoreGuard>::block_on<core[df12db4294e9bfd3]::pin::Pin<&mut core[df12db4294e9bfd3]::pin::Pin<&mut dyn core[df12db4294e9bfd3]::future::future::Future<Output = ()>>>>::{closure#0}, core[df12db4294e9bfd3]::option::Option<()>>::{closure#0}, (alloc[2182bb758b4b3781]::boxed::Box<tokio[a86046c109545240]::runtime::scheduler::current_thread::Core>, core[df12db4294e9bfd3]::option::Option<()>)>
[INFO] [stdout]   15:     0x615a2d1f696d - <tokio[a86046c109545240]::runtime::scheduler::current_thread::CurrentThread>::block_on::<core[df12db4294e9bfd3]::pin::Pin<&mut dyn core[df12db4294e9bfd3]::future::future::Future<Output = ()>>>
[INFO] [stdout]   16:     0x615a2d1e4a42 - <tokio[a86046c109545240]::runtime::runtime::Runtime>::block_on::<core[df12db4294e9bfd3]::pin::Pin<&mut dyn core[df12db4294e9bfd3]::future::future::Future<Output = ()>>>
[INFO] [stdout]   17:     0x615a2d10f82b - pingora_load_balancing[b161961b48d899d3]::test::test_backends
[INFO] [stdout]   18:     0x615a2d165799 - <pingora_load_balancing[b161961b48d899d3]::test::test_backends::{closure#0} as core[df12db4294e9bfd3]::ops::function::FnOnce<()>>::call_once
[INFO] [stdout]   19:     0x615a2d224f2b - test[9d35eded1c95d3be]::__rust_begin_short_backtrace::<core[df12db4294e9bfd3]::result::Result<(), alloc[2182bb758b4b3781]::string::String>, fn() -> core[df12db4294e9bfd3]::result::Result<(), alloc[2182bb758b4b3781]::string::String>>
[INFO] [stdout]   20:     0x615a2d232595 - test[9d35eded1c95d3be]::run_test::{closure#0}
[INFO] [stdout]   21:     0x615a2d22be54 - std[70759c8f55707aa4]::sys::backtrace::__rust_begin_short_backtrace::<test[9d35eded1c95d3be]::run_test::{closure#1}, ()>
[INFO] [stdout]   22:     0x615a2d2356f2 - <std[70759c8f55707aa4]::thread::lifecycle::spawn_unchecked<test[9d35eded1c95d3be]::run_test::{closure#1}, ()>::{closure#1} as core[df12db4294e9bfd3]::ops::function::FnOnce<()>>::call_once::{shim:vtable#0}
[INFO] [stdout]   23:     0x615a2d344ec9 - <std[70759c8f55707aa4]::sys::thread::unix::Thread>::new::thread_start
[INFO] [stdout]   24:     0x770fc8243dfa - <unknown>
[INFO] [stdout]   25:     0x770fc82d73d4 - clone
[INFO] [stdout]   26:                0x0 - <unknown>
[INFO] [stdout] 
[INFO] [stdout] ---- test::test_parallel_health_check stdout ----
[INFO] [stdout] 
[INFO] [stdout] thread 'test::test_parallel_health_check' (2403) panicked at src/lib.rs:2460:9:
[INFO] [stdout] assertion failed: backends.ready(&good1)
[INFO] [stdout] stack backtrace:
[INFO] [stdout]    0:     0x615a2d345dd1 - <<std[70759c8f55707aa4]::sys::backtrace::BacktraceLock>::print::DisplayBacktrace as core[df12db4294e9bfd3]::fmt::Display>::fmt
[INFO] [stdout]    1:     0x615a2d35ca8a - core[df12db4294e9bfd3]::fmt::write
[INFO] [stdout]    2:     0x615a2d34adfc - <alloc[2182bb758b4b3781]::vec::Vec<u8> as core[df12db4294e9bfd3]::io::write::Write>::write_fmt
[INFO] [stdout]    3:     0x615a2d320466 - std[70759c8f55707aa4]::panicking::default_hook::{closure#0}
[INFO] [stdout]    4:     0x615a2d33d819 - std[70759c8f55707aa4]::panicking::default_hook
[INFO] [stdout]    5:     0x615a2d231c40 - test[9d35eded1c95d3be]::test_main_inner::<test[9d35eded1c95d3be]::test_main_static::{closure#0}>::{closure#0}
[INFO] [stdout]    6:     0x615a2d33db42 - std[70759c8f55707aa4]::panicking::panic_with_hook
[INFO] [stdout]    7:     0x615a2d320544 - std[70759c8f55707aa4]::panicking::panic_handler::{closure#0}
[INFO] [stdout]    8:     0x615a2d3191e9 - std[70759c8f55707aa4]::sys::backtrace::__rust_end_short_backtrace::<std[70759c8f55707aa4]::panicking::panic_handler::{closure#0}, !>
[INFO] [stdout]    9:     0x615a2d3212fd - __rustc[8fa7c3cbc660c2b3]::rust_begin_unwind
[INFO] [stdout]   10:     0x615a2d35d1ec - core[df12db4294e9bfd3]::panicking::panic_fmt
[INFO] [stdout]   11:     0x615a2d35d1b2 - core[df12db4294e9bfd3]::panicking::panic
[INFO] [stdout]   12:     0x615a2d0d8111 - pingora_load_balancing[b161961b48d899d3]::test::test_parallel_health_check::{closure#0}
[INFO] [stdout]   13:     0x615a2d1f70d3 - <tokio[a86046c109545240]::runtime::scheduler::current_thread::Context>::enter::<core[df12db4294e9bfd3]::task::poll::Poll<()>, <tokio[a86046c109545240]::runtime::scheduler::current_thread::CoreGuard>::block_on<core[df12db4294e9bfd3]::pin::Pin<&mut core[df12db4294e9bfd3]::pin::Pin<&mut dyn core[df12db4294e9bfd3]::future::future::Future<Output = ()>>>>::{closure#0}::{closure#0}>
[INFO] [stdout]   14:     0x615a2d1c4cfc - <tokio[a86046c109545240]::runtime::context::scoped::Scoped<tokio[a86046c109545240]::runtime::scheduler::Context>>::set::<<tokio[a86046c109545240]::runtime::scheduler::current_thread::CoreGuard>::enter<<tokio[a86046c109545240]::runtime::scheduler::current_thread::CoreGuard>::block_on<core[df12db4294e9bfd3]::pin::Pin<&mut core[df12db4294e9bfd3]::pin::Pin<&mut dyn core[df12db4294e9bfd3]::future::future::Future<Output = ()>>>>::{closure#0}, core[df12db4294e9bfd3]::option::Option<()>>::{closure#0}, (alloc[2182bb758b4b3781]::boxed::Box<tokio[a86046c109545240]::runtime::scheduler::current_thread::Core>, core[df12db4294e9bfd3]::option::Option<()>)>
[INFO] [stdout]   15:     0x615a2d1f696d - <tokio[a86046c109545240]::runtime::scheduler::current_thread::CurrentThread>::block_on::<core[df12db4294e9bfd3]::pin::Pin<&mut dyn core[df12db4294e9bfd3]::future::future::Future<Output = ()>>>
[INFO] [stdout]   16:     0x615a2d1e4a42 - <tokio[a86046c109545240]::runtime::runtime::Runtime>::block_on::<core[df12db4294e9bfd3]::pin::Pin<&mut dyn core[df12db4294e9bfd3]::future::future::Future<Output = ()>>>
[INFO] [stdout]   17:     0x615a2d10fd4b - pingora_load_balancing[b161961b48d899d3]::test::test_parallel_health_check
[INFO] [stdout]   18:     0x615a2d165819 - <pingora_load_balancing[b161961b48d899d3]::test::test_parallel_health_check::{closure#0} as core[df12db4294e9bfd3]::ops::function::FnOnce<()>>::call_once
[INFO] [stdout]   19:     0x615a2d224f2b - test[9d35eded1c95d3be]::__rust_begin_short_backtrace::<core[df12db4294e9bfd3]::result::Result<(), alloc[2182bb758b4b3781]::string::String>, fn() -> core[df12db4294e9bfd3]::result::Result<(), alloc[2182bb758b4b3781]::string::String>>
[INFO] [stdout]   20:     0x615a2d232595 - test[9d35eded1c95d3be]::run_test::{closure#0}
[INFO] [stdout]   21:     0x615a2d22be54 - std[70759c8f55707aa4]::sys::backtrace::__rust_begin_short_backtrace::<test[9d35eded1c95d3be]::run_test::{closure#1}, ()>
[INFO] [stdout]   22:     0x615a2d2356f2 - <std[70759c8f55707aa4]::thread::lifecycle::spawn_unchecked<test[9d35eded1c95d3be]::run_test::{closure#1}, ()>::{closure#1} as core[df12db4294e9bfd3]::ops::function::FnOnce<()>>::call_once::{shim:vtable#0}
[INFO] [stdout]   23:     0x615a2d344ec9 - <std[70759c8f55707aa4]::sys::thread::unix::Thread>::new::thread_start
[INFO] [stdout]   24:     0x770fc8243dfa - <unknown>
[INFO] [stdout]   25:     0x770fc82d73d4 - clone
[INFO] [stdout]   26:                0x0 - <unknown>
[INFO] [stdout] 
[INFO] [stdout] 
[INFO] [stdout] failures:
[INFO] [stdout]     health_check::test::test_http_custom_check
[INFO] [stdout]     health_check::test::test_tcp_check
[INFO] [stdout]     test::test_backends
[INFO] [stdout]     test::test_parallel_health_check
[INFO] [stdout] 
[INFO] [stdout] test result: FAILED. 74 passed; 4 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.36s
[INFO] [stdout] 
[INFO] [stderr] error: test failed, to rerun pass `--lib`
[INFO] running `Command { std: "docker" "inspect" "4197699ac10e1e81658b74c592bcadc9d17357847a8272068c2a4d30a4411de5", kill_on_drop: false }`
[INFO] running `Command { std: "docker" "rm" "-f" "4197699ac10e1e81658b74c592bcadc9d17357847a8272068c2a4d30a4411de5", kill_on_drop: false }`
[INFO] [stdout] 4197699ac10e1e81658b74c592bcadc9d17357847a8272068c2a4d30a4411de5
