[INFO] fetching crate aurum_actors 0.0.1...
[INFO] testing aurum_actors-0.0.1 against beta-2021-08-01 for beta-1.55-1
[INFO] extracting crate aurum_actors 0.0.1 into /workspace/builds/worker-1/source
[INFO] validating manifest of crates.io crate aurum_actors 0.0.1 on toolchain beta-2021-08-01
[INFO] running `Command { std: "/workspace/cargo-home/bin/cargo" "+beta-2021-08-01" "metadata" "--manifest-path" "Cargo.toml" "--no-deps", kill_on_drop: false }`
[INFO] started tweaking crates.io crate aurum_actors 0.0.1
[INFO] finished tweaking crates.io crate aurum_actors 0.0.1
[INFO] tweaked toml for crates.io crate aurum_actors 0.0.1 written to /workspace/builds/worker-1/source/Cargo.toml
[INFO] running `Command { std: "/workspace/cargo-home/bin/cargo" "+beta-2021-08-01" "generate-lockfile" "--manifest-path" "Cargo.toml" "-Zno-index-update", kill_on_drop: false }`
[INFO] running `Command { std: "/workspace/cargo-home/bin/cargo" "+beta-2021-08-01" "fetch" "--manifest-path" "Cargo.toml", kill_on_drop: false }`
[INFO] [stderr]  Downloading crates ...
[INFO] [stderr]   Downloaded validator v0.14.0
[INFO] [stderr]   Downloaded validator_types v0.14.0
[INFO] [stderr]   Downloaded rpds v0.9.0
[INFO] [stderr]   Downloaded archery v0.4.0
[INFO] [stderr]   Downloaded wyhash v0.5.0
[INFO] [stderr]   Downloaded aurum_actors_macros v0.0.1
[INFO] running `Command { std: "docker" "create" "-v" "/var/lib/crater-agent-workspace/builds/worker-1/target:/opt/rustwide/target:rw,Z" "-v" "/var/lib/crater-agent-workspace/builds/worker-1/source:/opt/rustwide/workdir:ro,Z" "-v" "/var/lib/crater-agent-workspace/cargo-home:/opt/rustwide/cargo-home:ro,Z" "-v" "/var/lib/crater-agent-workspace/rustup-home:/opt/rustwide/rustup-home:ro,Z" "-e" "SOURCE_DIR=/opt/rustwide/workdir" "-e" "CARGO_TARGET_DIR=/opt/rustwide/target" "-e" "CARGO_HOME=/opt/rustwide/cargo-home" "-e" "RUSTUP_HOME=/opt/rustwide/rustup-home" "-w" "/opt/rustwide/workdir" "-m" "1610612736" "--user" "0:0" "--network" "none" "ghcr.io/rust-lang/crates-build-env/linux@sha256:34b6a614d2c27851fe6cbf88fbd1137609cefab8b10d0615aaeb6fd47975d74e" "/opt/rustwide/cargo-home/bin/cargo" "+beta-2021-08-01" "metadata" "--no-deps" "--format-version=1", kill_on_drop: false }`
[INFO] [stdout] 886652cf788e6f4809d75ae88ab9130cb31b3aecc88f73ee0b9cc35c8544212d
[INFO] [stderr] WARNING: Your kernel does not support swap limit capabilities or the cgroup is not mounted. Memory limited without swap.
[INFO] running `Command { std: "docker" "start" "-a" "886652cf788e6f4809d75ae88ab9130cb31b3aecc88f73ee0b9cc35c8544212d", kill_on_drop: false }`
[INFO] running `Command { std: "docker" "inspect" "886652cf788e6f4809d75ae88ab9130cb31b3aecc88f73ee0b9cc35c8544212d", kill_on_drop: false }`
[INFO] running `Command { std: "docker" "rm" "-f" "886652cf788e6f4809d75ae88ab9130cb31b3aecc88f73ee0b9cc35c8544212d", kill_on_drop: false }`
[INFO] [stdout] 886652cf788e6f4809d75ae88ab9130cb31b3aecc88f73ee0b9cc35c8544212d
[INFO] running `Command { std: "docker" "create" "-v" "/var/lib/crater-agent-workspace/builds/worker-1/target:/opt/rustwide/target:rw,Z" "-v" "/var/lib/crater-agent-workspace/builds/worker-1/source:/opt/rustwide/workdir:ro,Z" "-v" "/var/lib/crater-agent-workspace/cargo-home:/opt/rustwide/cargo-home:ro,Z" "-v" "/var/lib/crater-agent-workspace/rustup-home:/opt/rustwide/rustup-home:ro,Z" "-e" "SOURCE_DIR=/opt/rustwide/workdir" "-e" "CARGO_TARGET_DIR=/opt/rustwide/target" "-e" "CARGO_INCREMENTAL=0" "-e" "RUST_BACKTRACE=full" "-e" "RUSTFLAGS=--cap-lints=warn" "-e" "CARGO_HOME=/opt/rustwide/cargo-home" "-e" "RUSTUP_HOME=/opt/rustwide/rustup-home" "-w" "/opt/rustwide/workdir" "-m" "1610612736" "--user" "0:0" "--network" "none" "ghcr.io/rust-lang/crates-build-env/linux@sha256:34b6a614d2c27851fe6cbf88fbd1137609cefab8b10d0615aaeb6fd47975d74e" "/opt/rustwide/cargo-home/bin/cargo" "+beta-2021-08-01" "build" "--frozen" "--message-format=json", kill_on_drop: false }`
[INFO] [stdout] 07f3386c6c14cf688e2c56ecd0ff96a41e6a939dd464af7fd4f4d7d2d949ef4b
[INFO] [stderr] WARNING: Your kernel does not support swap limit capabilities or the cgroup is not mounted. Memory limited without swap.
[INFO] running `Command { std: "docker" "start" "-a" "07f3386c6c14cf688e2c56ecd0ff96a41e6a939dd464af7fd4f4d7d2d949ef4b", kill_on_drop: false }`
[INFO] [stderr]    Compiling serde v1.0.127
[INFO] [stderr]    Compiling once_cell v1.8.0
[INFO] [stderr]    Compiling num-complex v0.2.4
[INFO] [stderr]    Compiling tokio v1.9.0
[INFO] [stderr]    Compiling num-rational v0.2.4
[INFO] [stderr]    Compiling ahash v0.7.4
[INFO] [stderr]    Compiling im v15.0.0
[INFO] [stderr]    Compiling archery v0.4.0
[INFO] [stderr]    Compiling mio v0.7.13
[INFO] [stderr]    Compiling generic-array v0.13.3
[INFO] [stderr]    Compiling bitmaps v2.1.0
[INFO] [stderr]    Compiling approx v0.3.2
[INFO] [stderr]    Compiling num-integer v0.1.44
[INFO] [stderr]    Compiling tokio-macros v1.3.0
[INFO] [stderr]    Compiling async-stream-impl v0.3.2
[INFO] [stderr]    Compiling derivative v2.2.0
[INFO] [stderr]    Compiling validator_types v0.14.0
[INFO] [stderr]    Compiling rand_xoshiro v0.4.0
[INFO] [stderr]    Compiling aurum_actors_macros v0.0.1
[INFO] [stderr]    Compiling wyhash v0.5.0
[INFO] [stderr]    Compiling rand v0.7.3
[INFO] [stderr]    Compiling rand v0.8.4
[INFO] [stderr]    Compiling hashbrown v0.11.2
[INFO] [stderr]    Compiling rand_distr v0.2.2
[INFO] [stderr]    Compiling alga v0.9.3
[INFO] [stderr]    Compiling async-stream v0.3.2
[INFO] [stderr]    Compiling sized-chunks v0.6.5
[INFO] [stderr]    Compiling nalgebra v0.19.0
[INFO] [stderr]    Compiling tokio-stream v0.1.7
[INFO] [stderr]    Compiling toml v0.5.8
[INFO] [stderr]    Compiling serde_json v1.0.66
[INFO] [stderr]    Compiling rpds v0.9.0
[INFO] [stderr]    Compiling serde_cbor v0.11.1
[INFO] [stderr]    Compiling tokio-test v0.4.2
[INFO] [stderr]    Compiling proc-macro-crate v1.0.0
[INFO] [stderr]    Compiling num_enum_derive v0.5.3
[INFO] [stderr]    Compiling validator v0.14.0
[INFO] [stderr]    Compiling num_enum v0.5.3
[INFO] [stderr]    Compiling statrs v0.13.0
[INFO] [stderr]    Compiling aurum_actors v0.0.1 (/opt/rustwide/workdir)
[INFO] [stdout] warning: unused borrow that must be used
[INFO] [stdout]   --> src/core/packets.rs:82:5
[INFO] [stdout]    |
[INFO] [stdout] 82 |     &slice[..DatagramHeader::SIZE].copy_from_slice(&header_buf[..]);
[INFO] [stdout]    |     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[INFO] [stdout]    |
[INFO] [stdout]    = note: `#[warn(unused_must_use)]` on by default
[INFO] [stdout] 
[INFO] [stdout] 
[INFO] [stdout] warning: 1 warning emitted
[INFO] [stdout] 
[INFO] [stdout] 
[INFO] [stderr]     Finished dev [unoptimized + debuginfo] target(s) in 33.68s
[INFO] running `Command { std: "docker" "inspect" "07f3386c6c14cf688e2c56ecd0ff96a41e6a939dd464af7fd4f4d7d2d949ef4b", kill_on_drop: false }`
[INFO] running `Command { std: "docker" "rm" "-f" "07f3386c6c14cf688e2c56ecd0ff96a41e6a939dd464af7fd4f4d7d2d949ef4b", kill_on_drop: false }`
[INFO] [stdout] 07f3386c6c14cf688e2c56ecd0ff96a41e6a939dd464af7fd4f4d7d2d949ef4b
[INFO] running `Command { std: "docker" "create" "-v" "/var/lib/crater-agent-workspace/builds/worker-1/target:/opt/rustwide/target:rw,Z" "-v" "/var/lib/crater-agent-workspace/builds/worker-1/source:/opt/rustwide/workdir:ro,Z" "-v" "/var/lib/crater-agent-workspace/cargo-home:/opt/rustwide/cargo-home:ro,Z" "-v" "/var/lib/crater-agent-workspace/rustup-home:/opt/rustwide/rustup-home:ro,Z" "-e" "SOURCE_DIR=/opt/rustwide/workdir" "-e" "CARGO_TARGET_DIR=/opt/rustwide/target" "-e" "CARGO_INCREMENTAL=0" "-e" "RUST_BACKTRACE=full" "-e" "RUSTFLAGS=--cap-lints=warn" "-e" "CARGO_HOME=/opt/rustwide/cargo-home" "-e" "RUSTUP_HOME=/opt/rustwide/rustup-home" "-w" "/opt/rustwide/workdir" "-m" "1610612736" "--user" "0:0" "--network" "none" "ghcr.io/rust-lang/crates-build-env/linux@sha256:34b6a614d2c27851fe6cbf88fbd1137609cefab8b10d0615aaeb6fd47975d74e" "/opt/rustwide/cargo-home/bin/cargo" "+beta-2021-08-01" "test" "--frozen" "--no-run" "--message-format=json", kill_on_drop: false }`
[INFO] [stdout] d41a8af91355905e34bcf52ace0d8965b4611c1133304ba7790510460604b999
[INFO] [stderr] WARNING: Your kernel does not support swap limit capabilities or the cgroup is not mounted. Memory limited without swap.
[INFO] running `Command { std: "docker" "start" "-a" "d41a8af91355905e34bcf52ace0d8965b4611c1133304ba7790510460604b999", kill_on_drop: false }`
[INFO] [stderr]    Compiling aurum_actors v0.0.1 (/opt/rustwide/workdir)
[INFO] [stdout] warning: unused borrow that must be used
[INFO] [stdout]   --> src/core/packets.rs:82:5
[INFO] [stdout]    |
[INFO] [stdout] 82 |     &slice[..DatagramHeader::SIZE].copy_from_slice(&header_buf[..]);
[INFO] [stdout]    |     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[INFO] [stdout]    |
[INFO] [stdout]    = note: `#[warn(unused_must_use)]` on by default
[INFO] [stdout] 
[INFO] [stdout] 
[INFO] [stdout] warning: 1 warning emitted
[INFO] [stdout] 
[INFO] [stdout] 
[INFO] [stdout] warning: unused borrow that must be used
[INFO] [stdout]   --> src/core/packets.rs:82:5
[INFO] [stdout]    |
[INFO] [stdout] 82 |     &slice[..DatagramHeader::SIZE].copy_from_slice(&header_buf[..]);
[INFO] [stdout]    |     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[INFO] [stdout]    |
[INFO] [stdout]    = note: `#[warn(unused_must_use)]` on by default
[INFO] [stdout] 
[INFO] [stdout] 
[INFO] [stdout] warning: 1 warning emitted
[INFO] [stdout] 
[INFO] [stdout] 
[INFO] [stderr]     Finished test [unoptimized + debuginfo] target(s) in 53.46s
[INFO] running `Command { std: "docker" "inspect" "d41a8af91355905e34bcf52ace0d8965b4611c1133304ba7790510460604b999", kill_on_drop: false }`
[INFO] running `Command { std: "docker" "rm" "-f" "d41a8af91355905e34bcf52ace0d8965b4611c1133304ba7790510460604b999", kill_on_drop: false }`
[INFO] [stdout] d41a8af91355905e34bcf52ace0d8965b4611c1133304ba7790510460604b999
[INFO] running `Command { std: "docker" "create" "-v" "/var/lib/crater-agent-workspace/builds/worker-1/target:/opt/rustwide/target:rw,Z" "-v" "/var/lib/crater-agent-workspace/builds/worker-1/source:/opt/rustwide/workdir:ro,Z" "-v" "/var/lib/crater-agent-workspace/cargo-home:/opt/rustwide/cargo-home:ro,Z" "-v" "/var/lib/crater-agent-workspace/rustup-home:/opt/rustwide/rustup-home:ro,Z" "-e" "SOURCE_DIR=/opt/rustwide/workdir" "-e" "CARGO_TARGET_DIR=/opt/rustwide/target" "-e" "CARGO_INCREMENTAL=0" "-e" "RUST_BACKTRACE=full" "-e" "RUSTFLAGS=--cap-lints=warn" "-e" "CARGO_HOME=/opt/rustwide/cargo-home" "-e" "RUSTUP_HOME=/opt/rustwide/rustup-home" "-w" "/opt/rustwide/workdir" "-m" "1610612736" "--user" "0:0" "--network" "none" "ghcr.io/rust-lang/crates-build-env/linux@sha256:34b6a614d2c27851fe6cbf88fbd1137609cefab8b10d0615aaeb6fd47975d74e" "/opt/rustwide/cargo-home/bin/cargo" "+beta-2021-08-01" "test" "--frozen", kill_on_drop: false }`
[INFO] [stderr] WARNING: Your kernel does not support swap limit capabilities or the cgroup is not mounted. Memory limited without swap.
[INFO] [stdout] 6c822872320d1173f0c3193d27d4c0c8f1d1a363a26383aed2a2f2508c92fd4c
[INFO] running `Command { std: "docker" "start" "-a" "6c822872320d1173f0c3193d27d4c0c8f1d1a363a26383aed2a2f2508c92fd4c", kill_on_drop: false }`
[INFO] [stderr] warning: unused borrow that must be used
[INFO] [stderr]   --> src/core/packets.rs:82:5
[INFO] [stderr]    |
[INFO] [stderr] 82 |     &slice[..DatagramHeader::SIZE].copy_from_slice(&header_buf[..]);
[INFO] [stderr]    |     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[INFO] [stderr]    |
[INFO] [stderr]    = note: `#[warn(unused_must_use)]` on by default
[INFO] [stderr] 
[INFO] [stderr] warning: `aurum_actors` (lib) generated 1 warning
[INFO] [stderr] warning: `aurum_actors` (lib test) generated 1 warning (1 duplicate)
[INFO] [stderr]     Finished test [unoptimized + debuginfo] target(s) in 0.15s
[INFO] [stderr]      Running unittests (/opt/rustwide/target/debug/deps/aurum_actors-6e0d86ee4bb7c296)
[INFO] [stdout] 
[INFO] [stdout] running 8 tests
[INFO] [stdout] test cluster::devices::devices_state::devices_state_join_test ... ok
[INFO] [stdout] test cluster::devices::devices_state::devices_state_deltas_test ... ok
[INFO] [stdout] test cluster::node_ring::test_node_ring ... ok
[INFO] [stdout] test cluster::gossip::test_gossip_merge ... ok
[INFO] [stdout] test cluster::interval_storage::test_interval_storage ... ok
[INFO] [stdout] test cluster::node_ring::test_not_in_ring ... ok
[INFO] [stdout] test core::actor_ref::actor_ref_valid_test ... ok
[INFO] [stdout] test core::packets::test_datagram_header_serde ... ok
[INFO] [stdout] 
[INFO] [stdout] test result: ok. 8 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.01s
[INFO] [stdout] 
[INFO] [stderr]      Running tests/cluster-test.rs (/opt/rustwide/target/debug/deps/cluster_test-f36f0ba5388115a9)
[INFO] [stdout] 
[INFO] [stdout] running 4 tests
[INFO] [stdout] test cluster_test_cyclic_perfect ... ok
[INFO] [stdout] test cluster_test_perfect ... ok
[INFO] [stdout] test cluster_test_with_failures ... FAILED
[INFO] [stdout] test cluster_test_cyclic_failures ... FAILED
[INFO] [stdout] 
[INFO] [stdout] failures:
[INFO] [stdout] 
[INFO] [stdout] ---- cluster_test_with_failures stdout ----
[INFO] [stdout] Waiting for CONVERGENCE
[INFO] [stdout] 40101 MEMBERS - [40101]
[INFO] [stdout] 40101 CHARGES - []
[INFO] [stdout] 40101 MEMBERS - [40101, 40102]
[INFO] [stdout] 40101 CHARGES - [40102]
[INFO] [stdout] 40102 MEMBERS - [40101, 40102]
[INFO] [stdout] 40102 CHARGES - [40101]
[INFO] [stdout] 40102 MEMBERS - [40101, 40102, 40103]
[INFO] [stdout] 40102 CHARGES - [40101, 40103]
[INFO] [stdout] WARN src/cluster/heartbeat_receiver.rs:151:9 - IP(127.0.0.1):40102|0 - DOWNED charge IP(127.0.0.1):40103|0; after timeout: 200 ms
[INFO] [stdout] 40102 MEMBERS - [40101, 40102]
[INFO] [stdout] 40102 CHARGES - [40101]
[INFO] [stdout] 40101 MEMBERS - [40101, 40102, 40103]
[INFO] [stdout] 40101 CHARGES - [40102, 40103]
[INFO] [stdout] 40103 MEMBERS - [40101, 40102, 40103]
[INFO] [stdout] 40103 CHARGES - [40101, 40102]
[INFO] [stdout] WARN src/cluster/heartbeat_receiver.rs:151:9 - IP(127.0.0.1):40103|0 - DOWNED charge IP(127.0.0.1):40102|0; after timeout: 200 ms
[INFO] [stdout] 40103 MEMBERS - [40101, 40103]
[INFO] [stdout] 40103 CHARGES - [40101]
[INFO] [stdout] 40101 MEMBERS - [40101, 40103]
[INFO] [stdout] 40101 CHARGES - [40103]
[INFO] [stdout] 40102 MEMBERS - [40101, 40102]
[INFO] [stdout] 40102 CHARGES - [40101]
[INFO] [stdout] 40103 MEMBERS - [40101, 40103, 40104]
[INFO] [stdout] 40103 CHARGES - [40101, 40104]
[INFO] [stdout] 40101 MEMBERS - [40101, 40102]
[INFO] [stdout] 40101 CHARGES - [40102]
[INFO] [stdout] 40101 MEMBERS - [40101, 40102, 40104]
[INFO] [stdout] 40101 CHARGES - [40102, 40104]
[INFO] [stdout] 40104 MEMBERS - [40101, 40102, 40104]
[INFO] [stdout] 40104 CHARGES - [40101, 40102]
[INFO] [stdout] 40104 MEMBERS - [40101, 40102, 40104, 40105]
[INFO] [stdout] 40104 CHARGES - [40101, 40105]
[INFO] [stdout] 40101 MEMBERS - [40101, 40102, 40104, 40105]
[INFO] [stdout] 40101 CHARGES - [40102, 40105]
[INFO] [stdout] WARN src/cluster/heartbeat_receiver.rs:151:9 - IP(127.0.0.1):40103|0 - DOWNED charge IP(127.0.0.1):40104|0; after timeout: 200 ms
[INFO] [stdout] 40103 MEMBERS - [40101, 40103]
[INFO] [stdout] 40103 CHARGES - [40101]
[INFO] [stdout] 40105 MEMBERS - [40101, 40102, 40104, 40105]
[INFO] [stdout] 40105 CHARGES - [40102, 40104]
[INFO] [stdout] 40101 MEMBERS - [40101, 40102, 40105]
[INFO] [stdout] 40101 CHARGES - [40102, 40105]
[INFO] [stdout] 40104 MEMBERS - [40101, 40102, 40104, 40105]
[INFO] [stdout] 40104 CHARGES - [40102, 40105]
[INFO] [stdout] 40105 MEMBERS - [40101, 40102, 40104, 40105]
[INFO] [stdout] 40105 CHARGES - [40101, 40102]
[INFO] [stdout] 40102 MEMBERS - [40101, 40102, 40104, 40105]
[INFO] [stdout] 40102 CHARGES - [40101, 40104]
[INFO] [stdout] WARN src/cluster/heartbeat_receiver.rs:151:9 - IP(127.0.0.1):40103|0 - DOWNED charge IP(127.0.0.1):40101|0; after timeout: 200 ms
[INFO] [stdout] 40103 MEMBERS - [40103]
[INFO] [stdout] 40103 CHARGES - []
[INFO] [stdout] 40101 MEMBERS - [40101, 40102, 40105]
[INFO] [stdout] 40101 CHARGES - [40102, 40105]
[INFO] [stdout] 40105 MEMBERS - [40101, 40102, 40104, 40105]
[INFO] [stdout] 40105 CHARGES - [40101, 40102]
[INFO] [stdout] WARN src/cluster/heartbeat_receiver.rs:151:9 - IP(127.0.0.1):40101|0 - DOWNED charge IP(127.0.0.1):40102|0; after timeout: 200 ms
[INFO] [stdout] 40101 MEMBERS - [40101, 40105]
[INFO] [stdout] 40101 CHARGES - [40105]
[INFO] [stdout] 40104 MEMBERS - [40101, 40102, 40104, 40105]
[INFO] [stdout] 40104 CHARGES - [40101, 40105]
[INFO] [stdout] 40102 MEMBERS - [40101, 40102, 40104, 40105]
[INFO] [stdout] 40102 CHARGES - [40101, 40104]
[INFO] [stdout] 40105 MEMBERS - [40101, 40104, 40105]
[INFO] [stdout] 40105 CHARGES - [40101, 40104]
[INFO] [stdout] 40101 MEMBERS - [40101, 40104, 40105]
[INFO] [stdout] 40101 CHARGES - [40104, 40105]
[INFO] [stdout] 40101 MEMBERS - [40101, 40102, 40104, 40105]
[INFO] [stdout] 40101 CHARGES - [40104, 40105]
[INFO] [stdout] 40105 MEMBERS - [40101, 40102, 40104, 40105]
[INFO] [stdout] 40105 CHARGES - [40101, 40102]
[INFO] [stdout] 40104 MEMBERS - [40101, 40102, 40104, 40105]
[INFO] [stdout] 40104 CHARGES - [40102, 40105]
[INFO] [stdout] thread 'cluster_test_with_failures' panicked at 'called `Result::unwrap()` on an `Err` value: Elapsed(())', tests/cluster-test.rs:241:77
[INFO] [stdout] stack backtrace:
[INFO] [stdout]    0:     0x55f164c0b5e0 - std::backtrace_rs::backtrace::libunwind::trace::hff4d1766e93f64f1
[INFO] [stdout]                                at /rustc/53fd98ca776cb875bc9e5514f56b52eb74f9e7a9/library/std/src/../../backtrace/src/backtrace/libunwind.rs:90:5
[INFO] [stdout]    1:     0x55f164c0b5e0 - std::backtrace_rs::backtrace::trace_unsynchronized::h56bbc8eb78770bcb
[INFO] [stdout]                                at /rustc/53fd98ca776cb875bc9e5514f56b52eb74f9e7a9/library/std/src/../../backtrace/src/backtrace/mod.rs:66:5
[INFO] [stdout]    2:     0x55f164c0b5e0 - std::sys_common::backtrace::_print_fmt::h5c8e75464faca780
[INFO] [stdout]                                at /rustc/53fd98ca776cb875bc9e5514f56b52eb74f9e7a9/library/std/src/sys_common/backtrace.rs:67:5
[INFO] [stdout]    3:     0x55f164c0b5e0 - <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt::h48b88277def8b362
[INFO] [stdout]                                at /rustc/53fd98ca776cb875bc9e5514f56b52eb74f9e7a9/library/std/src/sys_common/backtrace.rs:46:22
[INFO] [stdout]    4:     0x55f164c2ecfc - core::fmt::write::h3381b3176fe2b3d3
[INFO] [stdout]                                at /rustc/53fd98ca776cb875bc9e5514f56b52eb74f9e7a9/library/core/src/fmt/mod.rs:1115:17
[INFO] [stdout]    5:     0x55f164c05a55 - std::io::Write::write_fmt::h5f34a676f60246b5
[INFO] [stdout]                                at /rustc/53fd98ca776cb875bc9e5514f56b52eb74f9e7a9/library/std/src/io/mod.rs:1665:15
[INFO] [stdout]    6:     0x55f164c0d7cb - std::sys_common::backtrace::_print::ha537687d6c17dd0d
[INFO] [stdout]                                at /rustc/53fd98ca776cb875bc9e5514f56b52eb74f9e7a9/library/std/src/sys_common/backtrace.rs:49:5
[INFO] [stdout]    7:     0x55f164c0d7cb - std::sys_common::backtrace::print::h61376ae57d131d72
[INFO] [stdout]                                at /rustc/53fd98ca776cb875bc9e5514f56b52eb74f9e7a9/library/std/src/sys_common/backtrace.rs:36:9
[INFO] [stdout]    8:     0x55f164c0d7cb - std::panicking::default_hook::{{closure}}::h0f7208c0b9644a54
[INFO] [stdout]                                at /rustc/53fd98ca776cb875bc9e5514f56b52eb74f9e7a9/library/std/src/panicking.rs:208:50
[INFO] [stdout]    9:     0x55f164c0d2dc - std::panicking::default_hook::hdac7fc06578c14ce
[INFO] [stdout]                                at /rustc/53fd98ca776cb875bc9e5514f56b52eb74f9e7a9/library/std/src/panicking.rs:222:9
[INFO] [stdout]   10:     0x55f164c0de94 - std::panicking::rust_panic_with_hook::h72b21c283da3d1ef
[INFO] [stdout]                                at /rustc/53fd98ca776cb875bc9e5514f56b52eb74f9e7a9/library/std/src/panicking.rs:622:17
[INFO] [stdout]   11:     0x55f164c0d977 - std::panicking::begin_panic_handler::{{closure}}::he5778f1729cff84a
[INFO] [stdout]                                at /rustc/53fd98ca776cb875bc9e5514f56b52eb74f9e7a9/library/std/src/panicking.rs:519:13
[INFO] [stdout]   12:     0x55f164c0babc - std::sys_common::backtrace::__rust_end_short_backtrace::h2d11ca8b453b4b1d
[INFO] [stdout]                                at /rustc/53fd98ca776cb875bc9e5514f56b52eb74f9e7a9/library/std/src/sys_common/backtrace.rs:141:18
[INFO] [stdout]   13:     0x55f164c0d8d9 - rust_begin_unwind
[INFO] [stdout]                                at /rustc/53fd98ca776cb875bc9e5514f56b52eb74f9e7a9/library/std/src/panicking.rs:515:5
[INFO] [stdout]   14:     0x55f1647ab9a1 - core::panicking::panic_fmt::hf9de31d4acbce4db
[INFO] [stdout]                                at /rustc/53fd98ca776cb875bc9e5514f56b52eb74f9e7a9/library/core/src/panicking.rs:92:14
[INFO] [stdout]   15:     0x55f1647aba93 - core::result::unwrap_failed::ha0c84e81219e5878
[INFO] [stdout]                                at /rustc/53fd98ca776cb875bc9e5514f56b52eb74f9e7a9/library/core/src/result.rs:1599:5
[INFO] [stdout]   16:     0x55f164950092 - core::result::Result<T,E>::unwrap::hbb9234d743ff2cf8
[INFO] [stdout]                                at /rustc/53fd98ca776cb875bc9e5514f56b52eb74f9e7a9/library/core/src/result.rs:1281:23
[INFO] [stdout]   17:     0x55f164a3eeec - cluster_test::run_cluster_test::{{closure}}::h42d4c4b69e31e5f6
[INFO] [stdout]                                at /opt/rustwide/workdir/tests/cluster-test.rs:241:30
[INFO] [stdout]   18:     0x55f164993e59 - <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll::h61cb0a49cfadd9fc
[INFO] [stdout]                                at /rustc/53fd98ca776cb875bc9e5514f56b52eb74f9e7a9/library/core/src/future/mod.rs:80:19
[INFO] [stdout]   19:     0x55f1649af780 - tokio::park::thread::CachedParkThread::block_on::{{closure}}::haa9f85167ab228ee
[INFO] [stdout]                                at /opt/rustwide/cargo-home/registry/src/github.com-1ecc6299db9ec823/tokio-1.9.0/src/park/thread.rs:263:54
[INFO] [stdout]   20:     0x55f1649522bc - tokio::coop::with_budget::{{closure}}::hd7ab26312c5c97ef
[INFO] [stdout]                                at /opt/rustwide/cargo-home/registry/src/github.com-1ecc6299db9ec823/tokio-1.9.0/src/coop.rs:106:9
[INFO] [stdout]   21:     0x55f164931da1 - std::thread::local::LocalKey<T>::try_with::h19c7f21e475a6979
[INFO] [stdout]                                at /rustc/53fd98ca776cb875bc9e5514f56b52eb74f9e7a9/library/std/src/thread/local.rs:399:16
[INFO] [stdout]   22:     0x55f164931c0d - std::thread::local::LocalKey<T>::with::hc02ebd2ff9a57747
[INFO] [stdout]                                at /rustc/53fd98ca776cb875bc9e5514f56b52eb74f9e7a9/library/std/src/thread/local.rs:375:9
[INFO] [stdout]   23:     0x55f1649af074 - tokio::coop::with_budget::hc76536276e45e5fd
[INFO] [stdout]                                at /opt/rustwide/cargo-home/registry/src/github.com-1ecc6299db9ec823/tokio-1.9.0/src/coop.rs:99:5
[INFO] [stdout]   24:     0x55f1649af074 - tokio::coop::budget::hba1f55163c57a64e
[INFO] [stdout]                                at /opt/rustwide/cargo-home/registry/src/github.com-1ecc6299db9ec823/tokio-1.9.0/src/coop.rs:76:5
[INFO] [stdout]   25:     0x55f1649af074 - tokio::park::thread::CachedParkThread::block_on::ha4e6b449189a069b
[INFO] [stdout]                                at /opt/rustwide/cargo-home/registry/src/github.com-1ecc6299db9ec823/tokio-1.9.0/src/park/thread.rs:263:31
[INFO] [stdout]   26:     0x55f16495265e - tokio::runtime::enter::Enter::block_on::hf927fd3f24b24da6
[INFO] [stdout]                                at /opt/rustwide/cargo-home/registry/src/github.com-1ecc6299db9ec823/tokio-1.9.0/src/runtime/enter.rs:151:13
[INFO] [stdout]   27:     0x55f1649b8de3 - tokio::runtime::thread_pool::ThreadPool::block_on::hdd9ad7e7826e3279
[INFO] [stdout]                                at /opt/rustwide/cargo-home/registry/src/github.com-1ecc6299db9ec823/tokio-1.9.0/src/runtime/thread_pool/mod.rs:71:9
[INFO] [stdout]   28:     0x55f1649b9e4a - tokio::runtime::Runtime::block_on::h577e04b19a95bbc4
[INFO] [stdout]                                at /opt/rustwide/cargo-home/registry/src/github.com-1ecc6299db9ec823/tokio-1.9.0/src/runtime/mod.rs:452:43
[INFO] [stdout]   29:     0x55f164a3eba0 - cluster_test::run_cluster_test::h986988b26ec12b18
[INFO] [stdout]                                at /opt/rustwide/workdir/tests/cluster-test.rs:241:3
[INFO] [stdout]   30:     0x55f164a407ea - cluster_test::cluster_complete::h7f5579c051c5ccf9
[INFO] [stdout]                                at /opt/rustwide/workdir/tests/cluster-test.rs:278:3
[INFO] [stdout]   31:     0x55f164a49989 - cluster_test::cluster_test_with_failures::h580a47359344129c
[INFO] [stdout]                                at /opt/rustwide/workdir/tests/cluster-test.rs:308:3
[INFO] [stdout]   32:     0x55f164a4961a - cluster_test::cluster_test_with_failures::{{closure}}::h37adc12b0ac12ce4
[INFO] [stdout]                                at /opt/rustwide/workdir/tests/cluster-test.rs:296:1
[INFO] [stdout]   33:     0x55f1649cbf6e - core::ops::function::FnOnce::call_once::hbd90ab29ab28722c
[INFO] [stdout]                                at /rustc/53fd98ca776cb875bc9e5514f56b52eb74f9e7a9/library/core/src/ops/function.rs:227:5
[INFO] [stdout]   34:     0x55f164aaa7b3 - core::ops::function::FnOnce::call_once::hbb52c4519d3137b0
[INFO] [stdout]                                at /rustc/53fd98ca776cb875bc9e5514f56b52eb74f9e7a9/library/core/src/ops/function.rs:227:5
[INFO] [stdout]   35:     0x55f164aaa7b3 - test::__rust_begin_short_backtrace::h86fe3a2a5a5121af
[INFO] [stdout]                                at /rustc/53fd98ca776cb875bc9e5514f56b52eb74f9e7a9/library/test/src/lib.rs:578:5
[INFO] [stdout]   36:     0x55f164aa9268 - <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once::hfd956bc3d57399be
[INFO] [stdout]                                at /rustc/53fd98ca776cb875bc9e5514f56b52eb74f9e7a9/library/alloc/src/boxed.rs:1572:9
[INFO] [stdout]   37:     0x55f164aa9268 - <std::panic::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once::h40dc93e0ddb5938a
[INFO] [stdout]                                at /rustc/53fd98ca776cb875bc9e5514f56b52eb74f9e7a9/library/std/src/panic.rs:347:9
[INFO] [stdout]   38:     0x55f164aa9268 - std::panicking::try::do_call::h90d07750cb47c562
[INFO] [stdout]                                at /rustc/53fd98ca776cb875bc9e5514f56b52eb74f9e7a9/library/std/src/panicking.rs:401:40
[INFO] [stdout]   39:     0x55f164aa9268 - std::panicking::try::h063c84733ad2a629
[INFO] [stdout]                                at /rustc/53fd98ca776cb875bc9e5514f56b52eb74f9e7a9/library/std/src/panicking.rs:365:19
[INFO] [stdout]   40:     0x55f164aa9268 - std::panic::catch_unwind::h0c464836a84e7c08
[INFO] [stdout]                                at /rustc/53fd98ca776cb875bc9e5514f56b52eb74f9e7a9/library/std/src/panic.rs:434:14
[INFO] [stdout]   41:     0x55f164aa9268 - test::run_test_in_process::h5cb06ef620b157fc
[INFO] [stdout]                                at /rustc/53fd98ca776cb875bc9e5514f56b52eb74f9e7a9/library/test/src/lib.rs:601:18
[INFO] [stdout]   42:     0x55f164aa9268 - test::run_test::run_test_inner::{{closure}}::h76f677f4b93f9828
[INFO] [stdout]                                at /rustc/53fd98ca776cb875bc9e5514f56b52eb74f9e7a9/library/test/src/lib.rs:493:39
[INFO] [stdout]   43:     0x55f164a779fd - test::run_test::run_test_inner::{{closure}}::h366d01fe04f4d67a
[INFO] [stdout]                                at /rustc/53fd98ca776cb875bc9e5514f56b52eb74f9e7a9/library/test/src/lib.rs:520:37
[INFO] [stdout]   44:     0x55f164a779fd - std::sys_common::backtrace::__rust_begin_short_backtrace::h7dfa0de4eb395af2
[INFO] [stdout]                                at /rustc/53fd98ca776cb875bc9e5514f56b52eb74f9e7a9/library/std/src/sys_common/backtrace.rs:125:18
[INFO] [stdout]   45:     0x55f164a7c1c8 - std::thread::Builder::spawn_unchecked::{{closure}}::{{closure}}::hf0a74bb58798037f
[INFO] [stdout]                                at /rustc/53fd98ca776cb875bc9e5514f56b52eb74f9e7a9/library/std/src/thread/mod.rs:476:17
[INFO] [stdout]   46:     0x55f164a7c1c8 - <std::panic::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once::hb024e854818586ac
[INFO] [stdout]                                at /rustc/53fd98ca776cb875bc9e5514f56b52eb74f9e7a9/library/std/src/panic.rs:347:9
[INFO] [stdout]   47:     0x55f164a7c1c8 - std::panicking::try::do_call::hb2dd9f1d0d48e41d
[INFO] [stdout]                                at /rustc/53fd98ca776cb875bc9e5514f56b52eb74f9e7a9/library/std/src/panicking.rs:401:40
[INFO] [stdout]   48:     0x55f164a7c1c8 - std::panicking::try::h4717ae7bac8b2bf3
[INFO] [stdout]                                at /rustc/53fd98ca776cb875bc9e5514f56b52eb74f9e7a9/library/std/src/panicking.rs:365:19
[INFO] [stdout]   49:     0x55f164a7c1c8 - std::panic::catch_unwind::h0dc8a061326ce8d8
[INFO] [stdout]                                at /rustc/53fd98ca776cb875bc9e5514f56b52eb74f9e7a9/library/std/src/panic.rs:434:14
[INFO] [stdout]   50:     0x55f164a7c1c8 - std::thread::Builder::spawn_unchecked::{{closure}}::haa369e2d6fd00aa1
[INFO] [stdout]                                at /rustc/53fd98ca776cb875bc9e5514f56b52eb74f9e7a9/library/std/src/thread/mod.rs:475:30
[INFO] [stdout]   51:     0x55f164a7c1c8 - core::ops::function::FnOnce::call_once{{vtable.shim}}::h36dcf3de8b056f17
[INFO] [stdout]                                at /rustc/53fd98ca776cb875bc9e5514f56b52eb74f9e7a9/library/core/src/ops/function.rs:227:5
[INFO] [stdout]   52:     0x55f164c13337 - <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once::h65c29e4642be2358
[INFO] [stdout]                                at /rustc/53fd98ca776cb875bc9e5514f56b52eb74f9e7a9/library/alloc/src/boxed.rs:1572:9
[INFO] [stdout]   53:     0x55f164c13337 - <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once::hd0f8f7fbbd9fffbf
[INFO] [stdout]                                at /rustc/53fd98ca776cb875bc9e5514f56b52eb74f9e7a9/library/alloc/src/boxed.rs:1572:9
[INFO] [stdout]   54:     0x55f164c13337 - std::sys::unix::thread::Thread::new::thread_start::h2fd22fee3ca77b65
[INFO] [stdout]                                at /rustc/53fd98ca776cb875bc9e5514f56b52eb74f9e7a9/library/std/src/sys/unix/thread.rs:74:17
[INFO] [stdout]   55:     0x7f0bcb3a0609 - start_thread
[INFO] [stdout]   56:     0x7f0bcb172293 - clone
[INFO] [stdout]   57:                0x0 - <unknown>
[INFO] [stdout] 
[INFO] [stdout] ---- cluster_test_cyclic_failures stdout ----
[INFO] [stdout] Waiting for CONVERGENCE
[INFO] [stdout] 40303 MEMBERS - [40301, 40303]
[INFO] [stdout] 40303 CHARGES - [40301]
[INFO] [stdout] 40302 MEMBERS - [40302, 40303]
[INFO] [stdout] 40302 CHARGES - [40303]
[INFO] [stdout] 40303 MEMBERS - [40301, 40302, 40303]
[INFO] [stdout] 40303 CHARGES - [40301, 40302]
[INFO] [stdout] 40301 MEMBERS - [40301, 40302]
[INFO] [stdout] 40301 CHARGES - [40302]
[INFO] [stdout] 40301 MEMBERS - [40301, 40302, 40303]
[INFO] [stdout] 40301 CHARGES - [40302, 40303]
[INFO] [stdout] 40303 MEMBERS - [40301, 40302, 40303, 40304]
[INFO] [stdout] 40303 CHARGES - [40302, 40304]
[INFO] [stdout] 40302 MEMBERS - [40301, 40302, 40303]
[INFO] [stdout] 40302 CHARGES - [40301, 40303]
[INFO] [stdout] 40302 MEMBERS - [40301, 40302, 40303, 40304]
[INFO] [stdout] 40302 CHARGES - [40301, 40303]
[INFO] [stdout] 40301 MEMBERS - [40301, 40302, 40303, 40304]
[INFO] [stdout] 40301 CHARGES - [40303, 40304]
[INFO] [stdout] 40304 MEMBERS - [40301, 40302, 40303, 40304]
[INFO] [stdout] 40304 CHARGES - [40301, 40302]
[INFO] [stdout] 40304 MEMBERS - [40301, 40302, 40303, 40304, 40305]
[INFO] [stdout] 40304 CHARGES - [40301, 40302]
[INFO] [stdout] 40301 MEMBERS - [40301, 40302, 40303, 40304, 40305]
[INFO] [stdout] 40301 CHARGES - [40303, 40305]
[INFO] [stdout] 40302 MEMBERS - [40301, 40302, 40303, 40304, 40305]
[INFO] [stdout] 40302 CHARGES - [40301, 40303]
[INFO] [stdout] 40305 MEMBERS - [40301, 40302, 40303, 40304, 40305]
[INFO] [stdout] 40305 CHARGES - [40302, 40304]
[INFO] [stdout] WARN src/cluster/heartbeat_receiver.rs:151:9 - IP(127.0.0.1):40303|0 - DOWNED charge IP(127.0.0.1):40302|0; after timeout: 200 ms
[INFO] [stdout] 40303 MEMBERS - [40301, 40303, 40304]
[INFO] [stdout] 40303 CHARGES - [40301, 40304]
[INFO] [stdout] 40301 MEMBERS - [40301, 40303, 40304, 40305]
[INFO] [stdout] 40301 CHARGES - [40303, 40305]
[INFO] [stdout] 40305 MEMBERS - [40301, 40303, 40304, 40305]
[INFO] [stdout] 40305 CHARGES - [40301, 40304]
[INFO] [stdout] 40304 MEMBERS - [40301, 40303, 40304, 40305]
[INFO] [stdout] 40304 CHARGES - [40301, 40303]
[INFO] [stdout] WARN src/cluster/heartbeat_receiver.rs:151:9 - IP(127.0.0.1):40303|0 - DOWNED charge IP(127.0.0.1):40301|0; after timeout: 200 ms
[INFO] [stdout] 40303 MEMBERS - [40303, 40304]
[INFO] [stdout] 40303 CHARGES - [40304]
[INFO] [stdout] 40301 MEMBERS - [40301, 40303, 40304, 40305]
[INFO] [stdout] 40301 CHARGES - [40303, 40305]
[INFO] [stdout] 40304 MEMBERS - [40303, 40304, 40305]
[INFO] [stdout] 40304 CHARGES - [40303, 40305]
[INFO] [stdout] 40303 MEMBERS - [40301, 40303, 40304, 40305]
[INFO] [stdout] 40303 CHARGES - [40304, 40305]
[INFO] [stdout] 40305 MEMBERS - [40303, 40304, 40305]
[INFO] [stdout] 40305 CHARGES - [40303, 40304]
[INFO] [stdout] 40304 MEMBERS - [40301, 40303, 40304, 40305]
[INFO] [stdout] 40304 CHARGES - [40301, 40303]
[INFO] [stdout] 40305 MEMBERS - [40301, 40303, 40304, 40305]
[INFO] [stdout] 40305 CHARGES - [40301, 40304]
[INFO] [stdout] WARN src/cluster/heartbeat_receiver.rs:151:9 - IP(127.0.0.1):40302|0 - DOWNED charge IP(127.0.0.1):40301|0; after timeout: 200 ms
[INFO] [stdout] 40302 MEMBERS - [40302, 40303, 40304, 40305]
[INFO] [stdout] 40302 CHARGES - [40303, 40305]
[INFO] [stdout] WARN src/cluster/heartbeat_receiver.rs:151:9 - IP(127.0.0.1):40302|0 - DOWNED charge IP(127.0.0.1):40305|0; after timeout: 200 ms
[INFO] [stdout] 40302 MEMBERS - [40302, 40303, 40304]
[INFO] [stdout] 40302 CHARGES - [40303, 40304]
[INFO] [stdout] WARN src/cluster/heartbeat_receiver.rs:151:9 - IP(127.0.0.1):40302|0 - DOWNED charge IP(127.0.0.1):40303|0; after timeout: 200 ms
[INFO] [stdout] 40302 MEMBERS - [40302, 40304]
[INFO] [stdout] 40302 CHARGES - [40304]
[INFO] [stdout] 40304 MEMBERS - [40301, 40304]
[INFO] [stdout] 40304 CHARGES - [40301]
[INFO] [stdout] 40303 MEMBERS - [40301, 40303, 40304]
[INFO] [stdout] 40303 CHARGES - [40301, 40304]
[INFO] [stdout] 40305 MEMBERS - [40301, 40304, 40305]
[INFO] [stdout] 40305 CHARGES - [40301, 40304]
[INFO] [stdout] 40303 MEMBERS - [40301, 40303, 40304, 40305]
[INFO] [stdout] 40303 CHARGES - [40301, 40305]
[INFO] [stdout] 40304 MEMBERS - [40301, 40304, 40305]
[INFO] [stdout] 40304 CHARGES - [40301, 40305]
[INFO] [stdout] 40304 MEMBERS - [40301, 40303, 40304, 40305]
[INFO] [stdout] 40304 CHARGES - [40301, 40303]
[INFO] [stdout] 40305 MEMBERS - [40301, 40303, 40304, 40305]
[INFO] [stdout] 40305 CHARGES - [40303, 40304]
[INFO] [stdout] 40301 MEMBERS - [40301, 40303, 40304]
[INFO] [stdout] 40301 CHARGES - [40303, 40304]
[INFO] [stdout] 40301 MEMBERS - [40301, 40303, 40304, 40305]
[INFO] [stdout] 40301 CHARGES - [40304, 40305]
[INFO] [stdout] WARN src/cluster/heartbeat_receiver.rs:151:9 - IP(127.0.0.1):40302|0 - DOWNED charge IP(127.0.0.1):40304|0; after timeout: 200 ms
[INFO] [stdout] 40302 MEMBERS - [40302]
[INFO] [stdout] 40302 CHARGES - []
[INFO] [stdout] thread 'cluster_test_cyclic_failures' panicked at 'called `Result::unwrap()` on an `Err` value: Elapsed(())', tests/cluster-test.rs:241:77
[INFO] [stdout] stack backtrace:
[INFO] [stdout]    0:     0x55f164c0b5e0 - std::backtrace_rs::backtrace::libunwind::trace::hff4d1766e93f64f1
[INFO] [stdout]                                at /rustc/53fd98ca776cb875bc9e5514f56b52eb74f9e7a9/library/std/src/../../backtrace/src/backtrace/libunwind.rs:90:5
[INFO] [stdout]    1:     0x55f164c0b5e0 - std::backtrace_rs::backtrace::trace_unsynchronized::h56bbc8eb78770bcb
[INFO] [stdout]                                at /rustc/53fd98ca776cb875bc9e5514f56b52eb74f9e7a9/library/std/src/../../backtrace/src/backtrace/mod.rs:66:5
[INFO] [stdout]    2:     0x55f164c0b5e0 - std::sys_common::backtrace::_print_fmt::h5c8e75464faca780
[INFO] [stdout]                                at /rustc/53fd98ca776cb875bc9e5514f56b52eb74f9e7a9/library/std/src/sys_common/backtrace.rs:67:5
[INFO] [stdout]    3:     0x55f164c0b5e0 - <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt::h48b88277def8b362
[INFO] [stdout]                                at /rustc/53fd98ca776cb875bc9e5514f56b52eb74f9e7a9/library/std/src/sys_common/backtrace.rs:46:22
[INFO] [stdout]    4:     0x55f164c2ecfc - core::fmt::write::h3381b3176fe2b3d3
[INFO] [stdout]                                at /rustc/53fd98ca776cb875bc9e5514f56b52eb74f9e7a9/library/core/src/fmt/mod.rs:1115:17
[INFO] [stdout]    5:     0x55f164c05a55 - std::io::Write::write_fmt::h5f34a676f60246b5
[INFO] [stdout]                                at /rustc/53fd98ca776cb875bc9e5514f56b52eb74f9e7a9/library/std/src/io/mod.rs:1665:15
[INFO] [stdout]    6:     0x55f164c0d7cb - std::sys_common::backtrace::_print::ha537687d6c17dd0d
[INFO] [stdout]                                at /rustc/53fd98ca776cb875bc9e5514f56b52eb74f9e7a9/library/std/src/sys_common/backtrace.rs:49:5
[INFO] [stdout]    7:     0x55f164c0d7cb - std::sys_common::backtrace::print::h61376ae57d131d72
[INFO] [stdout]                                at /rustc/53fd98ca776cb875bc9e5514f56b52eb74f9e7a9/library/std/src/sys_common/backtrace.rs:36:9
[INFO] [stdout]    8:     0x55f164c0d7cb - std::panicking::default_hook::{{closure}}::h0f7208c0b9644a54
[INFO] [stdout]                                at /rustc/53fd98ca776cb875bc9e5514f56b52eb74f9e7a9/library/std/src/panicking.rs:208:50
[INFO] [stdout]    9:     0x55f164c0d2dc - std::panicking::default_hook::hdac7fc06578c14ce
[INFO] [stdout]                                at /rustc/53fd98ca776cb875bc9e5514f56b52eb74f9e7a9/library/std/src/panicking.rs:222:9
[INFO] [stdout]   10:     0x55f164c0de94 - std::panicking::rust_panic_with_hook::h72b21c283da3d1ef
[INFO] [stdout]                                at /rustc/53fd98ca776cb875bc9e5514f56b52eb74f9e7a9/library/std/src/panicking.rs:622:17
[INFO] [stdout]   11:     0x55f164c0d977 - std::panicking::begin_panic_handler::{{closure}}::he5778f1729cff84a
[INFO] [stdout]                                at /rustc/53fd98ca776cb875bc9e5514f56b52eb74f9e7a9/library/std/src/panicking.rs:519:13
[INFO] [stdout]   12:     0x55f164c0babc - std::sys_common::backtrace::__rust_end_short_backtrace::h2d11ca8b453b4b1d
[INFO] [stdout]                                at /rustc/53fd98ca776cb875bc9e5514f56b52eb74f9e7a9/library/std/src/sys_common/backtrace.rs:141:18
[INFO] [stdout]   13:     0x55f164c0d8d9 - rust_begin_unwind
[INFO] [stdout]                                at /rustc/53fd98ca776cb875bc9e5514f56b52eb74f9e7a9/library/std/src/panicking.rs:515:5
[INFO] [stdout]   14:     0x55f1647ab9a1 - core::panicking::panic_fmt::hf9de31d4acbce4db
[INFO] [stdout]                                at /rustc/53fd98ca776cb875bc9e5514f56b52eb74f9e7a9/library/core/src/panicking.rs:92:14
[INFO] [stdout]   15:     0x55f1647aba93 - core::result::unwrap_failed::ha0c84e81219e5878
[INFO] [stdout]                                at /rustc/53fd98ca776cb875bc9e5514f56b52eb74f9e7a9/library/core/src/result.rs:1599:5
[INFO] [stdout]   16:     0x55f164950092 - core::result::Result<T,E>::unwrap::hbb9234d743ff2cf8
[INFO] [stdout]                                at /rustc/53fd98ca776cb875bc9e5514f56b52eb74f9e7a9/library/core/src/result.rs:1281:23
[INFO] [stdout]   17:     0x55f164a3eeec - cluster_test::run_cluster_test::{{closure}}::h42d4c4b69e31e5f6
[INFO] [stdout]                                at /opt/rustwide/workdir/tests/cluster-test.rs:241:30
[INFO] [stdout]   18:     0x55f164993e59 - <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll::h61cb0a49cfadd9fc
[INFO] [stdout]                                at /rustc/53fd98ca776cb875bc9e5514f56b52eb74f9e7a9/library/core/src/future/mod.rs:80:19
[INFO] [stdout]   19:     0x55f1649af780 - tokio::park::thread::CachedParkThread::block_on::{{closure}}::haa9f85167ab228ee
[INFO] [stdout]                                at /opt/rustwide/cargo-home/registry/src/github.com-1ecc6299db9ec823/tokio-1.9.0/src/park/thread.rs:263:54
[INFO] [stdout]   20:     0x55f1649522bc - tokio::coop::with_budget::{{closure}}::hd7ab26312c5c97ef
[INFO] [stdout]                                at /opt/rustwide/cargo-home/registry/src/github.com-1ecc6299db9ec823/tokio-1.9.0/src/coop.rs:106:9
[INFO] [stdout]   21:     0x55f164931da1 - std::thread::local::LocalKey<T>::try_with::h19c7f21e475a6979
[INFO] [stdout]                                at /rustc/53fd98ca776cb875bc9e5514f56b52eb74f9e7a9/library/std/src/thread/local.rs:399:16
[INFO] [stdout]   22:     0x55f164931c0d - std::thread::local::LocalKey<T>::with::hc02ebd2ff9a57747
[INFO] [stdout]                                at /rustc/53fd98ca776cb875bc9e5514f56b52eb74f9e7a9/library/std/src/thread/local.rs:375:9
[INFO] [stdout]   23:     0x55f1649af074 - tokio::coop::with_budget::hc76536276e45e5fd
[INFO] [stdout]                                at /opt/rustwide/cargo-home/registry/src/github.com-1ecc6299db9ec823/tokio-1.9.0/src/coop.rs:99:5
[INFO] [stdout]   24:     0x55f1649af074 - tokio::coop::budget::hba1f55163c57a64e
[INFO] [stdout]                                at /opt/rustwide/cargo-home/registry/src/github.com-1ecc6299db9ec823/tokio-1.9.0/src/coop.rs:76:5
[INFO] [stdout]   25:     0x55f1649af074 - tokio::park::thread::CachedParkThread::block_on::ha4e6b449189a069b
[INFO] [stdout]                                at /opt/rustwide/cargo-home/registry/src/github.com-1ecc6299db9ec823/tokio-1.9.0/src/park/thread.rs:263:31
[INFO] [stdout]   26:     0x55f16495265e - tokio::runtime::enter::Enter::block_on::hf927fd3f24b24da6
[INFO] [stdout]                                at /opt/rustwide/cargo-home/registry/src/github.com-1ecc6299db9ec823/tokio-1.9.0/src/runtime/enter.rs:151:13
[INFO] [stdout]   27:     0x55f1649b8de3 - tokio::runtime::thread_pool::ThreadPool::block_on::hdd9ad7e7826e3279
[INFO] [stdout]                                at /opt/rustwide/cargo-home/registry/src/github.com-1ecc6299db9ec823/tokio-1.9.0/src/runtime/thread_pool/mod.rs:71:9
[INFO] [stdout]   28:     0x55f1649b9e4a - tokio::runtime::Runtime::block_on::h577e04b19a95bbc4
[INFO] [stdout]                                at /opt/rustwide/cargo-home/registry/src/github.com-1ecc6299db9ec823/tokio-1.9.0/src/runtime/mod.rs:452:43
[INFO] [stdout]   29:     0x55f164a3eba0 - cluster_test::run_cluster_test::h986988b26ec12b18
[INFO] [stdout]                                at /opt/rustwide/workdir/tests/cluster-test.rs:241:3
[INFO] [stdout]   30:     0x55f164a41c9a - cluster_test::cluster_cyclic::h18cb0f0fc217cd4b
[INFO] [stdout]                                at /opt/rustwide/workdir/tests/cluster-test.rs:346:3
[INFO] [stdout]   31:     0x55f164a4a029 - cluster_test::cluster_test_cyclic_failures::hacb1bd8530e14566
[INFO] [stdout]                                at /opt/rustwide/workdir/tests/cluster-test.rs:375:3
[INFO] [stdout]   32:     0x55f164a49cba - cluster_test::cluster_test_cyclic_failures::{{closure}}::hcd4b607cea159ce1
[INFO] [stdout]                                at /opt/rustwide/workdir/tests/cluster-test.rs:363:1
[INFO] [stdout]   33:     0x55f1649ca6ee - core::ops::function::FnOnce::call_once::h17db52beb26a0ffa
[INFO] [stdout]                                at /rustc/53fd98ca776cb875bc9e5514f56b52eb74f9e7a9/library/core/src/ops/function.rs:227:5
[INFO] [stdout]   34:     0x55f164aaa7b3 - core::ops::function::FnOnce::call_once::hbb52c4519d3137b0
[INFO] [stdout]                                at /rustc/53fd98ca776cb875bc9e5514f56b52eb74f9e7a9/library/core/src/ops/function.rs:227:5
[INFO] [stdout]   35:     0x55f164aaa7b3 - test::__rust_begin_short_backtrace::h86fe3a2a5a5121af
[INFO] [stdout]                                at /rustc/53fd98ca776cb875bc9e5514f56b52eb74f9e7a9/library/test/src/lib.rs:578:5
[INFO] [stdout]   36:     0x55f164aa9268 - <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once::hfd956bc3d57399be
[INFO] [stdout]                                at /rustc/53fd98ca776cb875bc9e5514f56b52eb74f9e7a9/library/alloc/src/boxed.rs:1572:9
[INFO] [stdout]   37:     0x55f164aa9268 - <std::panic::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once::h40dc93e0ddb5938a
[INFO] [stdout]                                at /rustc/53fd98ca776cb875bc9e5514f56b52eb74f9e7a9/library/std/src/panic.rs:347:9
[INFO] [stdout]   38:     0x55f164aa9268 - std::panicking::try::do_call::h90d07750cb47c562
[INFO] [stdout]                                at /rustc/53fd98ca776cb875bc9e5514f56b52eb74f9e7a9/library/std/src/panicking.rs:401:40
[INFO] [stdout]   39:     0x55f164aa9268 - std::panicking::try::h063c84733ad2a629
[INFO] [stdout]                                at /rustc/53fd98ca776cb875bc9e5514f56b52eb74f9e7a9/library/std/src/panicking.rs:365:19
[INFO] [stdout]   40:     0x55f164aa9268 - std::panic::catch_unwind::h0c464836a84e7c08
[INFO] [stdout]                                at /rustc/53fd98ca776cb875bc9e5514f56b52eb74f9e7a9/library/std/src/panic.rs:434:14
[INFO] [stdout]   41:     0x55f164aa9268 - test::run_test_in_process::h5cb06ef620b157fc
[INFO] [stdout]                                at /rustc/53fd98ca776cb875bc9e5514f56b52eb74f9e7a9/library/test/src/lib.rs:601:18
[INFO] [stdout]   42:     0x55f164aa9268 - test::run_test::run_test_inner::{{closure}}::h76f677f4b93f9828
[INFO] [stdout]                                at /rustc/53fd98ca776cb875bc9e5514f56b52eb74f9e7a9/library/test/src/lib.rs:493:39
[INFO] [stdout]   43:     0x55f164a779fd - test::run_test::run_test_inner::{{closure}}::h366d01fe04f4d67a
[INFO] [stdout]                                at /rustc/53fd98ca776cb875bc9e5514f56b52eb74f9e7a9/library/test/src/lib.rs:520:37
[INFO] [stdout]   44:     0x55f164a779fd - std::sys_common::backtrace::__rust_begin_short_backtrace::h7dfa0de4eb395af2
[INFO] [stdout]                                at /rustc/53fd98ca776cb875bc9e5514f56b52eb74f9e7a9/library/std/src/sys_common/backtrace.rs:125:18
[INFO] [stdout]   45:     0x55f164a7c1c8 - std::thread::Builder::spawn_unchecked::{{closure}}::{{closure}}::hf0a74bb58798037f
[INFO] [stdout]                                at /rustc/53fd98ca776cb875bc9e5514f56b52eb74f9e7a9/library/std/src/thread/mod.rs:476:17
[INFO] [stdout]   46:     0x55f164a7c1c8 - <std::panic::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once::hb024e854818586ac
[INFO] [stdout]                                at /rustc/53fd98ca776cb875bc9e5514f56b52eb74f9e7a9/library/std/src/panic.rs:347:9
[INFO] [stdout]   47:     0x55f164a7c1c8 - std::panicking::try::do_call::hb2dd9f1d0d48e41d
[INFO] [stdout]                                at /rustc/53fd98ca776cb875bc9e5514f56b52eb74f9e7a9/library/std/src/panicking.rs:401:40
[INFO] [stdout]   48:     0x55f164a7c1c8 - std::panicking::try::h4717ae7bac8b2bf3
[INFO] [stdout]                                at /rustc/53fd98ca776cb875bc9e5514f56b52eb74f9e7a9/library/std/src/panicking.rs:365:19
[INFO] [stdout]   49:     0x55f164a7c1c8 - std::panic::catch_unwind::h0dc8a061326ce8d8
[INFO] [stdout]                                at /rustc/53fd98ca776cb875bc9e5514f56b52eb74f9e7a9/library/std/src/panic.rs:434:14
[INFO] [stdout]   50:     0x55f164a7c1c8 - std::thread::Builder::spawn_unchecked::{{closure}}::haa369e2d6fd00aa1
[INFO] [stdout]                                at /rustc/53fd98ca776cb875bc9e5514f56b52eb74f9e7a9/library/std/src/thread/mod.rs:475:30
[INFO] [stdout]   51:     0x55f164a7c1c8 - core::ops::function::FnOnce::call_once{{vtable.shim}}::h36dcf3de8b056f17
[INFO] [stdout]                                at /rustc/53fd98ca776cb875bc9e5514f56b52eb74f9e7a9/library/core/src/ops/function.rs:227:5
[INFO] [stdout]   52:     0x55f164c13337 - <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once::h65c29e4642be2358
[INFO] [stdout]                                at /rustc/53fd98ca776cb875bc9e5514f56b52eb74f9e7a9/library/alloc/src/boxed.rs:1572:9
[INFO] [stdout]   53:     0x55f164c13337 - <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once::hd0f8f7fbbd9fffbf
[INFO] [stdout]                                at /rustc/53fd98ca776cb875bc9e5514f56b52eb74f9e7a9/library/alloc/src/boxed.rs:1572:9
[INFO] [stdout]   54:     0x55f164c13337 - std::sys::unix::thread::Thread::new::thread_start::h2fd22fee3ca77b65
[INFO] [stdout]                                at /rustc/53fd98ca776cb875bc9e5514f56b52eb74f9e7a9/library/std/src/sys/unix/thread.rs:74:17
[INFO] [stdout]   55:     0x7f0bcb3a0609 - start_thread
[INFO] [stdout]   56:     0x7f0bcb172293 - clone
[INFO] [stdout]   57:                0x0 - <unknown>
[INFO] [stdout] 
[INFO] [stdout] 
[INFO] [stdout] failures:
[INFO] [stdout]     cluster_test_cyclic_failures
[INFO] [stdout]     cluster_test_with_failures
[INFO] [stdout] 
[INFO] [stdout] test result: FAILED. 2 passed; 2 failed; 0 ignored; 0 measured; 0 filtered out; finished in 10.07s
[INFO] [stdout] 
[INFO] [stderr] error: test failed, to rerun pass '--test cluster-test'
[INFO] running `Command { std: "docker" "inspect" "6c822872320d1173f0c3193d27d4c0c8f1d1a363a26383aed2a2f2508c92fd4c", kill_on_drop: false }`
[INFO] running `Command { std: "docker" "rm" "-f" "6c822872320d1173f0c3193d27d4c0c8f1d1a363a26383aed2a2f2508c92fd4c", kill_on_drop: false }`
[INFO] [stdout] 6c822872320d1173f0c3193d27d4c0c8f1d1a363a26383aed2a2f2508c92fd4c
