[INFO] fetching crate async_tasks_recorder 2.0.2...
[INFO] testing async_tasks_recorder-2.0.2 against master#2fd6efc32704647e64d3d646d21c4c68eae100e4 for pr-149852-1
[INFO] extracting crate async_tasks_recorder 2.0.2 into /workspace/builds/worker-1-tc1/source
[INFO] started tweaking crates.io crate async_tasks_recorder 2.0.2
[INFO] finished tweaking crates.io crate async_tasks_recorder 2.0.2
[INFO] tweaked toml for crates.io crate async_tasks_recorder 2.0.2 written to /workspace/builds/worker-1-tc1/source/Cargo.toml
[INFO] validating manifest of crates.io crate async_tasks_recorder 2.0.2 on toolchain 2fd6efc32704647e64d3d646d21c4c68eae100e4
[INFO] running `Command { std: CARGO_HOME="/workspace/cargo-home" RUSTUP_HOME="/workspace/rustup-home" "/workspace/cargo-home/bin/cargo" "+2fd6efc32704647e64d3d646d21c4c68eae100e4" "metadata" "--manifest-path" "Cargo.toml" "--no-deps", kill_on_drop: false }`
[INFO] crate crates.io crate async_tasks_recorder 2.0.2 already has a lockfile, it will not be regenerated
[INFO] running `Command { std: CARGO_HOME="/workspace/cargo-home" RUSTUP_HOME="/workspace/rustup-home" "/workspace/cargo-home/bin/cargo" "+2fd6efc32704647e64d3d646d21c4c68eae100e4" "fetch" "--manifest-path" "Cargo.toml", kill_on_drop: false }`
[INFO] [stderr]     Blocking waiting for file lock on package cache
[INFO] [stderr]     Updating crates.io index
[INFO] [stderr]     Blocking waiting for file lock on package cache
[INFO] [stderr]  Downloading crates ...
[INFO] [stderr]   Downloaded num_cpus v1.16.0
[INFO] [stderr]   Downloaded serial_test_derive v3.0.0
[INFO] [stderr]   Downloaded redox_syscall v0.4.1
[INFO] [stderr]   Downloaded futures-core v0.3.30
[INFO] [stderr]   Downloaded dashmap v5.5.3
[INFO] [stderr]   Downloaded once_cell v1.19.0
[INFO] [stderr]   Downloaded futures-executor v0.3.30
[INFO] [stderr]   Downloaded futures v0.3.30
[INFO] [stderr]   Downloaded cc v1.0.86
[INFO] [stderr]   Downloaded memchr v2.7.1
[INFO] [stderr]   Downloaded proc-macro2 v1.0.78
[INFO] [stderr]   Downloaded scc v2.0.16
[INFO] [stderr]   Downloaded futures-util v0.3.30
[INFO] [stderr]   Downloaded hashbrown v0.14.3
[INFO] [stderr]   Downloaded backtrace v0.3.69
[INFO] [stderr]   Downloaded miniz_oxide v0.7.2
[INFO] [stderr]   Downloaded rustc-demangle v0.1.23
[INFO] [stderr]   Downloaded syn v2.0.50
[INFO] [stderr]   Downloaded futures-channel v0.3.30
[INFO] [stderr]   Downloaded quote v1.0.35
[INFO] [stderr]   Downloaded hermit-abi v0.3.6
[INFO] [stderr]   Downloaded futures-io v0.3.30
[INFO] [stderr]   Downloaded futures-sink v0.3.30
[INFO] [stderr]   Downloaded futures-task v0.3.30
[INFO] [stderr]   Downloaded serial_test v3.0.0
[INFO] [stderr]   Downloaded smallvec v1.13.1
[INFO] [stderr]   Downloaded parking_lot_core v0.9.9
[INFO] [stderr]   Downloaded parking_lot v0.12.1
[INFO] [stderr]   Downloaded scopeguard v1.2.0
[INFO] [stderr]   Downloaded lock_api v0.4.11
[INFO] [stderr]   Downloaded libc v0.2.153
[INFO] [stderr]   Downloaded tokio v1.36.0
[INFO] running `Command { std: "docker" "create" "-v" "/var/lib/crater-agent-workspace/builds/worker-1-tc1/target:/opt/rustwide/target:rw,Z" "-v" "/var/lib/crater-agent-workspace/builds/worker-1-tc1/source:/opt/rustwide/workdir:ro,Z" "-v" "/var/lib/crater-agent-workspace/cargo-home:/opt/rustwide/cargo-home:ro,Z" "-v" "/var/lib/crater-agent-workspace/rustup-home:/opt/rustwide/rustup-home:ro,Z" "-e" "SOURCE_DIR=/opt/rustwide/workdir" "-e" "CARGO_TARGET_DIR=/opt/rustwide/target" "-e" "CARGO_HOME=/opt/rustwide/cargo-home" "-e" "RUSTUP_HOME=/opt/rustwide/rustup-home" "-w" "/opt/rustwide/workdir" "-m" "1610612736" "--user" "0:0" "--network" "none" "ghcr.io/rust-lang/crates-build-env/linux@sha256:aa71247004a7fa38d13ec170f48f06cdedf5bc50b2a8645e56ed7e992e6fa513" "/opt/rustwide/cargo-home/bin/cargo" "+2fd6efc32704647e64d3d646d21c4c68eae100e4" "metadata" "--no-deps" "--format-version=1", kill_on_drop: false }`
[INFO] [stdout] 0e17ac5ac8e4fa2bc000e1d564f005a10b44bca227a60999c20b9575b4fc54f5
[INFO] running `Command { std: "docker" "start" "-a" "0e17ac5ac8e4fa2bc000e1d564f005a10b44bca227a60999c20b9575b4fc54f5", kill_on_drop: false }`
[INFO] running `Command { std: "docker" "inspect" "0e17ac5ac8e4fa2bc000e1d564f005a10b44bca227a60999c20b9575b4fc54f5", kill_on_drop: false }`
[INFO] running `Command { std: "docker" "rm" "-f" "0e17ac5ac8e4fa2bc000e1d564f005a10b44bca227a60999c20b9575b4fc54f5", kill_on_drop: false }`
[INFO] [stdout] 0e17ac5ac8e4fa2bc000e1d564f005a10b44bca227a60999c20b9575b4fc54f5
[INFO] running `Command { std: "docker" "create" "-v" "/var/lib/crater-agent-workspace/builds/worker-1-tc1/target:/opt/rustwide/target:rw,Z" "-v" "/var/lib/crater-agent-workspace/builds/worker-1-tc1/source:/opt/rustwide/workdir:ro,Z" "-v" "/var/lib/crater-agent-workspace/cargo-home:/opt/rustwide/cargo-home:ro,Z" "-v" "/var/lib/crater-agent-workspace/rustup-home:/opt/rustwide/rustup-home:ro,Z" "-e" "SOURCE_DIR=/opt/rustwide/workdir" "-e" "CARGO_TARGET_DIR=/opt/rustwide/target" "-e" "CARGO_INCREMENTAL=0" "-e" "RUST_BACKTRACE=full" "-e" "RUSTFLAGS=--cap-lints=forbid" "-e" "RUSTDOCFLAGS=--cap-lints=forbid" "-e" "CARGO_HOME=/opt/rustwide/cargo-home" "-e" "RUSTUP_HOME=/opt/rustwide/rustup-home" "-w" "/opt/rustwide/workdir" "-m" "1610612736" "--user" "0:0" "--network" "none" "ghcr.io/rust-lang/crates-build-env/linux@sha256:aa71247004a7fa38d13ec170f48f06cdedf5bc50b2a8645e56ed7e992e6fa513" "/opt/rustwide/cargo-home/bin/cargo" "+2fd6efc32704647e64d3d646d21c4c68eae100e4" "build" "--frozen" "--message-format=json", kill_on_drop: false }`
[INFO] [stdout] 20009585a72f912a8db072ac6c77f5cdf5bc933e4f96eb03d644cf02cd7b40c5
[INFO] running `Command { std: "docker" "start" "-a" "20009585a72f912a8db072ac6c77f5cdf5bc933e4f96eb03d644cf02cd7b40c5", kill_on_drop: false }`
[INFO] [stderr]    Compiling pin-project-lite v0.2.13
[INFO] [stderr]    Compiling scc v2.0.16
[INFO] [stderr]    Compiling tokio v1.36.0
[INFO] [stderr]    Compiling async_tasks_recorder v2.0.2 (/opt/rustwide/workdir)
[INFO] [stderr]     Finished `dev` profile [unoptimized + debuginfo] target(s) in 2.14s
[INFO] running `Command { std: "docker" "inspect" "20009585a72f912a8db072ac6c77f5cdf5bc933e4f96eb03d644cf02cd7b40c5", kill_on_drop: false }`
[INFO] running `Command { std: "docker" "rm" "-f" "20009585a72f912a8db072ac6c77f5cdf5bc933e4f96eb03d644cf02cd7b40c5", kill_on_drop: false }`
[INFO] [stdout] 20009585a72f912a8db072ac6c77f5cdf5bc933e4f96eb03d644cf02cd7b40c5
[INFO] running `Command { std: "docker" "create" "-v" "/var/lib/crater-agent-workspace/builds/worker-1-tc1/target:/opt/rustwide/target:rw,Z" "-v" "/var/lib/crater-agent-workspace/builds/worker-1-tc1/source:/opt/rustwide/workdir:ro,Z" "-v" "/var/lib/crater-agent-workspace/cargo-home:/opt/rustwide/cargo-home:ro,Z" "-v" "/var/lib/crater-agent-workspace/rustup-home:/opt/rustwide/rustup-home:ro,Z" "-e" "SOURCE_DIR=/opt/rustwide/workdir" "-e" "CARGO_TARGET_DIR=/opt/rustwide/target" "-e" "CARGO_INCREMENTAL=0" "-e" "RUST_BACKTRACE=full" "-e" "RUSTFLAGS=--cap-lints=forbid" "-e" "RUSTDOCFLAGS=--cap-lints=forbid" "-e" "CARGO_HOME=/opt/rustwide/cargo-home" "-e" "RUSTUP_HOME=/opt/rustwide/rustup-home" "-w" "/opt/rustwide/workdir" "-m" "1610612736" "--user" "0:0" "--network" "none" "ghcr.io/rust-lang/crates-build-env/linux@sha256:aa71247004a7fa38d13ec170f48f06cdedf5bc50b2a8645e56ed7e992e6fa513" "/opt/rustwide/cargo-home/bin/cargo" "+2fd6efc32704647e64d3d646d21c4c68eae100e4" "test" "--frozen" "--no-run" "--message-format=json", kill_on_drop: false }`
[INFO] [stdout] 317dc7d9978caed191677b37a829ae29aaa896ee9409a94562a6015aa580e48d
[INFO] running `Command { std: "docker" "start" "-a" "317dc7d9978caed191677b37a829ae29aaa896ee9409a94562a6015aa580e48d", kill_on_drop: false }`
[INFO] [stderr]    Compiling autocfg v1.1.0
[INFO] [stderr]    Compiling libc v0.2.153
[INFO] [stderr]    Compiling parking_lot_core v0.9.9
[INFO] [stderr]    Compiling smallvec v1.13.1
[INFO] [stderr]    Compiling scopeguard v1.2.0
[INFO] [stderr]    Compiling proc-macro2 v1.0.78
[INFO] [stderr]    Compiling cfg-if v1.0.0
[INFO] [stderr]    Compiling futures-core v0.3.30
[INFO] [stderr]    Compiling futures-sink v0.3.30
[INFO] [stderr]    Compiling unicode-ident v1.0.12
[INFO] [stderr]    Compiling pin-utils v0.1.0
[INFO] [stderr]    Compiling memchr v2.7.1
[INFO] [stderr]    Compiling futures-task v0.3.30
[INFO] [stderr]    Compiling futures-io v0.3.30
[INFO] [stderr]    Compiling hashbrown v0.14.3
[INFO] [stderr]    Compiling once_cell v1.19.0
[INFO] [stderr]    Compiling lazy_static v1.4.0
[INFO] [stderr]    Compiling log v0.4.20
[INFO] [stderr]    Compiling fastrand v2.0.1
[INFO] [stderr]    Compiling futures-channel v0.3.30
[INFO] [stderr]    Compiling lock_api v0.4.11
[INFO] [stderr]    Compiling slab v0.4.9
[INFO] [stderr]    Compiling futures-util v0.3.30
[INFO] [stderr]    Compiling quote v1.0.35
[INFO] [stderr]    Compiling num_cpus v1.16.0
[INFO] [stderr]    Compiling syn v2.0.50
[INFO] [stderr]    Compiling parking_lot v0.12.1
[INFO] [stderr]    Compiling dashmap v5.5.3
[INFO] [stderr]    Compiling tokio v1.36.0
[INFO] [stderr]    Compiling async_tasks_recorder v2.0.2 (/opt/rustwide/workdir)
[INFO] [stderr]    Compiling serial_test_derive v3.0.0
[INFO] [stderr]    Compiling futures-executor v0.3.30
[INFO] [stderr]    Compiling futures v0.3.30
[INFO] [stderr]    Compiling serial_test v3.0.0
[INFO] [stderr]     Finished `test` profile [unoptimized + debuginfo] target(s) in 10.78s
[INFO] running `Command { std: "docker" "inspect" "317dc7d9978caed191677b37a829ae29aaa896ee9409a94562a6015aa580e48d", kill_on_drop: false }`
[INFO] running `Command { std: "docker" "rm" "-f" "317dc7d9978caed191677b37a829ae29aaa896ee9409a94562a6015aa580e48d", kill_on_drop: false }`
[INFO] [stdout] 317dc7d9978caed191677b37a829ae29aaa896ee9409a94562a6015aa580e48d
[INFO] running `Command { std: "docker" "create" "-v" "/var/lib/crater-agent-workspace/builds/worker-1-tc1/target:/opt/rustwide/target:rw,Z" "-v" "/var/lib/crater-agent-workspace/builds/worker-1-tc1/source:/opt/rustwide/workdir:ro,Z" "-v" "/var/lib/crater-agent-workspace/cargo-home:/opt/rustwide/cargo-home:ro,Z" "-v" "/var/lib/crater-agent-workspace/rustup-home:/opt/rustwide/rustup-home:ro,Z" "-e" "SOURCE_DIR=/opt/rustwide/workdir" "-e" "CARGO_TARGET_DIR=/opt/rustwide/target" "-e" "CARGO_INCREMENTAL=0" "-e" "RUST_BACKTRACE=full" "-e" "RUSTFLAGS=--cap-lints=forbid" "-e" "RUSTDOCFLAGS=--cap-lints=forbid" "-e" "CARGO_HOME=/opt/rustwide/cargo-home" "-e" "RUSTUP_HOME=/opt/rustwide/rustup-home" "-w" "/opt/rustwide/workdir" "-m" "1610612736" "--user" "0:0" "--network" "none" "ghcr.io/rust-lang/crates-build-env/linux@sha256:aa71247004a7fa38d13ec170f48f06cdedf5bc50b2a8645e56ed7e992e6fa513" "/opt/rustwide/cargo-home/bin/cargo" "+2fd6efc32704647e64d3d646d21c4c68eae100e4" "test" "--frozen", kill_on_drop: false }`
[INFO] [stdout] 6817a6c2fe43ca46e4905fcc52dd861f04e3932caf36565ea03c7668417c74f0
[INFO] running `Command { std: "docker" "start" "-a" "6817a6c2fe43ca46e4905fcc52dd861f04e3932caf36565ea03c7668417c74f0", kill_on_drop: false }`
[INFO] [stderr]     Finished `test` profile [unoptimized + debuginfo] target(s) in 0.06s
[INFO] [stderr]      Running unittests src/lib.rs (/opt/rustwide/target/debug/deps/async_tasks_recorder-81fe20b5b2f20e8e)
[INFO] [stdout] 
[INFO] [stdout] running 0 tests
[INFO] [stderr]      Running tests/async_tasks_recorder_tests.rs (/opt/rustwide/target/debug/deps/async_tasks_recorder_tests-52fe557d64de669b)
[INFO] [stdout] 
[INFO] [stdout] test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s
[INFO] [stdout] 
[INFO] [stdout] 
[INFO] [stdout] running 21 tests
[INFO] [stdout] test test_once_single ... ok
[INFO] [stdout] test test_once_fail_single - should panic ... ok
[INFO] [stdout] test test_once_fail_multi - should panic ... ok
[INFO] [stdout] test test_basic_single ... ok
[INFO] [stdout] test test_redo_multi ... FAILED
[INFO] [stdout] test test_once_multi ... ok
[INFO] [stdout] test test_random_single ... ok
[INFO] [stdout] test test_redo_single ... ok
[INFO] [stdout] test test_basic_multi ... ok
[INFO] [stdout] test test_random_multi ... ok
[INFO] [stdout] test test_interleave_single ... ok
[INFO] [stdout] test test_interleave_multi ... ok
[INFO] [stdout] test test_simple_launch_check_revoke_multi ... ok
[INFO] [stdout] test test_simple_launch_check_multi ... ok
[INFO] [stdout] test test_simple_launch_check_revoke_loop_single ... ok
[INFO] [stdout] test test_simple_launch_check_single ... ok
[INFO] [stdout] test test_simple_launch_check_revoke_loop_multi ... ok
[INFO] [stdout] test test_simple_launch_and_check_and_revoke_single ... ok
[INFO] [stdout] test test_stress_large_task_latency_range ... ok
[INFO] [stdout] test test_stress_large_group_size_range ... ok
[INFO] [stdout] test test_stress_large_group_num ... ok
[INFO] [stdout] 
[INFO] [stdout] failures:
[INFO] [stdout] 
[INFO] [stdout] ---- test_redo_multi stdout ----
[INFO] [stdout] Instant { tv_sec: 44463, tv_nsec: 668390428 } | Check State Trans task_id_Agxziw_000000: NotFound -> Working
[INFO] [stdout] Instant { tv_sec: 44463, tv_nsec: 676148718 } | Check State Trans task_id_Agxziw_000000: Working -> Working
[INFO] [stdout] Instant { tv_sec: 44463, tv_nsec: 682661597 } | ---->fail task_id_Agxziw_000000, task latency: 13
[INFO] [stdout] 
[INFO] [stdout] thread 'test_redo_multi' (51) panicked at tests/async_tasks_recorder_tests_repo/check.rs:120:13:
[INFO] [stdout] Instant { tv_sec: 44463, tv_nsec: 821832857 } | Timeout before success. task_id: "task_id_Agxziw_000000"
[INFO] [stdout] stack backtrace:
[INFO] [stdout]    0:     0x63d76aed6e42 - std[716c9a7a72e5c14e]::backtrace_rs::backtrace::libunwind::trace
[INFO] [stdout]                                at /rustc/2fd6efc32704647e64d3d646d21c4c68eae100e4/library/std/src/../../backtrace/src/backtrace/libunwind.rs:117:9
[INFO] [stdout]    1:     0x63d76aed6e42 - std[716c9a7a72e5c14e]::backtrace_rs::backtrace::trace_unsynchronized::<std[716c9a7a72e5c14e]::sys::backtrace::_print_fmt::{closure#1}>
[INFO] [stdout]                                at /rustc/2fd6efc32704647e64d3d646d21c4c68eae100e4/library/std/src/../../backtrace/src/backtrace/mod.rs:66:14
[INFO] [stdout]    2:     0x63d76aed6e42 - std[716c9a7a72e5c14e]::sys::backtrace::_print_fmt
[INFO] [stdout]                                at /rustc/2fd6efc32704647e64d3d646d21c4c68eae100e4/library/std/src/sys/backtrace.rs:74:9
[INFO] [stdout]    3:     0x63d76aed6e42 - <<std[716c9a7a72e5c14e]::sys::backtrace::BacktraceLock>::print::DisplayBacktrace as core[c5ed12ab89cc536a]::fmt::Display>::fmt
[INFO] [stdout]                                at /rustc/2fd6efc32704647e64d3d646d21c4c68eae100e4/library/std/src/sys/backtrace.rs:44:26
[INFO] [stdout]    4:     0x63d76aeebfea - <core[c5ed12ab89cc536a]::fmt::rt::Argument>::fmt
[INFO] [stdout]                                at /rustc/2fd6efc32704647e64d3d646d21c4c68eae100e4/library/core/src/fmt/rt.rs:152:76
[INFO] [stdout]    5:     0x63d76aeebfea - core[c5ed12ab89cc536a]::fmt::write
[INFO] [stdout]    6:     0x63d76aedbe46 - std[716c9a7a72e5c14e]::io::default_write_fmt::<alloc[9c68fdf4f4f29218]::vec::Vec<u8>>
[INFO] [stdout]                                at /rustc/2fd6efc32704647e64d3d646d21c4c68eae100e4/library/std/src/io/mod.rs:639:11
[INFO] [stdout]    7:     0x63d76aedbe46 - <alloc[9c68fdf4f4f29218]::vec::Vec<u8> as std[716c9a7a72e5c14e]::io::Write>::write_fmt
[INFO] [stdout]                                at /rustc/2fd6efc32704647e64d3d646d21c4c68eae100e4/library/std/src/io/mod.rs:1994:13
[INFO] [stdout]    8:     0x63d76aeb4d2f - <std[716c9a7a72e5c14e]::sys::backtrace::BacktraceLock>::print
[INFO] [stdout]                                at /rustc/2fd6efc32704647e64d3d646d21c4c68eae100e4/library/std/src/sys/backtrace.rs:47:9
[INFO] [stdout]    9:     0x63d76aeb4d2f - std[716c9a7a72e5c14e]::panicking::default_hook::{closure#0}
[INFO] [stdout]                                at /rustc/2fd6efc32704647e64d3d646d21c4c68eae100e4/library/std/src/panicking.rs:292:27
[INFO] [stdout]   10:     0x63d76aecee19 - std[716c9a7a72e5c14e]::panicking::default_hook
[INFO] [stdout]                                at /rustc/2fd6efc32704647e64d3d646d21c4c68eae100e4/library/std/src/panicking.rs:316:9
[INFO] [stdout]   11:     0x63d76add191e - <alloc[9c68fdf4f4f29218]::boxed::Box<dyn for<'a, 'b> core[c5ed12ab89cc536a]::ops::function::Fn<(&'a std[716c9a7a72e5c14e]::panic::PanicHookInfo<'b>,), Output = ()> + core[c5ed12ab89cc536a]::marker::Sync + core[c5ed12ab89cc536a]::marker::Send> as core[c5ed12ab89cc536a]::ops::function::Fn<(&std[716c9a7a72e5c14e]::panic::PanicHookInfo,)>>::call
[INFO] [stdout]                                at /rustc/2fd6efc32704647e64d3d646d21c4c68eae100e4/library/alloc/src/boxed.rs:2220:9
[INFO] [stdout]   12:     0x63d76add191e - test[64760d2bdea328cc]::test_main_with_exit_callback::<test[64760d2bdea328cc]::test_main::{closure#0}>::{closure#0}
[INFO] [stdout]                                at /rustc/2fd6efc32704647e64d3d646d21c4c68eae100e4/library/test/src/lib.rs:145:21
[INFO] [stdout]   13:     0x63d76aecf082 - <alloc[9c68fdf4f4f29218]::boxed::Box<dyn for<'a, 'b> core[c5ed12ab89cc536a]::ops::function::Fn<(&'a std[716c9a7a72e5c14e]::panic::PanicHookInfo<'b>,), Output = ()> + core[c5ed12ab89cc536a]::marker::Sync + core[c5ed12ab89cc536a]::marker::Send> as core[c5ed12ab89cc536a]::ops::function::Fn<(&std[716c9a7a72e5c14e]::panic::PanicHookInfo,)>>::call
[INFO] [stdout]                                at /rustc/2fd6efc32704647e64d3d646d21c4c68eae100e4/library/alloc/src/boxed.rs:2220:9
[INFO] [stdout]   14:     0x63d76aecf082 - std[716c9a7a72e5c14e]::panicking::panic_with_hook
[INFO] [stdout]                                at /rustc/2fd6efc32704647e64d3d646d21c4c68eae100e4/library/std/src/panicking.rs:833:13
[INFO] [stdout]   15:     0x63d76aeb4de8 - std[716c9a7a72e5c14e]::panicking::panic_handler::{closure#0}
[INFO] [stdout]                                at /rustc/2fd6efc32704647e64d3d646d21c4c68eae100e4/library/std/src/panicking.rs:698:13
[INFO] [stdout]   16:     0x63d76aeac789 - std[716c9a7a72e5c14e]::sys::backtrace::__rust_end_short_backtrace::<std[716c9a7a72e5c14e]::panicking::panic_handler::{closure#0}, !>
[INFO] [stdout]                                at /rustc/2fd6efc32704647e64d3d646d21c4c68eae100e4/library/std/src/sys/backtrace.rs:182:18
[INFO] [stdout]   17:     0x63d76aeb5c3d - __rustc[4f0b026143eab78e]::rust_begin_unwind
[INFO] [stdout]                                at /rustc/2fd6efc32704647e64d3d646d21c4c68eae100e4/library/std/src/panicking.rs:689:5
[INFO] [stdout]   18:     0x63d76aeec72c - core[c5ed12ab89cc536a]::panicking::panic_fmt
[INFO] [stdout]                                at /rustc/2fd6efc32704647e64d3d646d21c4c68eae100e4/library/core/src/panicking.rs:80:14
[INFO] [stdout]   19:     0x63d76ad8ec2f - async_tasks_recorder_tests[d8cac5fd5cbfc16]::async_tasks_recorder_tests_repo::check::check_success_auto_redo::{closure#0}
[INFO] [stdout]                                at /opt/rustwide/workdir/tests/async_tasks_recorder_tests_repo/check.rs:120:13
[INFO] [stdout]   20:     0x63d76ad8af6c - async_tasks_recorder_tests[d8cac5fd5cbfc16]::async_tasks_recorder_tests_repo::test_once_redo::{closure#0}
[INFO] [stdout]                                at /opt/rustwide/workdir/tests/async_tasks_recorder_tests_repo/mod.rs:57:45
[INFO] [stdout]   21:     0x63d76ad3fd7d - <tokio[ecd4c57ae6680948]::runtime::park::CachedParkThread>::block_on::<async_tasks_recorder_tests[d8cac5fd5cbfc16]::async_tasks_recorder_tests_repo::test_once_redo::{closure#0}>::{closure#0}
[INFO] [stdout]                                at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.36.0/src/runtime/park.rs:281:74
[INFO] [stdout]   22:     0x63d76ad3e858 - tokio[ecd4c57ae6680948]::runtime::coop::with_budget::<core[c5ed12ab89cc536a]::task::poll::Poll<()>, <tokio[ecd4c57ae6680948]::runtime::park::CachedParkThread>::block_on<async_tasks_recorder_tests[d8cac5fd5cbfc16]::async_tasks_recorder_tests_repo::test_once_redo::{closure#0}>::{closure#0}>
[INFO] [stdout]                                at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.36.0/src/runtime/coop.rs:107:5
[INFO] [stdout]   23:     0x63d76ad3e858 - tokio[ecd4c57ae6680948]::runtime::coop::budget::<core[c5ed12ab89cc536a]::task::poll::Poll<()>, <tokio[ecd4c57ae6680948]::runtime::park::CachedParkThread>::block_on<async_tasks_recorder_tests[d8cac5fd5cbfc16]::async_tasks_recorder_tests_repo::test_once_redo::{closure#0}>::{closure#0}>
[INFO] [stdout]                                at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.36.0/src/runtime/coop.rs:73:5
[INFO] [stdout]   24:     0x63d76ad3e858 - <tokio[ecd4c57ae6680948]::runtime::park::CachedParkThread>::block_on::<async_tasks_recorder_tests[d8cac5fd5cbfc16]::async_tasks_recorder_tests_repo::test_once_redo::{closure#0}>
[INFO] [stdout]                                at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.36.0/src/runtime/park.rs:281:31
[INFO] [stdout]   25:     0x63d76adaa7b3 - <tokio[ecd4c57ae6680948]::runtime::context::blocking::BlockingRegionGuard>::block_on::<async_tasks_recorder_tests[d8cac5fd5cbfc16]::async_tasks_recorder_tests_repo::test_once_redo::{closure#0}>
[INFO] [stdout]                                at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.36.0/src/runtime/context/blocking.rs:66:14
[INFO] [stdout]   26:     0x63d76adbdbb8 - <tokio[ecd4c57ae6680948]::runtime::scheduler::multi_thread::MultiThread>::block_on::<async_tasks_recorder_tests[d8cac5fd5cbfc16]::async_tasks_recorder_tests_repo::test_once_redo::{closure#0}>::{closure#0}
[INFO] [stdout]                                at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.36.0/src/runtime/scheduler/multi_thread/mod.rs:87:22
[INFO] [stdout]   27:     0x63d76adbc927 - tokio[ecd4c57ae6680948]::runtime::context::runtime::enter_runtime::<<tokio[ecd4c57ae6680948]::runtime::scheduler::multi_thread::MultiThread>::block_on<async_tasks_recorder_tests[d8cac5fd5cbfc16]::async_tasks_recorder_tests_repo::test_once_redo::{closure#0}>::{closure#0}, ()>
[INFO] [stdout]                                at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.36.0/src/runtime/context/runtime.rs:65:16
[INFO] [stdout]   28:     0x63d76adb8ff1 - <tokio[ecd4c57ae6680948]::runtime::scheduler::multi_thread::MultiThread>::block_on::<async_tasks_recorder_tests[d8cac5fd5cbfc16]::async_tasks_recorder_tests_repo::test_once_redo::{closure#0}>
[INFO] [stdout]                                at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.36.0/src/runtime/scheduler/multi_thread/mod.rs:86:9
[INFO] [stdout]   29:     0x63d76ad67d49 - <tokio[ecd4c57ae6680948]::runtime::runtime::Runtime>::block_on::<async_tasks_recorder_tests[d8cac5fd5cbfc16]::async_tasks_recorder_tests_repo::test_once_redo::{closure#0}>
[INFO] [stdout]                                at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.36.0/src/runtime/runtime.rs:350:50
[INFO] [stdout]   30:     0x63d76ad5b156 - async_tasks_recorder_tests[d8cac5fd5cbfc16]::async_tasks_recorder_tests_repo::tools::do_async_test::<async_tasks_recorder_tests[d8cac5fd5cbfc16]::async_tasks_recorder_tests_repo::test_once_redo::{closure#0}>
[INFO] [stdout]                                at /opt/rustwide/workdir/tests/async_tasks_recorder_tests_repo/tools.rs:26:13
[INFO] [stdout]   31:     0x63d76ada2af4 - async_tasks_recorder_tests[d8cac5fd5cbfc16]::test_redo_multi::{closure#0}
[INFO] [stdout]                                at /opt/rustwide/workdir/tests/async_tasks_recorder_tests.rs:73:5
[INFO] [stdout]   32:     0x63d76ad23d6e - <async_tasks_recorder_tests[d8cac5fd5cbfc16]::test_redo_multi::{closure#0} as core[c5ed12ab89cc536a]::ops::function::FnOnce<()>>::call_once
[INFO] [stdout]                                at /rustc/2fd6efc32704647e64d3d646d21c4c68eae100e4/library/core/src/ops/function.rs:250:5
[INFO] [stdout]   33:     0x63d76ae060d7 - serial_test[1961064367b0be09]::parallel_code_lock::local_parallel_core::{closure#1}
[INFO] [stdout]                                at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/serial_test-3.0.0/src/parallel_code_lock.rs:44:9
[INFO] [stdout]   34:     0x63d76ae0e65b - std[716c9a7a72e5c14e]::panicking::catch_unwind::do_call::<serial_test[1961064367b0be09]::parallel_code_lock::local_parallel_core::{closure#1}, ()>
[INFO] [stdout]                                at /rustc/2fd6efc32704647e64d3d646d21c4c68eae100e4/library/std/src/panicking.rs:581:40
[INFO] [stdout]   35:     0x63d76ae12cfb - __rust_try
[INFO] [stdout]   36:     0x63d76ae0db20 - std[716c9a7a72e5c14e]::panicking::catch_unwind::<(), serial_test[1961064367b0be09]::parallel_code_lock::local_parallel_core::{closure#1}>
[INFO] [stdout]                                at /rustc/2fd6efc32704647e64d3d646d21c4c68eae100e4/library/std/src/panicking.rs:544:19
[INFO] [stdout]   37:     0x63d76ae0db20 - std[716c9a7a72e5c14e]::panic::catch_unwind::<serial_test[1961064367b0be09]::parallel_code_lock::local_parallel_core::{closure#1}, ()>
[INFO] [stdout]                                at /rustc/2fd6efc32704647e64d3d646d21c4c68eae100e4/library/std/src/panic.rs:359:14
[INFO] [stdout]   38:     0x63d76ae08deb - serial_test[1961064367b0be09]::parallel_code_lock::local_parallel_core
[INFO] [stdout]                                at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/serial_test-3.0.0/src/parallel_code_lock.rs:43:15
[INFO] [stdout]   39:     0x63d76ada34d9 - async_tasks_recorder_tests[d8cac5fd5cbfc16]::test_redo_multi
[INFO] [stdout]                                at /opt/rustwide/workdir/tests/async_tasks_recorder_tests.rs:71:1
[INFO] [stdout]   40:     0x63d76ada26f7 - async_tasks_recorder_tests[d8cac5fd5cbfc16]::test_redo_multi::{closure#0}
[INFO] [stdout]                                at /opt/rustwide/workdir/tests/async_tasks_recorder_tests.rs:72:21
[INFO] [stdout]   41:     0x63d76ad23846 - <async_tasks_recorder_tests[d8cac5fd5cbfc16]::test_redo_multi::{closure#0} as core[c5ed12ab89cc536a]::ops::function::FnOnce<()>>::call_once
[INFO] [stdout]                                at /rustc/2fd6efc32704647e64d3d646d21c4c68eae100e4/library/core/src/ops/function.rs:250:5
[INFO] [stdout]   42:     0x63d76adc5ccb - <fn() -> core[c5ed12ab89cc536a]::result::Result<(), alloc[9c68fdf4f4f29218]::string::String> as core[c5ed12ab89cc536a]::ops::function::FnOnce<()>>::call_once
[INFO] [stdout]                                at /rustc/2fd6efc32704647e64d3d646d21c4c68eae100e4/library/core/src/ops/function.rs:250:5
[INFO] [stdout]   43:     0x63d76adc5ccb - test[64760d2bdea328cc]::__rust_begin_short_backtrace::<core[c5ed12ab89cc536a]::result::Result<(), alloc[9c68fdf4f4f29218]::string::String>, fn() -> core[c5ed12ab89cc536a]::result::Result<(), alloc[9c68fdf4f4f29218]::string::String>>
[INFO] [stdout]                                at /rustc/2fd6efc32704647e64d3d646d21c4c68eae100e4/library/test/src/lib.rs:663:18
[INFO] [stdout]   44:     0x63d76add252a - test[64760d2bdea328cc]::run_test_in_process::{closure#0}
[INFO] [stdout]                                at /rustc/2fd6efc32704647e64d3d646d21c4c68eae100e4/library/test/src/lib.rs:686:74
[INFO] [stdout]   45:     0x63d76add252a - <core[c5ed12ab89cc536a]::panic::unwind_safe::AssertUnwindSafe<test[64760d2bdea328cc]::run_test_in_process::{closure#0}> as core[c5ed12ab89cc536a]::ops::function::FnOnce<()>>::call_once
[INFO] [stdout]                                at /rustc/2fd6efc32704647e64d3d646d21c4c68eae100e4/library/core/src/panic/unwind_safe.rs:274:9
[INFO] [stdout]   46:     0x63d76add252a - std[716c9a7a72e5c14e]::panicking::catch_unwind::do_call::<core[c5ed12ab89cc536a]::panic::unwind_safe::AssertUnwindSafe<test[64760d2bdea328cc]::run_test_in_process::{closure#0}>, core[c5ed12ab89cc536a]::result::Result<(), alloc[9c68fdf4f4f29218]::string::String>>
[INFO] [stdout]                                at /rustc/2fd6efc32704647e64d3d646d21c4c68eae100e4/library/std/src/panicking.rs:581:40
[INFO] [stdout]   47:     0x63d76add252a - std[716c9a7a72e5c14e]::panicking::catch_unwind::<core[c5ed12ab89cc536a]::result::Result<(), alloc[9c68fdf4f4f29218]::string::String>, core[c5ed12ab89cc536a]::panic::unwind_safe::AssertUnwindSafe<test[64760d2bdea328cc]::run_test_in_process::{closure#0}>>
[INFO] [stdout]                                at /rustc/2fd6efc32704647e64d3d646d21c4c68eae100e4/library/std/src/panicking.rs:544:19
[INFO] [stdout]   48:     0x63d76add252a - std[716c9a7a72e5c14e]::panic::catch_unwind::<core[c5ed12ab89cc536a]::panic::unwind_safe::AssertUnwindSafe<test[64760d2bdea328cc]::run_test_in_process::{closure#0}>, core[c5ed12ab89cc536a]::result::Result<(), alloc[9c68fdf4f4f29218]::string::String>>
[INFO] [stdout]                                at /rustc/2fd6efc32704647e64d3d646d21c4c68eae100e4/library/std/src/panic.rs:359:14
[INFO] [stdout]   49:     0x63d76add252a - test[64760d2bdea328cc]::run_test_in_process
[INFO] [stdout]                                at /rustc/2fd6efc32704647e64d3d646d21c4c68eae100e4/library/test/src/lib.rs:686:27
[INFO] [stdout]   50:     0x63d76add252a - test[64760d2bdea328cc]::run_test::{closure#0}
[INFO] [stdout]                                at /rustc/2fd6efc32704647e64d3d646d21c4c68eae100e4/library/test/src/lib.rs:607:43
[INFO] [stdout]   51:     0x63d76adcc9f4 - test[64760d2bdea328cc]::run_test::{closure#1}
[INFO] [stdout]                                at /rustc/2fd6efc32704647e64d3d646d21c4c68eae100e4/library/test/src/lib.rs:637:41
[INFO] [stdout]   52:     0x63d76adcc9f4 - std[716c9a7a72e5c14e]::sys::backtrace::__rust_begin_short_backtrace::<test[64760d2bdea328cc]::run_test::{closure#1}, ()>
[INFO] [stdout]                                at /rustc/2fd6efc32704647e64d3d646d21c4c68eae100e4/library/std/src/sys/backtrace.rs:166:18
[INFO] [stdout]   53:     0x63d76add5022 - std[716c9a7a72e5c14e]::thread::lifecycle::spawn_unchecked::<test[64760d2bdea328cc]::run_test::{closure#1}, ()>::{closure#1}::{closure#0}
[INFO] [stdout]                                at /rustc/2fd6efc32704647e64d3d646d21c4c68eae100e4/library/std/src/thread/lifecycle.rs:91:13
[INFO] [stdout]   54:     0x63d76add5022 - <core[c5ed12ab89cc536a]::panic::unwind_safe::AssertUnwindSafe<std[716c9a7a72e5c14e]::thread::lifecycle::spawn_unchecked<test[64760d2bdea328cc]::run_test::{closure#1}, ()>::{closure#1}::{closure#0}> as core[c5ed12ab89cc536a]::ops::function::FnOnce<()>>::call_once
[INFO] [stdout]                                at /rustc/2fd6efc32704647e64d3d646d21c4c68eae100e4/library/core/src/panic/unwind_safe.rs:274:9
[INFO] [stdout]   55:     0x63d76add5022 - std[716c9a7a72e5c14e]::panicking::catch_unwind::do_call::<core[c5ed12ab89cc536a]::panic::unwind_safe::AssertUnwindSafe<std[716c9a7a72e5c14e]::thread::lifecycle::spawn_unchecked<test[64760d2bdea328cc]::run_test::{closure#1}, ()>::{closure#1}::{closure#0}>, ()>
[INFO] [stdout]                                at /rustc/2fd6efc32704647e64d3d646d21c4c68eae100e4/library/std/src/panicking.rs:581:40
[INFO] [stdout]   56:     0x63d76add5022 - std[716c9a7a72e5c14e]::panicking::catch_unwind::<(), core[c5ed12ab89cc536a]::panic::unwind_safe::AssertUnwindSafe<std[716c9a7a72e5c14e]::thread::lifecycle::spawn_unchecked<test[64760d2bdea328cc]::run_test::{closure#1}, ()>::{closure#1}::{closure#0}>>
[INFO] [stdout]                                at /rustc/2fd6efc32704647e64d3d646d21c4c68eae100e4/library/std/src/panicking.rs:544:19
[INFO] [stdout]   57:     0x63d76add5022 - std[716c9a7a72e5c14e]::panic::catch_unwind::<core[c5ed12ab89cc536a]::panic::unwind_safe::AssertUnwindSafe<std[716c9a7a72e5c14e]::thread::lifecycle::spawn_unchecked<test[64760d2bdea328cc]::run_test::{closure#1}, ()>::{closure#1}::{closure#0}>, ()>
[INFO] [stdout]                                at /rustc/2fd6efc32704647e64d3d646d21c4c68eae100e4/library/std/src/panic.rs:359:14
[INFO] [stdout]   58:     0x63d76add5022 - std[716c9a7a72e5c14e]::thread::lifecycle::spawn_unchecked::<test[64760d2bdea328cc]::run_test::{closure#1}, ()>::{closure#1}
[INFO] [stdout]                                at /rustc/2fd6efc32704647e64d3d646d21c4c68eae100e4/library/std/src/thread/lifecycle.rs:89:26
[INFO] [stdout]   59:     0x63d76add5022 - <std[716c9a7a72e5c14e]::thread::lifecycle::spawn_unchecked<test[64760d2bdea328cc]::run_test::{closure#1}, ()>::{closure#1} as core[c5ed12ab89cc536a]::ops::function::FnOnce<()>>::call_once::{shim:vtable#0}
[INFO] [stdout]                                at /rustc/2fd6efc32704647e64d3d646d21c4c68eae100e4/library/core/src/ops/function.rs:250:5
[INFO] [stdout]   60:     0x63d76aed66af - <alloc[9c68fdf4f4f29218]::boxed::Box<dyn core[c5ed12ab89cc536a]::ops::function::FnOnce<(), Output = ()> + core[c5ed12ab89cc536a]::marker::Send> as core[c5ed12ab89cc536a]::ops::function::FnOnce<()>>::call_once
[INFO] [stdout]                                at /rustc/2fd6efc32704647e64d3d646d21c4c68eae100e4/library/alloc/src/boxed.rs:2206:9
[INFO] [stdout]   61:     0x63d76aed66af - <std[716c9a7a72e5c14e]::sys::thread::unix::Thread>::new::thread_start
[INFO] [stdout]                                at /rustc/2fd6efc32704647e64d3d646d21c4c68eae100e4/library/std/src/sys/thread/unix.rs:119:17
[INFO] [stdout]   62:     0x755488e0baa4 - <unknown>
[INFO] [stdout]   63:     0x755488e98a64 - clone
[INFO] [stdout]   64:                0x0 - <unknown>
[INFO] [stdout] 
[INFO] [stdout] 
[INFO] [stdout] failures:
[INFO] [stdout]     test_redo_multi
[INFO] [stdout] 
[INFO] [stdout] test result: FAILED. 20 passed; 1 failed; 0 ignored; 0 measured; 0 filtered out; finished in 9.11s
[INFO] [stdout] 
[INFO] [stderr] error: test failed, to rerun pass `--test async_tasks_recorder_tests`
[INFO] running `Command { std: "docker" "inspect" "6817a6c2fe43ca46e4905fcc52dd861f04e3932caf36565ea03c7668417c74f0", kill_on_drop: false }`
[INFO] running `Command { std: "docker" "rm" "-f" "6817a6c2fe43ca46e4905fcc52dd861f04e3932caf36565ea03c7668417c74f0", kill_on_drop: false }`
[INFO] [stdout] 6817a6c2fe43ca46e4905fcc52dd861f04e3932caf36565ea03c7668417c74f0