[INFO] fetching crate shimmytok 0.5.0...
[INFO] testing shimmytok-0.5.0 against beta-2025-10-28 for beta-1.92-2
[INFO] extracting crate shimmytok 0.5.0 into /workspace/builds/worker-1-tc2/source
[INFO] started tweaking crates.io crate shimmytok 0.5.0
[INFO] removed 0 missing examples
[INFO] removed 0 missing tests
[INFO] finished tweaking crates.io crate shimmytok 0.5.0
[INFO] tweaked toml for crates.io crate shimmytok 0.5.0 written to /workspace/builds/worker-1-tc2/source/Cargo.toml
[INFO] validating manifest of crates.io crate shimmytok 0.5.0 on toolchain beta-2025-10-28
[INFO] running `Command { std: CARGO_HOME="/workspace/cargo-home" RUSTUP_HOME="/workspace/rustup-home" "/workspace/cargo-home/bin/cargo" "+beta-2025-10-28" "metadata" "--manifest-path" "Cargo.toml" "--no-deps", kill_on_drop: false }`
[INFO] crate crates.io crate shimmytok 0.5.0 already has a lockfile, it will not be regenerated
[INFO] running `Command { std: CARGO_HOME="/workspace/cargo-home" RUSTUP_HOME="/workspace/rustup-home" "/workspace/cargo-home/bin/cargo" "+beta-2025-10-28" "fetch" "--manifest-path" "Cargo.toml", kill_on_drop: false }`
[INFO] running `Command { std: "docker" "create" "-v" "/var/lib/crater-agent-workspace/builds/worker-1-tc2/target:/opt/rustwide/target:rw,Z" "-v" "/var/lib/crater-agent-workspace/builds/worker-1-tc2/source:/opt/rustwide/workdir:ro,Z" "-v" "/var/lib/crater-agent-workspace/cargo-home:/opt/rustwide/cargo-home:ro,Z" "-v" "/var/lib/crater-agent-workspace/rustup-home:/opt/rustwide/rustup-home:ro,Z" "-e" "SOURCE_DIR=/opt/rustwide/workdir" "-e" "CARGO_TARGET_DIR=/opt/rustwide/target" "-e" "CARGO_HOME=/opt/rustwide/cargo-home" "-e" "RUSTUP_HOME=/opt/rustwide/rustup-home" "-w" "/opt/rustwide/workdir" "-m" "1610612736" "--user" "0:0" "--network" "none" "ghcr.io/rust-lang/crates-build-env/linux@sha256:4848fb76d95f26979359cc7e45710b1dbc8f3acb7aeedee7c460d7702230f228" "/opt/rustwide/cargo-home/bin/cargo" "+beta-2025-10-28" "metadata" "--no-deps" "--format-version=1", kill_on_drop: false }`
[INFO] [stdout] ebda4d095e16184687351b5a1efae77548bda386b6309b8e0a4525fe087706b9
[INFO] running `Command { std: "docker" "start" "-a" "ebda4d095e16184687351b5a1efae77548bda386b6309b8e0a4525fe087706b9", kill_on_drop: false }`
[INFO] running `Command { std: "docker" "inspect" "ebda4d095e16184687351b5a1efae77548bda386b6309b8e0a4525fe087706b9", kill_on_drop: false }`
[INFO] running `Command { std: "docker" "rm" "-f" "ebda4d095e16184687351b5a1efae77548bda386b6309b8e0a4525fe087706b9", kill_on_drop: false }`
[INFO] [stdout] ebda4d095e16184687351b5a1efae77548bda386b6309b8e0a4525fe087706b9
[INFO] running `Command { std: "docker" "create" "-v" "/var/lib/crater-agent-workspace/builds/worker-1-tc2/target:/opt/rustwide/target:rw,Z" "-v" "/var/lib/crater-agent-workspace/builds/worker-1-tc2/source:/opt/rustwide/workdir:ro,Z" "-v" "/var/lib/crater-agent-workspace/cargo-home:/opt/rustwide/cargo-home:ro,Z" "-v" "/var/lib/crater-agent-workspace/rustup-home:/opt/rustwide/rustup-home:ro,Z" "-e" "SOURCE_DIR=/opt/rustwide/workdir" "-e" "CARGO_TARGET_DIR=/opt/rustwide/target" "-e" "CARGO_INCREMENTAL=0" "-e" "RUST_BACKTRACE=full" "-e" "RUSTFLAGS=--cap-lints=warn" "-e" "RUSTDOCFLAGS=--cap-lints=warn" "-e" "CARGO_HOME=/opt/rustwide/cargo-home" "-e" "RUSTUP_HOME=/opt/rustwide/rustup-home" "-w" "/opt/rustwide/workdir" "-m" "1610612736" "--user" "0:0" "--network" "none" "ghcr.io/rust-lang/crates-build-env/linux@sha256:4848fb76d95f26979359cc7e45710b1dbc8f3acb7aeedee7c460d7702230f228" "/opt/rustwide/cargo-home/bin/cargo" "+beta-2025-10-28" "build" "--frozen" "--message-format=json", kill_on_drop: false }`
[INFO] [stdout] 96cc70e4f5fbd2ef58376d29a5ad7ee3953b08d5141dcab8f33dfd27fc29b1a5
[INFO] running `Command { std: "docker" "start" "-a" "96cc70e4f5fbd2ef58376d29a5ad7ee3953b08d5141dcab8f33dfd27fc29b1a5", kill_on_drop: false }`
[INFO] [stderr]    Compiling bit-set v0.5.3
[INFO] [stderr]    Compiling regex-automata v0.4.13
[INFO] [stderr]    Compiling syn v2.0.107
[INFO] [stderr]    Compiling thiserror-impl v2.0.17
[INFO] [stderr]    Compiling thiserror v2.0.17
[INFO] [stderr]    Compiling fancy-regex v0.13.0
[INFO] [stderr]    Compiling regex v1.12.2
[INFO] [stderr]    Compiling shimmytok v0.5.0 (/opt/rustwide/workdir)
[INFO] [stderr]     Finished `dev` profile [unoptimized + debuginfo] target(s) in 7.67s
[INFO] running `Command { std: "docker" "inspect" "96cc70e4f5fbd2ef58376d29a5ad7ee3953b08d5141dcab8f33dfd27fc29b1a5", kill_on_drop: false }`
[INFO] running `Command { std: "docker" "rm" "-f" "96cc70e4f5fbd2ef58376d29a5ad7ee3953b08d5141dcab8f33dfd27fc29b1a5", kill_on_drop: false }`
[INFO] [stdout] 96cc70e4f5fbd2ef58376d29a5ad7ee3953b08d5141dcab8f33dfd27fc29b1a5
[INFO] running `Command { std: "docker" "create" "-v" "/var/lib/crater-agent-workspace/builds/worker-1-tc2/target:/opt/rustwide/target:rw,Z" "-v" "/var/lib/crater-agent-workspace/builds/worker-1-tc2/source:/opt/rustwide/workdir:ro,Z" "-v" "/var/lib/crater-agent-workspace/cargo-home:/opt/rustwide/cargo-home:ro,Z" "-v" "/var/lib/crater-agent-workspace/rustup-home:/opt/rustwide/rustup-home:ro,Z" "-e" "SOURCE_DIR=/opt/rustwide/workdir" "-e" "CARGO_TARGET_DIR=/opt/rustwide/target" "-e" "CARGO_INCREMENTAL=0" "-e" "RUST_BACKTRACE=full" "-e" "RUSTFLAGS=--cap-lints=warn" "-e" "RUSTDOCFLAGS=--cap-lints=warn" "-e" "CARGO_HOME=/opt/rustwide/cargo-home" "-e" "RUSTUP_HOME=/opt/rustwide/rustup-home" "-w" "/opt/rustwide/workdir" "-m" "1610612736" "--user" "0:0" "--network" "none" "ghcr.io/rust-lang/crates-build-env/linux@sha256:4848fb76d95f26979359cc7e45710b1dbc8f3acb7aeedee7c460d7702230f228" "/opt/rustwide/cargo-home/bin/cargo" "+beta-2025-10-28" "test" "--frozen" "--no-run" "--message-format=json", kill_on_drop: false }`
[INFO] [stdout] c62f9ffc66580e5ffe7f8cc7c5f97d077a0c07bb2a2d6565b0b154e7fecf3292
[INFO] running `Command { std: "docker" "start" "-a" "c62f9ffc66580e5ffe7f8cc7c5f97d077a0c07bb2a2d6565b0b154e7fecf3292", kill_on_drop: false }`
[INFO] [stderr]    Compiling clap_lex v0.7.6
[INFO] [stderr]    Compiling syn v2.0.107
[INFO] [stderr]    Compiling is-terminal v0.4.16
[INFO] [stderr]    Compiling criterion-plot v0.5.0
[INFO] [stderr]    Compiling dirs-sys v0.4.1
[INFO] [stderr]    Compiling clap_builder v4.5.50
[INFO] [stderr]    Compiling dirs v5.0.1
[INFO] [stderr]    Compiling clap v4.5.50
[INFO] [stderr]    Compiling zerocopy-derive v0.8.27
[INFO] [stderr]    Compiling serde_derive v1.0.228
[INFO] [stderr]    Compiling thiserror-impl v2.0.17
[INFO] [stderr]    Compiling zerocopy v0.8.27
[INFO] [stderr]    Compiling thiserror v2.0.17
[INFO] [stderr]    Compiling shimmytok v0.5.0 (/opt/rustwide/workdir)
[INFO] [stderr]    Compiling serde v1.0.228
[INFO] [stderr]    Compiling half v2.7.1
[INFO] [stderr]    Compiling ciborium-ll v0.2.2
[INFO] [stderr]    Compiling tinytemplate v1.2.1
[INFO] [stderr]    Compiling ciborium v0.2.2
[INFO] [stderr]    Compiling criterion v0.5.1
[INFO] [stderr]     Finished `test` profile [unoptimized + debuginfo] target(s) in 21.98s
[INFO] running `Command { std: "docker" "inspect" "c62f9ffc66580e5ffe7f8cc7c5f97d077a0c07bb2a2d6565b0b154e7fecf3292", kill_on_drop: false }`
[INFO] running `Command { std: "docker" "rm" "-f" "c62f9ffc66580e5ffe7f8cc7c5f97d077a0c07bb2a2d6565b0b154e7fecf3292", kill_on_drop: false }`
[INFO] [stdout] c62f9ffc66580e5ffe7f8cc7c5f97d077a0c07bb2a2d6565b0b154e7fecf3292
[INFO] running `Command { std: "docker" "create" "-v" "/var/lib/crater-agent-workspace/builds/worker-1-tc2/target:/opt/rustwide/target:rw,Z" "-v" "/var/lib/crater-agent-workspace/builds/worker-1-tc2/source:/opt/rustwide/workdir:ro,Z" "-v" "/var/lib/crater-agent-workspace/cargo-home:/opt/rustwide/cargo-home:ro,Z" "-v" "/var/lib/crater-agent-workspace/rustup-home:/opt/rustwide/rustup-home:ro,Z" "-e" "SOURCE_DIR=/opt/rustwide/workdir" "-e" "CARGO_TARGET_DIR=/opt/rustwide/target" "-e" "CARGO_INCREMENTAL=0" "-e" "RUST_BACKTRACE=full" "-e" "RUSTFLAGS=--cap-lints=warn" "-e" "RUSTDOCFLAGS=--cap-lints=warn" "-e" "CARGO_HOME=/opt/rustwide/cargo-home" "-e" "RUSTUP_HOME=/opt/rustwide/rustup-home" "-w" "/opt/rustwide/workdir" "-m" "1610612736" "--user" "0:0" "--network" "none" "ghcr.io/rust-lang/crates-build-env/linux@sha256:4848fb76d95f26979359cc7e45710b1dbc8f3acb7aeedee7c460d7702230f228" "/opt/rustwide/cargo-home/bin/cargo" "+beta-2025-10-28" "test" "--frozen", kill_on_drop: false }`
[INFO] [stdout] f43a227ed17df43a08efcfe90b81c16420c46396299cbd2cc37e4696de174e0f
[INFO] running `Command { std: "docker" "start" "-a" "f43a227ed17df43a08efcfe90b81c16420c46396299cbd2cc37e4696de174e0f", kill_on_drop: false }`
[INFO] [stderr]     Finished `test` profile [unoptimized + debuginfo] target(s) in 0.16s
[INFO] [stderr]      Running unittests src/lib.rs (/opt/rustwide/target/debug/deps/shimmytok-988ccd6fac9e8792)
[INFO] [stdout] 
[INFO] [stdout] running 2 tests
[INFO] [stdout] test byte_encoder::tests::test_hello ... ok
[INFO] [stdout] test byte_encoder::tests::test_space_encoding ... ok
[INFO] [stderr]      Running tests/debug_merges.rs (/opt/rustwide/target/debug/deps/debug_merges-86eb48572617ca9b)
[INFO] [stdout] 
[INFO] [stderr]      Running tests/test_basic.rs (/opt/rustwide/target/debug/deps/test_basic-c835945f6b3315fb)
[INFO] [stdout] test result: ok. 2 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.02s
[INFO] [stdout] 
[INFO] [stdout] 
[INFO] [stdout] running 1 test
[INFO] [stdout] test debug_gpt2_merges ... ignored
[INFO] [stdout] 
[INFO] [stdout] test result: ok. 0 passed; 0 failed; 1 ignored; 0 measured; 0 filtered out; finished in 0.00s
[INFO] [stdout] 
[INFO] [stdout] 
[INFO] [stdout] running 2 tests
[INFO] [stdout] test test_encode_batch ... ok
[INFO] [stderr]      Running tests/test_bpe.rs (/opt/rustwide/target/debug/deps/test_bpe-e4765407a30d353c)
[INFO] [stdout] test test_load_gguf ... ok
[INFO] [stdout] 
[INFO] [stdout] test result: ok. 2 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.02s
[INFO] [stdout] 
[INFO] [stdout] 
[INFO] [stdout] running 1 test
[INFO] [stderr]      Running tests/test_bpe_edge_cases.rs (/opt/rustwide/target/debug/deps/test_bpe_edge_cases-b7c5f976b7cae609)
[INFO] [stdout] test test_gpt2_tokenization ... ok
[INFO] [stdout] 
[INFO] [stdout] test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s
[INFO] [stdout] 
[INFO] [stdout] 
[INFO] [stdout] running 8 tests
[INFO] [stdout] test test_empty_string ... ok
[INFO] [stdout] test test_long_string ... ok
[INFO] [stdout] test test_multiple_spaces ... ok
[INFO] [stdout] test test_newlines_and_tabs ... ok
[INFO] [stdout] test test_single_char ... ok
[INFO] [stdout] test test_unicode_emoji ... ok
[INFO] [stdout] test test_special_chars ... ok
[INFO] [stdout] test test_unicode_cjk ... ok
[INFO] [stdout] 
[INFO] [stdout] test result: ok. 8 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.02s
[INFO] [stdout] 
[INFO] [stderr]      Running tests/test_bpe_validation.rs (/opt/rustwide/target/debug/deps/test_bpe_validation-017ae1110317cba5)
[INFO] [stdout] 
[INFO] [stderr]      Running tests/test_comprehensive.rs (/opt/rustwide/target/debug/deps/test_comprehensive-a0fd2b168d7da425)
[INFO] [stdout] running 6 tests
[INFO] [stdout] test test_deepseek_coder_validation ... ignored
[INFO] [stdout] test test_deepseek_llm_validation ... ignored
[INFO] [stdout] test test_gpt2_validation ... ignored
[INFO] [stdout] test test_phi2_validation ... ignored
[INFO] [stdout] test test_qwen2_validation ... ignored
[INFO] [stdout] test test_starcoder_validation ... ignored
[INFO] [stdout] 
[INFO] [stdout] test result: ok. 0 passed; 0 failed; 6 ignored; 0 measured; 0 filtered out; finished in 0.00s
[INFO] [stdout] 
[INFO] [stdout] 
[INFO] [stdout] running 1 test
[INFO] [stdout] test test_against_llama_cpp ... ok
[INFO] [stderr]      Running tests/test_debug.rs (/opt/rustwide/target/debug/deps/test_debug-6395f6f28164ae5e)
[INFO] [stdout] 
[INFO] [stdout] test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s
[INFO] [stdout] 
[INFO] [stdout] 
[INFO] [stdout] running 1 test
[INFO] [stdout] test test_debug_tokens ... ok
[INFO] [stdout] 
[INFO] [stdout] test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s
[INFO] [stdout] 
[INFO] [stderr]      Running tests/test_detailed.rs (/opt/rustwide/target/debug/deps/test_detailed-f7b65190ceac474f)
[INFO] [stdout] 
[INFO] [stdout] running 1 test
[INFO] [stderr]      Running tests/test_error_handling.rs (/opt/rustwide/target/debug/deps/test_error_handling-7e83fd10189581cc)
[INFO] [stdout] test test_detailed_merge ... ok
[INFO] [stdout] 
[INFO] [stdout] test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s
[INFO] [stdout] 
[INFO] [stdout] 
[INFO] [stdout] running 7 tests
[INFO] [stdout] test test_decode_with_special_tokens ... ok
[INFO] [stdout] test test_empty_input ... ok
[INFO] [stdout] test test_max_token_validation ... ok
[INFO] [stdout] test test_invalid_token_id ... ok
[INFO] [stdout] test test_missing_file ... ok
[INFO] [stdout] test test_very_large_input ... ok
[INFO] [stdout] test test_round_trip_fuzz ... ok
[INFO] [stdout] 
[INFO] [stdout] test result: ok. 7 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s
[INFO] [stdout] 
[INFO] [stderr]      Running tests/test_merge_debug.rs (/opt/rustwide/target/debug/deps/test_merge_debug-22ee61c805dcfe9b)
[INFO] [stdout] 
[INFO] [stdout] running 1 test
[INFO] [stdout] test test_merge_sequence ... ok
[INFO] [stdout] 
[INFO] [stdout] test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s
[INFO] [stdout] 
[INFO] [stderr]      Running tests/test_streaming.rs (/opt/rustwide/target/debug/deps/test_streaming-b66dc9142156dee1)
[INFO] [stdout] 
[INFO] [stdout] running 6 tests
[INFO] [stdout] test test_decode_single ... ok
[INFO] [stdout] test test_is_special_token ... ok
[INFO] [stdout] test test_token_to_piece ... ok
[INFO] [stdout] test test_decode_single_with_special_tokens ... ok
[INFO] [stdout] test test_streaming_simulation ... ok
[INFO] [stdout] test test_token_type ... ok
[INFO] [stdout] 
[INFO] [stdout] test result: ok. 6 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s
[INFO] [stdout] 
[INFO] [stderr]      Running tests/test_tokens.rs (/opt/rustwide/target/debug/deps/test_tokens-809aac19c07981df)
[INFO] [stdout] 
[INFO] [stdout] running 1 test
[INFO] [stdout] test test_vocabulary_loading ... ok
[INFO] [stderr]    Doc-tests shimmytok
[INFO] [stdout] 
[INFO] [stdout] test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.20s
[INFO] [stdout] 
[INFO] [stdout] 
[INFO] [stdout] running 11 tests
[INFO] [stdout] test src/lib.rs - Tokenizer::token_to_piece (line 364) - compile ... ok
[INFO] [stdout] test src/lib.rs - Tokenizer::encode (line 146) - compile ... ok
[INFO] [stdout] test src/lib.rs - Tokenizer::decode (line 187) - compile ... ok
[INFO] [stdout] test src/lib.rs - Tokenizer::model_type (line 250) - compile ... ok
[INFO] [stdout] test src/lib.rs - Tokenizer::token_type (line 396) - compile ... ok
[INFO] [stdout] test src/lib.rs - Tokenizer::decode_single (line 322) - compile ... ok
[INFO] [stdout] test src/lib.rs - Tokenizer::from_gguf_file (line 106) - compile ... ok
[INFO] [stdout] test src/lib.rs - Tokenizer (line 72) - compile ... ok
[INFO] [stdout] test src/lib.rs - Tokenizer::encode_batch (line 280) - compile ... ok
[INFO] [stdout] test src/lib.rs - Tokenizer::is_special_token (line 427) - compile ... ok
[INFO] [stdout] test src/lib.rs - (line 15) - compile ... ok
[INFO] [stdout] 
[INFO] [stdout] test result: ok. 11 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.28s
[INFO] [stdout] 
[INFO] running `Command { std: "docker" "inspect" "f43a227ed17df43a08efcfe90b81c16420c46396299cbd2cc37e4696de174e0f", kill_on_drop: false }`
[INFO] running `Command { std: "docker" "rm" "-f" "f43a227ed17df43a08efcfe90b81c16420c46396299cbd2cc37e4696de174e0f", kill_on_drop: false }`
[INFO] [stdout] f43a227ed17df43a08efcfe90b81c16420c46396299cbd2cc37e4696de174e0f