[INFO] fetching crate shimmytok 0.5.0...
[INFO] testing shimmytok-0.5.0 against master#c90bcb9571b7aab0d8beaa2ce8a998ffaf079d38 for pr-146098-8
[INFO] extracting crate shimmytok 0.5.0 into /workspace/builds/worker-7-tc1/source
[INFO] started tweaking crates.io crate shimmytok 0.5.0
[INFO] removed 0 missing examples
[INFO] removed 0 missing tests
[INFO] finished tweaking crates.io crate shimmytok 0.5.0
[INFO] tweaked toml for crates.io crate shimmytok 0.5.0 written to /workspace/builds/worker-7-tc1/source/Cargo.toml
[INFO] validating manifest of crates.io crate shimmytok 0.5.0 on toolchain c90bcb9571b7aab0d8beaa2ce8a998ffaf079d38
[INFO] running `Command { std: CARGO_HOME="/workspace/cargo-home" RUSTUP_HOME="/workspace/rustup-home" "/workspace/cargo-home/bin/cargo" "+c90bcb9571b7aab0d8beaa2ce8a998ffaf079d38" "metadata" "--manifest-path" "Cargo.toml" "--no-deps", kill_on_drop: false }`
[INFO] crate crates.io crate shimmytok 0.5.0 already has a lockfile, it will not be regenerated
[INFO] running `Command { std: CARGO_HOME="/workspace/cargo-home" RUSTUP_HOME="/workspace/rustup-home" "/workspace/cargo-home/bin/cargo" "+c90bcb9571b7aab0d8beaa2ce8a998ffaf079d38" "fetch" "--manifest-path" "Cargo.toml", kill_on_drop: false }`
[INFO] running `Command { std: "docker" "create" "-v" "/var/lib/crater-agent-workspace/builds/worker-7-tc1/target:/opt/rustwide/target:rw,Z" "-v" "/var/lib/crater-agent-workspace/builds/worker-7-tc1/source:/opt/rustwide/workdir:ro,Z" "-v" "/var/lib/crater-agent-workspace/cargo-home:/opt/rustwide/cargo-home:ro,Z" "-v" "/var/lib/crater-agent-workspace/rustup-home:/opt/rustwide/rustup-home:ro,Z" "-e" "SOURCE_DIR=/opt/rustwide/workdir" "-e" "CARGO_TARGET_DIR=/opt/rustwide/target" "-e" "CARGO_HOME=/opt/rustwide/cargo-home" "-e" "RUSTUP_HOME=/opt/rustwide/rustup-home" "-w" "/opt/rustwide/workdir" "-m" "1610612736" "--user" "0:0" "--network" "none" "ghcr.io/rust-lang/crates-build-env/linux@sha256:4848fb76d95f26979359cc7e45710b1dbc8f3acb7aeedee7c460d7702230f228" "/opt/rustwide/cargo-home/bin/cargo" "+c90bcb9571b7aab0d8beaa2ce8a998ffaf079d38" "metadata" "--no-deps" "--format-version=1", kill_on_drop: false }`
[INFO] [stdout] 18b612c862df303a8ed52a8e3a3d2156b6f1bd42c24c729e665bf2a522082b96
[INFO] running `Command { std: "docker" "start" "-a" "18b612c862df303a8ed52a8e3a3d2156b6f1bd42c24c729e665bf2a522082b96", kill_on_drop: false }`
[INFO] running `Command { std: "docker" "inspect" "18b612c862df303a8ed52a8e3a3d2156b6f1bd42c24c729e665bf2a522082b96", kill_on_drop: false }`
[INFO] running `Command { std: "docker" "rm" "-f" "18b612c862df303a8ed52a8e3a3d2156b6f1bd42c24c729e665bf2a522082b96", kill_on_drop: false }`
[INFO] [stdout] 18b612c862df303a8ed52a8e3a3d2156b6f1bd42c24c729e665bf2a522082b96
[INFO] running `Command { std: "docker" "create" "-v" "/var/lib/crater-agent-workspace/builds/worker-7-tc1/target:/opt/rustwide/target:rw,Z" "-v" "/var/lib/crater-agent-workspace/builds/worker-7-tc1/source:/opt/rustwide/workdir:ro,Z" "-v" "/var/lib/crater-agent-workspace/cargo-home:/opt/rustwide/cargo-home:ro,Z" "-v" "/var/lib/crater-agent-workspace/rustup-home:/opt/rustwide/rustup-home:ro,Z" "-e" "SOURCE_DIR=/opt/rustwide/workdir" "-e" "CARGO_TARGET_DIR=/opt/rustwide/target" "-e" "CARGO_INCREMENTAL=0" "-e" "RUST_BACKTRACE=full" "-e" "RUSTFLAGS=--cap-lints=forbid" "-e" "RUSTDOCFLAGS=--cap-lints=forbid" "-e" "CARGO_HOME=/opt/rustwide/cargo-home" "-e" "RUSTUP_HOME=/opt/rustwide/rustup-home" "-w" "/opt/rustwide/workdir" "-m" "1610612736" "--user" "0:0" "--network" "none" "ghcr.io/rust-lang/crates-build-env/linux@sha256:4848fb76d95f26979359cc7e45710b1dbc8f3acb7aeedee7c460d7702230f228" "/opt/rustwide/cargo-home/bin/cargo" "+c90bcb9571b7aab0d8beaa2ce8a998ffaf079d38" "build" "--frozen" "--message-format=json", kill_on_drop: false }`
[INFO] [stdout] 035749398c524e174329c32256b321f2d16bf414ab6cbf50fc1da356c8cfe0a7
[INFO] running `Command { std: "docker" "start" "-a" "035749398c524e174329c32256b321f2d16bf414ab6cbf50fc1da356c8cfe0a7", kill_on_drop: false }`
[INFO] [stderr]    Compiling aho-corasick v1.1.3
[INFO] [stderr]    Compiling syn v2.0.107
[INFO] [stderr]    Compiling regex-automata v0.4.13
[INFO] [stderr]    Compiling thiserror-impl v2.0.17
[INFO] [stderr]    Compiling thiserror v2.0.17
[INFO] [stderr]    Compiling fancy-regex v0.13.0
[INFO] [stderr]    Compiling regex v1.12.2
[INFO] [stderr]    Compiling shimmytok v0.5.0 (/opt/rustwide/workdir)
[INFO] [stderr]     Finished `dev` profile [unoptimized + debuginfo] target(s) in 14.70s
[INFO] running `Command { std: "docker" "inspect" "035749398c524e174329c32256b321f2d16bf414ab6cbf50fc1da356c8cfe0a7", kill_on_drop: false }`
[INFO] running `Command { std: "docker" "rm" "-f" "035749398c524e174329c32256b321f2d16bf414ab6cbf50fc1da356c8cfe0a7", kill_on_drop: false }`
[INFO] [stdout] 035749398c524e174329c32256b321f2d16bf414ab6cbf50fc1da356c8cfe0a7
[INFO] running `Command { std: "docker" "create" "-v" "/var/lib/crater-agent-workspace/builds/worker-7-tc1/target:/opt/rustwide/target:rw,Z" "-v" "/var/lib/crater-agent-workspace/builds/worker-7-tc1/source:/opt/rustwide/workdir:ro,Z" "-v" "/var/lib/crater-agent-workspace/cargo-home:/opt/rustwide/cargo-home:ro,Z" "-v" "/var/lib/crater-agent-workspace/rustup-home:/opt/rustwide/rustup-home:ro,Z" "-e" "SOURCE_DIR=/opt/rustwide/workdir" "-e" "CARGO_TARGET_DIR=/opt/rustwide/target" "-e" "CARGO_INCREMENTAL=0" "-e" "RUST_BACKTRACE=full" "-e" "RUSTFLAGS=--cap-lints=forbid" "-e" "RUSTDOCFLAGS=--cap-lints=forbid" "-e" "CARGO_HOME=/opt/rustwide/cargo-home" "-e" "RUSTUP_HOME=/opt/rustwide/rustup-home" "-w" "/opt/rustwide/workdir" "-m" "1610612736" "--user" "0:0" "--network" "none" "ghcr.io/rust-lang/crates-build-env/linux@sha256:4848fb76d95f26979359cc7e45710b1dbc8f3acb7aeedee7c460d7702230f228" "/opt/rustwide/cargo-home/bin/cargo" "+c90bcb9571b7aab0d8beaa2ce8a998ffaf079d38" "test" "--frozen" "--no-run" "--message-format=json", kill_on_drop: false }`
[INFO] [stdout] c481bdb9973cb89c0dd47272914044c38bd918be187a64f65b49bdc307e241b6
[INFO] running `Command { std: "docker" "start" "-a" "c481bdb9973cb89c0dd47272914044c38bd918be187a64f65b49bdc307e241b6", kill_on_drop: false }`
[INFO] [stderr]    Compiling zerocopy v0.8.27
[INFO] [stderr]    Compiling syn v2.0.107
[INFO] [stderr]    Compiling ciborium-io v0.2.2
[INFO] [stderr]    Compiling clap_lex v0.7.6
[INFO] [stderr]    Compiling plotters-backend v0.3.7
[INFO] [stderr]    Compiling anstyle v1.0.13
[INFO] [stderr]    Compiling serde_json v1.0.145
[INFO] [stderr]    Compiling itertools v0.10.5
[INFO] [stderr]    Compiling cast v0.3.0
[INFO] [stderr]    Compiling dirs-sys v0.4.1
[INFO] [stderr]    Compiling plotters-svg v0.3.7
[INFO] [stderr]    Compiling clap_builder v4.5.50
[INFO] [stderr]    Compiling is-terminal v0.4.16
[INFO] [stderr]    Compiling anes v0.1.6
[INFO] [stderr]    Compiling oorandom v11.1.5
[INFO] [stderr]    Compiling tempfile v3.23.0
[INFO] [stderr]    Compiling plotters v0.3.7
[INFO] [stderr]    Compiling dirs v5.0.1
[INFO] [stderr]    Compiling criterion-plot v0.5.0
[INFO] [stderr]    Compiling clap v4.5.50
[INFO] [stderr]    Compiling zerocopy-derive v0.8.27
[INFO] [stderr]    Compiling serde_derive v1.0.228
[INFO] [stderr]    Compiling thiserror-impl v2.0.17
[INFO] [stderr]    Compiling thiserror v2.0.17
[INFO] [stderr]    Compiling shimmytok v0.5.0 (/opt/rustwide/workdir)
[INFO] [stderr]    Compiling half v2.7.1
[INFO] [stderr]    Compiling ciborium-ll v0.2.2
[INFO] [stderr]    Compiling serde v1.0.228
[INFO] [stderr]    Compiling tinytemplate v1.2.1
[INFO] [stderr]    Compiling ciborium v0.2.2
[INFO] [stderr]    Compiling criterion v0.5.1
[INFO] [stderr]     Finished `test` profile [unoptimized + debuginfo] target(s) in 40.72s
[INFO] running `Command { std: "docker" "inspect" "c481bdb9973cb89c0dd47272914044c38bd918be187a64f65b49bdc307e241b6", kill_on_drop: false }`
[INFO] running `Command { std: "docker" "rm" "-f" "c481bdb9973cb89c0dd47272914044c38bd918be187a64f65b49bdc307e241b6", kill_on_drop: false }`
[INFO] [stdout] c481bdb9973cb89c0dd47272914044c38bd918be187a64f65b49bdc307e241b6
[INFO] running `Command { std: "docker" "create" "-v" "/var/lib/crater-agent-workspace/builds/worker-7-tc1/target:/opt/rustwide/target:rw,Z" "-v" "/var/lib/crater-agent-workspace/builds/worker-7-tc1/source:/opt/rustwide/workdir:ro,Z" "-v" "/var/lib/crater-agent-workspace/cargo-home:/opt/rustwide/cargo-home:ro,Z" "-v" "/var/lib/crater-agent-workspace/rustup-home:/opt/rustwide/rustup-home:ro,Z" "-e" "SOURCE_DIR=/opt/rustwide/workdir" "-e" "CARGO_TARGET_DIR=/opt/rustwide/target" "-e" "CARGO_INCREMENTAL=0" "-e" "RUST_BACKTRACE=full" "-e" "RUSTFLAGS=--cap-lints=forbid" "-e" "RUSTDOCFLAGS=--cap-lints=forbid" "-e" "CARGO_HOME=/opt/rustwide/cargo-home" "-e" "RUSTUP_HOME=/opt/rustwide/rustup-home" "-w" "/opt/rustwide/workdir" "-m" "1610612736" "--user" "0:0" "--network" "none" "ghcr.io/rust-lang/crates-build-env/linux@sha256:4848fb76d95f26979359cc7e45710b1dbc8f3acb7aeedee7c460d7702230f228" "/opt/rustwide/cargo-home/bin/cargo" "+c90bcb9571b7aab0d8beaa2ce8a998ffaf079d38" "test" "--frozen", kill_on_drop: false }`
[INFO] [stdout] a9928dd2c6f82835d3ec8b4959a9ba538496f2be5bffd4976841c1e6159dc07f
[INFO] running `Command { std: "docker" "start" "-a" "a9928dd2c6f82835d3ec8b4959a9ba538496f2be5bffd4976841c1e6159dc07f", kill_on_drop: false }`
[INFO] [stderr]     Finished `test` profile [unoptimized + debuginfo] target(s) in 0.17s
[INFO] [stderr]      Running unittests src/lib.rs (/opt/rustwide/target/debug/deps/shimmytok-c1b06137e5169834)
[INFO] [stderr]      Running tests/debug_merges.rs (/opt/rustwide/target/debug/deps/debug_merges-675da8903885cc9a)
[INFO] [stderr]      Running tests/test_basic.rs (/opt/rustwide/target/debug/deps/test_basic-3162141013e05cc3)
[INFO] [stderr]      Running tests/test_bpe.rs (/opt/rustwide/target/debug/deps/test_bpe-d5505e50ede00215)
[INFO] [stderr]      Running tests/test_bpe_edge_cases.rs (/opt/rustwide/target/debug/deps/test_bpe_edge_cases-51962bd8888998a4)
[INFO] [stderr]      Running tests/test_bpe_validation.rs (/opt/rustwide/target/debug/deps/test_bpe_validation-cf0f31322eb4a08d)
[INFO] [stderr]      Running tests/test_comprehensive.rs (/opt/rustwide/target/debug/deps/test_comprehensive-618e71917fe28aa0)
[INFO] [stdout] 
[INFO] [stdout] running 2 tests
[INFO] [stdout] test byte_encoder::tests::test_space_encoding ... ok
[INFO] [stdout] test byte_encoder::tests::test_hello ... ok
[INFO] [stdout] 
[INFO] [stdout] test result: ok. 2 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s
[INFO] [stdout] 
[INFO] [stdout] 
[INFO] [stdout] running 1 test
[INFO] [stdout] test debug_gpt2_merges ... ignored
[INFO] [stdout] 
[INFO] [stdout] test result: ok. 0 passed; 0 failed; 1 ignored; 0 measured; 0 filtered out; finished in 0.00s
[INFO] [stdout] 
[INFO] [stdout] 
[INFO] [stdout] running 2 tests
[INFO] [stdout] test test_encode_batch ... ok
[INFO] [stdout] test test_load_gguf ... ok
[INFO] [stdout] 
[INFO] [stdout] test result: ok. 2 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s
[INFO] [stdout] 
[INFO] [stdout] 
[INFO] [stdout] running 1 test
[INFO] [stdout] test test_gpt2_tokenization ... ok
[INFO] [stdout] 
[INFO] [stdout] test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s
[INFO] [stdout] 
[INFO] [stdout] 
[INFO] [stdout] running 8 tests
[INFO] [stdout] test test_empty_string ... ok
[INFO] [stdout] test test_long_string ... ok
[INFO] [stdout] test test_unicode_cjk ... ok
[INFO] [stdout] test test_single_char ... ok
[INFO] [stdout] test test_newlines_and_tabs ... ok
[INFO] [stdout] test test_unicode_emoji ... ok
[INFO] [stdout] test test_multiple_spaces ... ok
[INFO] [stdout] test test_special_chars ... ok
[INFO] [stdout] 
[INFO] [stdout] test result: ok. 8 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.01s
[INFO] [stdout] 
[INFO] [stdout] 
[INFO] [stdout] running 6 tests
[INFO] [stdout] test test_deepseek_coder_validation ... ignored
[INFO] [stdout] test test_deepseek_llm_validation ... ignored
[INFO] [stdout] test test_gpt2_validation ... ignored
[INFO] [stdout] test test_phi2_validation ... ignored
[INFO] [stdout] test test_qwen2_validation ... ignored
[INFO] [stdout] test test_starcoder_validation ... ignored
[INFO] [stdout] 
[INFO] [stdout] test result: ok. 0 passed; 0 failed; 6 ignored; 0 measured; 0 filtered out; finished in 0.00s
[INFO] [stdout] 
[INFO] [stdout] 
[INFO] [stdout] running 1 test
[INFO] [stderr]      Running tests/test_debug.rs (/opt/rustwide/target/debug/deps/test_debug-db617b491ad4755e)
[INFO] [stdout] test test_against_llama_cpp ... ok
[INFO] [stdout] 
[INFO] [stdout] test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s
[INFO] [stdout] 
[INFO] [stdout] 
[INFO] [stdout] running 1 test
[INFO] [stdout] test test_debug_tokens ... ok
[INFO] [stdout] 
[INFO] [stdout] test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.01s
[INFO] [stdout] 
[INFO] [stderr]      Running tests/test_detailed.rs (/opt/rustwide/target/debug/deps/test_detailed-0effd82fd9e514ec)
[INFO] [stdout] 
[INFO] [stdout] running 1 test
[INFO] [stdout] test test_detailed_merge ... ok
[INFO] [stdout] 
[INFO] [stdout] test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s
[INFO] [stdout] 
[INFO] [stderr]      Running tests/test_error_handling.rs (/opt/rustwide/target/debug/deps/test_error_handling-10fd0d98b4188de2)
[INFO] [stdout] 
[INFO] [stdout] running 7 tests
[INFO] [stdout] test test_decode_with_special_tokens ... ok
[INFO] [stdout] test test_empty_input ... ok
[INFO] [stdout] test test_invalid_token_id ... ok
[INFO] [stdout] test test_max_token_validation ... ok
[INFO] [stdout] test test_missing_file ... ok
[INFO] [stdout] test test_round_trip_fuzz ... ok
[INFO] [stdout] test test_very_large_input ... ok
[INFO] [stdout] 
[INFO] [stdout] test result: ok. 7 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s
[INFO] [stdout] 
[INFO] [stderr]      Running tests/test_merge_debug.rs (/opt/rustwide/target/debug/deps/test_merge_debug-004e4f2c4f1a9a7b)
[INFO] [stderr]      Running tests/test_streaming.rs (/opt/rustwide/target/debug/deps/test_streaming-956d730b033d1834)
[INFO] [stdout] 
[INFO] [stdout] running 1 test
[INFO] [stdout] test test_merge_sequence ... ok
[INFO] [stdout] 
[INFO] [stdout] test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s
[INFO] [stdout] 
[INFO] [stdout] 
[INFO] [stdout] running 6 tests
[INFO] [stdout] test test_decode_single ... ok
[INFO] [stdout] test test_streaming_simulation ... ok
[INFO] [stdout] test test_token_to_piece ... ok
[INFO] [stdout] test test_decode_single_with_special_tokens ... ok
[INFO] [stdout] test test_is_special_token ... ok
[INFO] [stdout] test test_token_type ... ok
[INFO] [stdout] 
[INFO] [stdout] test result: ok. 6 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.01s
[INFO] [stdout] 
[INFO] [stderr]      Running tests/test_tokens.rs (/opt/rustwide/target/debug/deps/test_tokens-949e670f080e1d0c)
[INFO] [stdout] 
[INFO] [stdout] running 1 test
[INFO] [stdout] test test_vocabulary_loading ... ok
[INFO] [stdout] 
[INFO] [stdout] test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s
[INFO] [stdout] 
[INFO] [stderr]    Doc-tests shimmytok
[INFO] [stdout] 
[INFO] [stdout] running 11 tests
[INFO] [stdout] test src/lib.rs - Tokenizer (line 72) - compile ... ok
[INFO] [stdout] test src/lib.rs - Tokenizer::from_gguf_file (line 106) - compile ... ok
[INFO] [stdout] test src/lib.rs - (line 15) - compile ... ok
[INFO] [stdout] test src/lib.rs - Tokenizer::encode_batch (line 280) - compile ... ok
[INFO] [stdout] test src/lib.rs - Tokenizer::model_type (line 250) - compile ... ok
[INFO] [stdout] test src/lib.rs - Tokenizer::decode_single (line 322) - compile ... ok
[INFO] [stdout] test src/lib.rs - Tokenizer::decode (line 187) - compile ... ok
[INFO] [stdout] test src/lib.rs - Tokenizer::is_special_token (line 427) - compile ... ok
[INFO] [stdout] test src/lib.rs - Tokenizer::token_to_piece (line 364) - compile ... ok
[INFO] [stdout] test src/lib.rs - Tokenizer::encode (line 146) - compile ... ok
[INFO] [stdout] test src/lib.rs - Tokenizer::token_type (line 396) - compile ... ok
[INFO] [stdout] 
[INFO] [stdout] test result: ok. 11 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.47s
[INFO] [stdout] 
[INFO] running `Command { std: "docker" "inspect" "a9928dd2c6f82835d3ec8b4959a9ba538496f2be5bffd4976841c1e6159dc07f", kill_on_drop: false }`
[INFO] running `Command { std: "docker" "rm" "-f" "a9928dd2c6f82835d3ec8b4959a9ba538496f2be5bffd4976841c1e6159dc07f", kill_on_drop: false }`
[INFO] [stdout] a9928dd2c6f82835d3ec8b4959a9ba538496f2be5bffd4976841c1e6159dc07f