[INFO] cloning repository https://github.com/deltartificial/tokenizer
[INFO] running `Command { std: "git" "-c" "credential.helper=" "-c" "credential.helper=/workspace/cargo-home/bin/git-credential-null" "clone" "--bare" "https://github.com/deltartificial/tokenizer" "/workspace/cache/git-repos/https%3A%2F%2Fgithub.com%2Fdeltartificial%2Ftokenizer", kill_on_drop: false }`
[INFO] [stderr] Cloning into bare repository '/workspace/cache/git-repos/https%3A%2F%2Fgithub.com%2Fdeltartificial%2Ftokenizer'...
[INFO] running `Command { std: "git" "rev-parse" "HEAD", kill_on_drop: false }`
[INFO] [stdout] 8782c3753f6cf34c0fa7cc617233b21ad6824b20
[INFO] checking deltartificial/tokenizer against master#80b898258da78fdd1262438126aa0cf90e395f0c for pr-149195-4
[INFO] running `Command { std: "git" "clone" "/workspace/cache/git-repos/https%3A%2F%2Fgithub.com%2Fdeltartificial%2Ftokenizer" "/workspace/builds/worker-2-tc1/source", kill_on_drop: false }`
[INFO] [stderr] Cloning into '/workspace/builds/worker-2-tc1/source'...
[INFO] [stderr] done.
[INFO] started tweaking git repo https://github.com/deltartificial/tokenizer
[INFO] finished tweaking git repo https://github.com/deltartificial/tokenizer
[INFO] tweaked toml for git repo https://github.com/deltartificial/tokenizer written to /workspace/builds/worker-2-tc1/source/Cargo.toml
[INFO] validating manifest of git repo https://github.com/deltartificial/tokenizer on toolchain 80b898258da78fdd1262438126aa0cf90e395f0c
[INFO] running `Command { std: CARGO_HOME="/workspace/cargo-home" RUSTUP_HOME="/workspace/rustup-home" "/workspace/cargo-home/bin/cargo" "+80b898258da78fdd1262438126aa0cf90e395f0c" "metadata" "--manifest-path" "Cargo.toml" "--no-deps", kill_on_drop: false }`
[INFO] crate git repo https://github.com/deltartificial/tokenizer already has a lockfile, it will not be regenerated
[INFO] running `Command { std: CARGO_HOME="/workspace/cargo-home" RUSTUP_HOME="/workspace/rustup-home" "/workspace/cargo-home/bin/cargo" "+80b898258da78fdd1262438126aa0cf90e395f0c" "fetch" "--manifest-path" "Cargo.toml", kill_on_drop: false }`
[INFO] [stderr]     Updating crates.io index
[INFO] [stderr]  Downloading crates ...
[INFO] [stderr]   Downloaded rayon-cond v0.3.0
[INFO] [stderr]   Downloaded libflate_lz77 v2.1.0
[INFO] [stderr]   Downloaded hf-hub v0.4.2
[INFO] [stderr]   Downloaded macro_rules_attribute v0.2.0
[INFO] [stderr]   Downloaded macro_rules_attribute-proc_macro v0.2.0
[INFO] [stderr]   Downloaded monostate-impl v0.1.14
[INFO] [stderr]   Downloaded datasize v0.2.15
[INFO] [stderr]   Downloaded fax_derive v0.1.0
[INFO] [stderr]   Downloaded tuple v0.5.2
[INFO] [stderr]   Downloaded fax v0.2.4
[INFO] [stderr]   Downloaded istring v0.3.4
[INFO] [stderr]   Downloaded libflate v2.1.0
[INFO] [stderr]   Downloaded html2text v0.6.0
[INFO] [stderr]   Downloaded globalcache v0.2.4
[INFO] [stderr]   Downloaded tokenizers v0.21.1
[INFO] [stderr]   Downloaded jiff-static v0.2.5
[INFO] [stderr]   Downloaded datasize_derive v0.2.15
[INFO] [stderr]   Downloaded pdf v0.9.0
[INFO] [stderr]   Downloaded onig v6.4.0
[INFO] [stderr]   Downloaded dary_heap v0.3.7
[INFO] [stderr]   Downloaded monostate v0.1.14
[INFO] [stderr]   Downloaded pdf_derive v0.2.0
[INFO] [stderr]   Downloaded onig_sys v69.8.1
[INFO] [stderr]   Downloaded jiff v0.2.5
[INFO] running `Command { std: "docker" "create" "-v" "/var/lib/crater-agent-workspace/builds/worker-2-tc1/target:/opt/rustwide/target:rw,Z" "-v" "/var/lib/crater-agent-workspace/builds/worker-2-tc1/source:/opt/rustwide/workdir:ro,Z" "-v" "/var/lib/crater-agent-workspace/cargo-home:/opt/rustwide/cargo-home:ro,Z" "-v" "/var/lib/crater-agent-workspace/rustup-home:/opt/rustwide/rustup-home:ro,Z" "-e" "SOURCE_DIR=/opt/rustwide/workdir" "-e" "CARGO_TARGET_DIR=/opt/rustwide/target" "-e" "CARGO_HOME=/opt/rustwide/cargo-home" "-e" "RUSTUP_HOME=/opt/rustwide/rustup-home" "-w" "/opt/rustwide/workdir" "-m" "1610612736" "--user" "0:0" "--network" "none" "ghcr.io/rust-lang/crates-build-env/linux@sha256:29356a839a4d14451438f794ce8414e707d72405f44418718d724d22562fe786" "/opt/rustwide/cargo-home/bin/cargo" "+80b898258da78fdd1262438126aa0cf90e395f0c" "metadata" "--no-deps" "--format-version=1", kill_on_drop: false }`
[INFO] [stdout] 25b1e19c3b7e4779be721fafa7a3ce207d73d2165a4500282e79626a4e870560
[INFO] running `Command { std: "docker" "start" "-a" "25b1e19c3b7e4779be721fafa7a3ce207d73d2165a4500282e79626a4e870560", kill_on_drop: false }`
[INFO] running `Command { std: "docker" "inspect" "25b1e19c3b7e4779be721fafa7a3ce207d73d2165a4500282e79626a4e870560", kill_on_drop: false }`
[INFO] running `Command { std: "docker" "rm" "-f" "25b1e19c3b7e4779be721fafa7a3ce207d73d2165a4500282e79626a4e870560", kill_on_drop: false }`
[INFO] [stdout] 25b1e19c3b7e4779be721fafa7a3ce207d73d2165a4500282e79626a4e870560
[INFO] running `Command { std: "docker" "create" "-v" "/var/lib/crater-agent-workspace/builds/worker-2-tc1/target:/opt/rustwide/target:rw,Z" "-v" "/var/lib/crater-agent-workspace/builds/worker-2-tc1/source:/opt/rustwide/workdir:ro,Z" "-v" "/var/lib/crater-agent-workspace/cargo-home:/opt/rustwide/cargo-home:ro,Z" "-v" "/var/lib/crater-agent-workspace/rustup-home:/opt/rustwide/rustup-home:ro,Z" "-e" "SOURCE_DIR=/opt/rustwide/workdir" "-e" "CARGO_TARGET_DIR=/opt/rustwide/target" "-e" "CARGO_INCREMENTAL=0" "-e" "RUST_BACKTRACE=full" "-e" "RUSTFLAGS=--cap-lints=forbid" "-e" "RUSTDOCFLAGS=--cap-lints=forbid" "-e" "CARGO_HOME=/opt/rustwide/cargo-home" "-e" "RUSTUP_HOME=/opt/rustwide/rustup-home" "-w" "/opt/rustwide/workdir" "-m" "1610612736" "--user" "0:0" "--network" "none" "ghcr.io/rust-lang/crates-build-env/linux@sha256:29356a839a4d14451438f794ce8414e707d72405f44418718d724d22562fe786" "/opt/rustwide/cargo-home/bin/cargo" "+80b898258da78fdd1262438126aa0cf90e395f0c" "check" "--frozen" "--all" "--all-targets" "--message-format=json", kill_on_drop: false }`
[INFO] [stdout] b213199dd86eabad6aa33bf5ee9c3ed5beeac7b4fa2e9414bf31dd92beadb41a
[INFO] running `Command { std: "docker" "start" "-a" "b213199dd86eabad6aa33bf5ee9c3ed5beeac7b4fa2e9414bf31dd92beadb41a", kill_on_drop: false }`
[INFO] [stderr]    Compiling quote v1.0.40
[INFO] [stderr]    Compiling cc v1.2.18
[INFO] [stderr]    Compiling libc v0.2.171
[INFO] [stderr]    Compiling syn v1.0.109
[INFO] [stderr]    Compiling zerocopy v0.8.24
[INFO] [stderr]     Checking getrandom v0.2.15
[INFO] [stderr]    Compiling phf_shared v0.10.0
[INFO] [stderr]     Checking parking_lot_core v0.9.10
[INFO] [stderr]     Checking parking_lot v0.12.3
[INFO] [stderr]     Checking block-padding v0.3.3
[INFO] [stderr]    Compiling syn v2.0.100
[INFO] [stderr]     Checking aho-corasick v1.1.3
[INFO] [stderr]     Checking zerocopy v0.7.35
[INFO] [stderr]    Compiling rustls v0.23.25
[INFO] [stderr]     Checking miniz_oxide v0.8.7
[INFO] [stderr]     Checking ahash v0.8.11
[INFO] [stderr]    Compiling rand_core v0.6.4
[INFO] [stderr]     Checking phf v0.10.1
[INFO] [stderr]     Checking inout v0.1.4
[INFO] [stderr]     Checking dirs-sys v0.4.1
[INFO] [stderr]     Checking console v0.15.11
[INFO] [stderr]     Checking hashbrown v0.14.5
[INFO] [stderr]     Checking regex-automata v0.4.9
[INFO] [stderr]     Checking rand_chacha v0.3.1
[INFO] [stderr]     Checking socks v0.3.4
[INFO] [stderr]     Checking cipher v0.4.4
[INFO] [stderr]     Checking flate2 v1.1.1
[INFO] [stderr]     Checking webpki-roots v0.26.8
[INFO] [stderr]    Compiling ppv-lite86 v0.2.21
[INFO] [stderr]     Checking core2 v0.4.0
[INFO] [stderr]     Checking rle-decode-fast v1.0.3
[INFO] [stderr]     Checking indicatif v0.17.11
[INFO] [stderr]     Checking libflate_lz77 v2.1.0
[INFO] [stderr]     Checking nom v7.1.3
[INFO] [stderr]     Checking rayon v1.10.0
[INFO] [stderr]    Compiling ring v0.17.14
[INFO] [stderr]    Compiling rand v0.8.5
[INFO] [stderr]    Compiling onig_sys v69.8.1
[INFO] [stderr]    Compiling phf_generator v0.10.0
[INFO] [stderr]    Compiling phf_generator v0.11.3
[INFO] [stderr]    Compiling string_cache_codegen v0.5.4
[INFO] [stderr]    Compiling phf_codegen v0.10.0
[INFO] [stderr]     Checking regex v1.11.1
[INFO] [stderr]    Compiling esaxx-rs v0.1.10
[INFO] [stderr]    Compiling markup5ever v0.11.0
[INFO] [stderr]     Checking unicode-normalization v0.1.24
[INFO] [stderr]     Checking dirs v5.0.1
[INFO] [stderr]     Checking itertools v0.11.0
[INFO] [stderr]     Checking dary_heap v0.3.7
[INFO] [stderr]    Compiling macro_rules_attribute-proc_macro v0.2.0
[INFO] [stderr]    Compiling synstructure v0.13.1
[INFO] [stderr]    Compiling darling_core v0.20.11
[INFO] [stderr]    Compiling anyhow v1.0.97
[INFO] [stderr]     Checking indexmap v2.9.0
[INFO] [stderr]     Checking clap_builder v4.5.35
[INFO] [stderr]     Checking rayon-cond v0.3.0
[INFO] [stderr]     Checking macro_rules_attribute v0.2.0
[INFO] [stderr]     Checking libflate v2.1.0
[INFO] [stderr]     Checking rustls-webpki v0.103.1
[INFO] [stderr]     Checking stringprep v0.1.5
[INFO] [stderr]     Checking env_filter v0.1.3
[INFO] [stderr]     Checking sha2 v0.10.8
[INFO] [stderr]    Compiling zerofrom-derive v0.1.6
[INFO] [stderr]    Compiling serde_derive v1.0.219
[INFO] [stderr]    Compiling yoke-derive v0.7.5
[INFO] [stderr]    Compiling zerovec-derive v0.10.3
[INFO] [stderr]    Compiling displaydoc v0.2.5
[INFO] [stderr]    Compiling icu_provider_macros v1.5.0
[INFO] [stderr]    Compiling html5ever v0.26.0
[INFO] [stderr]    Compiling datasize_derive v0.2.15
[INFO] [stderr]    Compiling thiserror-impl v2.0.12
[INFO] [stderr]    Compiling fax_derive v0.1.0
[INFO] [stderr]    Compiling darling_macro v0.20.11
[INFO] [stderr]     Checking datasize v0.2.15
[INFO] [stderr]     Checking zerofrom v0.1.6
[INFO] [stderr]    Compiling snafu-derive v0.7.5
[INFO] [stderr]     Checking yoke v0.7.5
[INFO] [stderr]    Compiling darling v0.20.11
[INFO] [stderr]    Compiling derive_builder_core v0.20.2
[INFO] [stderr]     Checking zerovec v0.10.4
[INFO] [stderr]    Compiling async-trait v0.1.88
[INFO] [stderr]     Checking thiserror v2.0.12
[INFO] [stderr]    Compiling monostate-impl v0.1.14
[INFO] [stderr]     Checking tinystr v0.7.6
[INFO] [stderr]     Checking icu_collections v1.5.0
[INFO] [stderr]     Checking icu_locid v1.5.0
[INFO] [stderr]    Compiling derive_builder_macro v0.20.2
[INFO] [stderr]     Checking fax v0.2.4
[INFO] [stderr]     Checking derive_builder v0.20.2
[INFO] [stderr]     Checking istring v0.3.4
[INFO] [stderr]    Compiling pdf_derive v0.2.0
[INFO] [stderr]     Checking onig v6.4.0
[INFO] [stderr]    Compiling tokio-macros v2.5.0
[INFO] [stderr]    Compiling clap_derive v4.5.32
[INFO] [stderr]     Checking deflate v1.0.0
[INFO] [stderr]     Checking icu_provider v1.5.0
[INFO] [stderr]     Checking aes v0.8.4
[INFO] [stderr]     Checking cbc v0.1.2
[INFO] [stderr]     Checking icu_locid_transform v1.5.0
[INFO] [stderr]     Checking itertools v0.10.5
[INFO] [stderr]     Checking itertools v0.13.0
[INFO] [stderr]     Checking signal-hook-registry v1.4.2
[INFO] [stderr]     Checking unicode-normalization-alignments v0.1.12
[INFO] [stderr]     Checking snafu v0.7.5
[INFO] [stderr]     Checking mio v1.0.3
[INFO] [stderr]     Checking serde v1.0.219
[INFO] [stderr]     Checking socket2 v0.5.9
[INFO] [stderr]     Checking icu_properties v1.5.1
[INFO] [stderr]     Checking weezl v0.1.8
[INFO] [stderr]     Checking unicode_categories v0.1.1
[INFO] [stderr]     Checking md5 v0.7.0
[INFO] [stderr]     Checking jiff v0.2.5
[INFO] [stderr]     Checking tokio v1.44.2
[INFO] [stderr]     Checking clap v4.5.35
[INFO] [stderr]     Checking icu_normalizer v1.5.0
[INFO] [stderr]     Checking string_cache v0.8.9
[INFO] [stderr]     Checking serde_json v1.0.140
[INFO] [stderr]     Checking tuple v0.5.2
[INFO] [stderr]     Checking monostate v0.1.14
[INFO] [stderr]     Checking spm_precompiled v0.1.4
[INFO] [stderr]     Checking idna_adapter v1.2.0
[INFO] [stderr]     Checking idna v1.0.3
[INFO] [stderr]     Checking url v2.5.4
[INFO] [stderr]     Checking env_logger v0.11.8
[INFO] [stderr]     Checking xml5ever v0.17.0
[INFO] [stderr]     Checking ureq v2.12.1
[INFO] [stderr]     Checking html2text v0.6.0
[INFO] [stderr]     Checking hf-hub v0.4.2
[INFO] [stderr]     Checking globalcache v0.2.4
[INFO] [stderr]     Checking tokenizers v0.21.1
[INFO] [stderr]     Checking pdf v0.9.0
[INFO] [stderr]     Checking tokenizer v0.1.0 (/opt/rustwide/workdir)
[INFO] [stdout] warning: associated function `new` is never used
[INFO] [stdout]   --> src/infrastructure/token_counter.rs:17:12
[INFO] [stdout]    |
[INFO] [stdout] 16 | impl HuggingFaceTokenizerService {
[INFO] [stdout]    | -------------------------------- associated function in this implementation
[INFO] [stdout] 17 |     pub fn new() -> Result<Self> {
[INFO] [stdout]    |            ^^^
[INFO] [stdout]    |
[INFO] [stdout]    = note: `#[warn(dead_code)]` (part of `#[warn(unused)]`) on by default
[INFO] [stdout] 
[INFO] [stdout] 
[INFO] [stdout] warning: associated function `new` is never used
[INFO] [stdout]   --> src/infrastructure/token_counter.rs:17:12
[INFO] [stdout]    |
[INFO] [stdout] 16 | impl HuggingFaceTokenizerService {
[INFO] [stdout]    | -------------------------------- associated function in this implementation
[INFO] [stdout] 17 |     pub fn new() -> Result<Self> {
[INFO] [stdout]    |            ^^^
[INFO] [stdout]    |
[INFO] [stdout]    = note: `#[warn(dead_code)]` (part of `#[warn(unused)]`) on by default
[INFO] [stdout] 
[INFO] [stdout] 
[INFO] [stderr]     Finished `dev` profile [unoptimized + debuginfo] target(s) in 1m 35s
[INFO] [stderr] warning: the following packages contain code that will be rejected by a future version of Rust: pdf v0.9.0
[INFO] [stderr] note: to see what the problems were, use the option `--future-incompat-report`, or run `cargo report future-incompatibilities --id 4`
[INFO] running `Command { std: "docker" "inspect" "b213199dd86eabad6aa33bf5ee9c3ed5beeac7b4fa2e9414bf31dd92beadb41a", kill_on_drop: false }`
[INFO] running `Command { std: "docker" "rm" "-f" "b213199dd86eabad6aa33bf5ee9c3ed5beeac7b4fa2e9414bf31dd92beadb41a", kill_on_drop: false }`
[INFO] [stdout] b213199dd86eabad6aa33bf5ee9c3ed5beeac7b4fa2e9414bf31dd92beadb41a