[INFO] fetching crate fibpetokenizer 0.1.0...
[INFO] testing fibpetokenizer-0.1.0 against beta-2026-03-05 for beta-1.95-1
[INFO] extracting crate fibpetokenizer 0.1.0 into /workspace/builds/worker-2-tc2/source
[INFO] started tweaking crates.io crate fibpetokenizer 0.1.0
[INFO] removed 0 missing examples
[INFO] finished tweaking crates.io crate fibpetokenizer 0.1.0
[INFO] tweaked toml for crates.io crate fibpetokenizer 0.1.0 written to /workspace/builds/worker-2-tc2/source/Cargo.toml
[INFO] validating manifest of crates.io crate fibpetokenizer 0.1.0 on toolchain beta-2026-03-05
[INFO] running `Command { std: CARGO_HOME="/workspace/cargo-home" RUSTUP_HOME="/workspace/rustup-home" "/workspace/cargo-home/bin/cargo" "+beta-2026-03-05" "metadata" "--manifest-path" "Cargo.toml" "--no-deps", kill_on_drop: false }`
[INFO] crate crates.io crate fibpetokenizer 0.1.0 already has a lockfile, it will not be regenerated
[INFO] running `Command { std: CARGO_HOME="/workspace/cargo-home" RUSTUP_HOME="/workspace/rustup-home" "/workspace/cargo-home/bin/cargo" "+beta-2026-03-05" "fetch" "--manifest-path" "Cargo.toml", kill_on_drop: false }`
[INFO] running `Command { std: "docker" "create" "-v" "/var/lib/crater-agent-workspace/builds/worker-2-tc2/target:/opt/rustwide/target:rw,Z" "-v" "/var/lib/crater-agent-workspace/builds/worker-2-tc2/source:/opt/rustwide/workdir:ro,Z" "-v" "/var/lib/crater-agent-workspace/cargo-home:/opt/rustwide/cargo-home:ro,Z" "-v" "/var/lib/crater-agent-workspace/rustup-home:/opt/rustwide/rustup-home:ro,Z" "-e" "SOURCE_DIR=/opt/rustwide/workdir" "-e" "CARGO_TARGET_DIR=/opt/rustwide/target" "-e" "CARGO_HOME=/opt/rustwide/cargo-home" "-e" "RUSTUP_HOME=/opt/rustwide/rustup-home" "-w" "/opt/rustwide/workdir" "-m" "1610612736" "--user" "0:0" "--network" "none" "ghcr.io/rust-lang/crates-build-env/linux@sha256:d429b63d4308055ea97f60fb1d3dfca48854a00942f1bd2ad806beaf015945ec" "/opt/rustwide/cargo-home/bin/cargo" "+beta-2026-03-05" "metadata" "--no-deps" "--format-version=1", kill_on_drop: false }`
[INFO] [stdout] ffdb877fd410faca68be424f42cb6932ea8ac89fce2f6d02bc9b1037ed9d07fd
[INFO] running `Command { std: "docker" "start" "-a" "ffdb877fd410faca68be424f42cb6932ea8ac89fce2f6d02bc9b1037ed9d07fd", kill_on_drop: false }`
[INFO] running `Command { std: "docker" "inspect" "ffdb877fd410faca68be424f42cb6932ea8ac89fce2f6d02bc9b1037ed9d07fd", kill_on_drop: false }`
[INFO] running `Command { std: "docker" "rm" "-f" "ffdb877fd410faca68be424f42cb6932ea8ac89fce2f6d02bc9b1037ed9d07fd", kill_on_drop: false }`
[INFO] [stdout] ffdb877fd410faca68be424f42cb6932ea8ac89fce2f6d02bc9b1037ed9d07fd
[INFO] running `Command { std: "docker" "create" "-v" "/var/lib/crater-agent-workspace/builds/worker-2-tc2/target:/opt/rustwide/target:rw,Z" "-v" "/var/lib/crater-agent-workspace/builds/worker-2-tc2/source:/opt/rustwide/workdir:ro,Z" "-v" "/var/lib/crater-agent-workspace/cargo-home:/opt/rustwide/cargo-home:ro,Z" "-v" "/var/lib/crater-agent-workspace/rustup-home:/opt/rustwide/rustup-home:ro,Z" "-e" "SOURCE_DIR=/opt/rustwide/workdir" "-e" "CARGO_TARGET_DIR=/opt/rustwide/target" "-e" "CARGO_INCREMENTAL=0" "-e" "RUST_BACKTRACE=full" "-e" "RUSTFLAGS=--cap-lints=warn" "-e" "RUSTDOCFLAGS=--cap-lints=warn" "-e" "CARGO_HOME=/opt/rustwide/cargo-home" "-e" "RUSTUP_HOME=/opt/rustwide/rustup-home" "-w" "/opt/rustwide/workdir" "-m" "1610612736" "--user" "0:0" "--network" "none" "ghcr.io/rust-lang/crates-build-env/linux@sha256:d429b63d4308055ea97f60fb1d3dfca48854a00942f1bd2ad806beaf015945ec" "/opt/rustwide/cargo-home/bin/cargo" "+beta-2026-03-05" "build" "--frozen" "--message-format=json", kill_on_drop: false }`
[INFO] [stdout] 917799d9da54ae0740160014a5197e3388f5293d73822a9ca4db4fc5d705a842
[INFO] running `Command { std: "docker" "start" "-a" "917799d9da54ae0740160014a5197e3388f5293d73822a9ca4db4fc5d705a842", kill_on_drop: false }`
[INFO] [stderr]    Compiling serde_core v1.0.226
[INFO] [stderr]    Compiling serde v1.0.226
[INFO] [stderr]    Compiling aho-corasick v1.1.3
[INFO] [stderr]    Compiling syn v2.0.106
[INFO] [stderr]    Compiling rayon v1.10.0
[INFO] [stderr]    Compiling serde_derive v1.0.226
[INFO] [stderr]    Compiling thiserror-impl v1.0.69
[INFO] [stderr]    Compiling thiserror v1.0.69
[INFO] [stderr]    Compiling serde_json v1.0.145
[INFO] [stderr]    Compiling fibpetokenizer v0.1.0 (/opt/rustwide/workdir)
[INFO] [stdout] warning: unused imports: `error` and `hash`
[INFO] [stdout]  --> src/bpe_tokenizer.rs:1:12
[INFO] [stdout]   |
[INFO] [stdout] 1 | use core::{error, hash};
[INFO] [stdout]   |            ^^^^^  ^^^^
[INFO] [stdout]   |
[INFO] [stdout]   = note: `#[warn(unused_imports)]` (part of `#[warn(unused)]`) on by default
[INFO] [stdout] 
[INFO] [stdout] 
[INFO] [stdout] warning: unused import: `fmt::format`
[INFO] [stdout]  --> src/bpe_tokenizer.rs:2:56
[INFO] [stdout]   |
[INFO] [stdout] 2 | use std::{collections::{BinaryHeap, HashMap, HashSet}, fmt::format, fs::File, u32, usize};
[INFO] [stdout]   |                                                        ^^^^^^^^^^^
[INFO] [stdout] 
[INFO] [stdout] 
[INFO] [stdout] warning: unused import: `std::io`
[INFO] [stdout]  --> src/bpe_tokenizer.rs:5:5
[INFO] [stdout]   |
[INFO] [stdout] 5 | use std::io;
[INFO] [stdout]   |     ^^^^^^^
[INFO] [stdout] 
[INFO] [stdout] 
[INFO] [stdout] warning: unused import: `std::rc::Rc`
[INFO] [stdout]  --> src/bpe_tokenizer.rs:7:5
[INFO] [stdout]   |
[INFO] [stdout] 7 | use std::rc::Rc;
[INFO] [stdout]   |     ^^^^^^^^^^^
[INFO] [stdout] 
[INFO] [stdout] 
[INFO] [stdout] warning: unnecessary parentheses around pattern
[INFO] [stdout]    --> src/bpe_tokenizer.rs:884:9
[INFO] [stdout]     |
[INFO] [stdout] 884 |     for (word) in vocab{
[INFO] [stdout]     |         ^    ^
[INFO] [stdout]     |
[INFO] [stdout]     = note: `#[warn(unused_parens)]` (part of `#[warn(unused)]`) on by default
[INFO] [stdout] help: remove these parentheses
[INFO] [stdout]     |
[INFO] [stdout] 884 -     for (word) in vocab{
[INFO] [stdout] 884 +     for word  in vocab{
[INFO] [stdout]     |
[INFO] [stdout] 
[INFO] [stdout] 
[INFO] [stdout] warning: unused import: `core::panic`
[INFO] [stdout]  --> src/tokenizer_tables.rs:1:5
[INFO] [stdout]   |
[INFO] [stdout] 1 | use core::panic;
[INFO] [stdout]   |     ^^^^^^^^^^^
[INFO] [stdout] 
[INFO] [stdout] 
[INFO] [stdout] warning: unused import: `std::rc::Rc`
[INFO] [stdout]  --> src/tokenizer_tables.rs:3:5
[INFO] [stdout]   |
[INFO] [stdout] 3 | use std::rc::Rc;
[INFO] [stdout]   |     ^^^^^^^^^^^
[INFO] [stdout] 
[INFO] [stdout] 
[INFO] [stdout] warning: value assigned to `cleaned_content_string` is never read
[INFO] [stdout]    --> src/bpe_tokenizer.rs:198:63
[INFO] [stdout]     |
[INFO] [stdout] 198 |                         let mut cleaned_content_string:String=String::new();
[INFO] [stdout]     |                                                               ^^^^^^^^^^^^^
[INFO] [stdout]     |
[INFO] [stdout]     = help: maybe it is overwritten before being read?
[INFO] [stdout]     = note: `#[warn(unused_assignments)]` (part of `#[warn(unused)]`) on by default
[INFO] [stdout] 
[INFO] [stdout] 
[INFO] [stdout] warning: unused variable: `indx`
[INFO] [stdout]    --> src/bpe_tokenizer.rs:797:10
[INFO] [stdout]     |
[INFO] [stdout] 797 |     for (indx,word )in pretokenized_text.iter().enumerate(){
[INFO] [stdout]     |          ^^^^ help: if this is intentional, prefix it with an underscore: `_indx`
[INFO] [stdout]     |
[INFO] [stdout]     = note: `#[warn(unused_variables)]` (part of `#[warn(unused)]`) on by default
[INFO] [stdout] 
[INFO] [stdout] 
[INFO] [stdout] warning: method `get_merge_pair_from_pairs_to_merge_and_ranks` is never used
[INFO] [stdout]    --> src/bpe_tokenizer.rs:673:4
[INFO] [stdout]     |
[INFO] [stdout] 187 | impl BpeTokenizer {
[INFO] [stdout]     | ----------------- method in this implementation
[INFO] [stdout] ...
[INFO] [stdout] 673 | fn get_merge_pair_from_pairs_to_merge_and_ranks(&self,tuple:(u32,u32)){
[INFO] [stdout]     |    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[INFO] [stdout]     |
[INFO] [stdout]     = note: `#[warn(dead_code)]` (part of `#[warn(unused)]`) on by default
[INFO] [stdout] 
[INFO] [stdout] 
[INFO] [stdout] warning: variable `Positions_vec` should have a snake case name
[INFO] [stdout]    --> src/bpe_tokenizer.rs:961:17
[INFO] [stdout]     |
[INFO] [stdout] 961 |         let mut Positions_vec:Vec<(usize,usize)>=positions.iter().cloned().collect();
[INFO] [stdout]     |                 ^^^^^^^^^^^^^ help: convert the identifier to snake case: `positions_vec`
[INFO] [stdout]     |
[INFO] [stdout]     = note: `#[warn(non_snake_case)]` (part of `#[warn(nonstandard_style)]`) on by default
[INFO] [stdout] 
[INFO] [stdout] 
[INFO] [stdout] warning: unused import: `core::hash`
[INFO] [stdout]  --> src/main.rs:1:5
[INFO] [stdout]   |
[INFO] [stdout] 1 | use core::hash;
[INFO] [stdout]   |     ^^^^^^^^^^
[INFO] [stdout]   |
[INFO] [stdout]   = note: `#[warn(unused_imports)]` (part of `#[warn(unused)]`) on by default
[INFO] [stdout] 
[INFO] [stdout] 
[INFO] [stdout] warning: unused imports: `HashMap`, `HashSet`, `any::type_name`, `fmt::format`, and `usize`
[INFO] [stdout]  --> src/main.rs:2:11
[INFO] [stdout]   |
[INFO] [stdout] 2 | use std::{any::type_name, collections::{HashMap, HashSet}, fmt::format, usize};
[INFO] [stdout]   |           ^^^^^^^^^^^^^^                ^^^^^^^  ^^^^^^^   ^^^^^^^^^^^  ^^^^^
[INFO] [stdout] 
[INFO] [stdout] 
[INFO] [stdout] warning: unused import: `std::fs`
[INFO] [stdout]  --> src/main.rs:3:5
[INFO] [stdout]   |
[INFO] [stdout] 3 | use std::fs;
[INFO] [stdout]   |     ^^^^^^^
[INFO] [stdout] 
[INFO] [stdout] 
[INFO] [stdout] warning: unused import: `std::io`
[INFO] [stdout]  --> src/main.rs:4:5
[INFO] [stdout]   |
[INFO] [stdout] 4 | use std::io;
[INFO] [stdout]   |     ^^^^^^^
[INFO] [stdout] 
[INFO] [stdout] 
[INFO] [stdout] warning: unused imports: `error` and `hash`
[INFO] [stdout]  --> src/bpe_tokenizer.rs:1:12
[INFO] [stdout]   |
[INFO] [stdout] 1 | use core::{error, hash};
[INFO] [stdout]   |            ^^^^^  ^^^^
[INFO] [stdout] 
[INFO] [stdout] 
[INFO] [stdout] warning: unused import: `fmt::format`
[INFO] [stdout]  --> src/bpe_tokenizer.rs:2:56
[INFO] [stdout]   |
[INFO] [stdout] 2 | use std::{collections::{BinaryHeap, HashMap, HashSet}, fmt::format, fs::File, u32, usize};
[INFO] [stdout]   |                                                        ^^^^^^^^^^^
[INFO] [stdout] 
[INFO] [stdout] 
[INFO] [stdout] warning: unused import: `std::io`
[INFO] [stdout]  --> src/bpe_tokenizer.rs:5:5
[INFO] [stdout]   |
[INFO] [stdout] 5 | use std::io;
[INFO] [stdout]   |     ^^^^^^^
[INFO] [stdout] 
[INFO] [stdout] 
[INFO] [stdout] warning: unused import: `std::rc::Rc`
[INFO] [stdout]  --> src/bpe_tokenizer.rs:7:5
[INFO] [stdout]   |
[INFO] [stdout] 7 | use std::rc::Rc;
[INFO] [stdout]   |     ^^^^^^^^^^^
[INFO] [stdout] 
[INFO] [stdout] 
[INFO] [stdout] warning: unnecessary parentheses around pattern
[INFO] [stdout]    --> src/bpe_tokenizer.rs:884:9
[INFO] [stdout]     |
[INFO] [stdout] 884 |     for (word) in vocab{
[INFO] [stdout]     |         ^    ^
[INFO] [stdout]     |
[INFO] [stdout]     = note: `#[warn(unused_parens)]` (part of `#[warn(unused)]`) on by default
[INFO] [stdout] help: remove these parentheses
[INFO] [stdout]     |
[INFO] [stdout] 884 -     for (word) in vocab{
[INFO] [stdout] 884 +     for word  in vocab{
[INFO] [stdout]     |
[INFO] [stdout] 
[INFO] [stdout] 
[INFO] [stdout] warning: unused import: `core::panic`
[INFO] [stdout]  --> src/tokenizer_tables.rs:1:5
[INFO] [stdout]   |
[INFO] [stdout] 1 | use core::panic;
[INFO] [stdout]   |     ^^^^^^^^^^^
[INFO] [stdout] 
[INFO] [stdout] 
[INFO] [stdout] warning: unused import: `std::rc::Rc`
[INFO] [stdout]  --> src/tokenizer_tables.rs:3:5
[INFO] [stdout]   |
[INFO] [stdout] 3 | use std::rc::Rc;
[INFO] [stdout]   |     ^^^^^^^^^^^
[INFO] [stdout] 
[INFO] [stdout] 
[INFO] [stdout] warning: value assigned to `cleaned_content_string` is never read
[INFO] [stdout]    --> src/bpe_tokenizer.rs:198:63
[INFO] [stdout]     |
[INFO] [stdout] 198 |                         let mut cleaned_content_string:String=String::new();
[INFO] [stdout]     |                                                               ^^^^^^^^^^^^^
[INFO] [stdout]     |
[INFO] [stdout]     = help: maybe it is overwritten before being read?
[INFO] [stdout]     = note: `#[warn(unused_assignments)]` (part of `#[warn(unused)]`) on by default
[INFO] [stdout] 
[INFO] [stdout] 
[INFO] [stdout] warning: unused variable: `indx`
[INFO] [stdout]    --> src/bpe_tokenizer.rs:797:10
[INFO] [stdout]     |
[INFO] [stdout] 797 |     for (indx,word )in pretokenized_text.iter().enumerate(){
[INFO] [stdout]     |          ^^^^ help: if this is intentional, prefix it with an underscore: `_indx`
[INFO] [stdout]     |
[INFO] [stdout]     = note: `#[warn(unused_variables)]` (part of `#[warn(unused)]`) on by default
[INFO] [stdout] 
[INFO] [stdout] 
[INFO] [stdout] warning: unused variable: `special_tokens2`
[INFO] [stdout]   --> src/main.rs:21:9
[INFO] [stdout]    |
[INFO] [stdout] 21 |     let special_tokens2:Vec<String>=Vec::new();
[INFO] [stdout]    |         ^^^^^^^^^^^^^^^ help: if this is intentional, prefix it with an underscore: `_special_tokens2`
[INFO] [stdout] 
[INFO] [stdout] 
[INFO] [stdout] warning: variants `JsonFileNotExists`, `DirNotExists`, and `IdNotExists` are never constructed
[INFO] [stdout]   --> src/bpe_tokenizer.rs:41:5
[INFO] [stdout]    |
[INFO] [stdout] 30 | pub enum TokenizerError {
[INFO] [stdout]    |          -------------- variants in this enum
[INFO] [stdout] ...
[INFO] [stdout] 41 |     JsonFileNotExists(String),
[INFO] [stdout]    |     ^^^^^^^^^^^^^^^^^
[INFO] [stdout] ...
[INFO] [stdout] 44 |     DirNotExists(String),
[INFO] [stdout]    |     ^^^^^^^^^^^^
[INFO] [stdout] ...
[INFO] [stdout] 50 |     IdNotExists(String),
[INFO] [stdout]    |     ^^^^^^^^^^^
[INFO] [stdout]    |
[INFO] [stdout]    = note: `TokenizerError` has a derived impl for the trait `Debug`, but this is intentionally ignored during dead code analysis
[INFO] [stdout]    = note: `#[warn(dead_code)]` (part of `#[warn(unused)]`) on by default
[INFO] [stdout] 
[INFO] [stdout] 
[INFO] [stdout] warning: associated function `new_from_pretrained` is never used
[INFO] [stdout]    --> src/bpe_tokenizer.rs:130:12
[INFO] [stdout]     |
[INFO] [stdout] 108 | impl BpeTokenizer {
[INFO] [stdout]     | ----------------- associated function in this implementation
[INFO] [stdout] ...
[INFO] [stdout] 130 |     pub fn new_from_pretrained(files_path:&str)->Self{
[INFO] [stdout]     |            ^^^^^^^^^^^^^^^^^^^
[INFO] [stdout] 
[INFO] [stdout] 
[INFO] [stdout] warning: methods `display`, `get_id_by_token`, `get_token_by_id`, and `get_merge_pair_from_pairs_to_merge_and_ranks` are never used
[INFO] [stdout]    --> src/bpe_tokenizer.rs:646:8
[INFO] [stdout]     |
[INFO] [stdout] 187 | impl BpeTokenizer {
[INFO] [stdout]     | ----------------- methods in this implementation
[INFO] [stdout] ...
[INFO] [stdout] 646 | pub fn display(& self){
[INFO] [stdout]     |        ^^^^^^^
[INFO] [stdout] ...
[INFO] [stdout] 654 | pub fn get_id_by_token(&self,token:String)->Result<u32,TokenizerError>{
[INFO] [stdout]     |        ^^^^^^^^^^^^^^^
[INFO] [stdout] ...
[INFO] [stdout] 663 | pub fn get_token_by_id(&self,id:u32)->Result<String,TokenizerError>{
[INFO] [stdout]     |        ^^^^^^^^^^^^^^^
[INFO] [stdout] ...
[INFO] [stdout] 673 | fn get_merge_pair_from_pairs_to_merge_and_ranks(&self,tuple:(u32,u32)){
[INFO] [stdout]     |    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[INFO] [stdout] 
[INFO] [stdout] 
[INFO] [stdout] warning: function `extract_from_filepath_json_and_populate_bpetokenizer_struct` is never used
[INFO] [stdout]    --> src/bpe_tokenizer.rs:702:4
[INFO] [stdout]     |
[INFO] [stdout] 702 | fn extract_from_filepath_json_and_populate_bpetokenizer_struct(files_path:&str)-> Result<(Config,TokenTable,Vec<(u32,u32)>,HashMap<...
[INFO] [stdout]     |    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[INFO] [stdout] 
[INFO] [stdout] 
[INFO] [stdout] warning: function `read_config` is never used
[INFO] [stdout]    --> src/bpe_tokenizer.rs:726:4
[INFO] [stdout]     |
[INFO] [stdout] 726 | fn read_config(files_path:&str)->Result<Config,TokenizerError>{
[INFO] [stdout]     |    ^^^^^^^^^^^
[INFO] [stdout] 
[INFO] [stdout] 
[INFO] [stdout] warning: function `read_token_table` is never used
[INFO] [stdout]    --> src/bpe_tokenizer.rs:733:8
[INFO] [stdout]     |
[INFO] [stdout] 733 |     fn read_token_table(files_path:&str)->Result<TokenTable,TokenizerError>{
[INFO] [stdout]     |        ^^^^^^^^^^^^^^^^
[INFO] [stdout] 
[INFO] [stdout] 
[INFO] [stdout] warning: function `read_merge_rules` is never used
[INFO] [stdout]    --> src/bpe_tokenizer.rs:740:8
[INFO] [stdout]     |
[INFO] [stdout] 740 |     fn read_merge_rules(files_path:&str)->Result<Vec<(u32,u32)>,TokenizerError>{
[INFO] [stdout]     |        ^^^^^^^^^^^^^^^^
[INFO] [stdout] 
[INFO] [stdout] 
[INFO] [stdout] warning: function `read_ranks` is never used
[INFO] [stdout]    --> src/bpe_tokenizer.rs:746:8
[INFO] [stdout]     |
[INFO] [stdout] 746 |     fn read_ranks(files_path:&str)-> Result<HashMap<(u32, u32), usize>,TokenizerError>{
[INFO] [stdout]     |        ^^^^^^^^^^
[INFO] [stdout] 
[INFO] [stdout] 
[INFO] [stdout] warning: function `read_pairs_to_merge` is never used
[INFO] [stdout]    --> src/bpe_tokenizer.rs:760:8
[INFO] [stdout]     |
[INFO] [stdout] 760 |     fn read_pairs_to_merge(files_path:&str)-> Result<HashMap<(u32, u32), u32>, TokenizerError>{
[INFO] [stdout]     |        ^^^^^^^^^^^^^^^^^^^
[INFO] [stdout] 
[INFO] [stdout] 
[INFO] [stdout] warning: function `vocab_builder` is never used
[INFO] [stdout]    --> src/bpe_tokenizer.rs:793:8
[INFO] [stdout]     |
[INFO] [stdout] 793 | pub fn vocab_builder(pretokenized_text:&[String],vocab:&mut Vec<Word>, table: &mut TokenTable,pair_occurrences: &mut HashMap<(u32, ...
[INFO] [stdout]     |        ^^^^^^^^^^^^^
[INFO] [stdout] 
[INFO] [stdout] 
[INFO] [stdout] warning: method `tokens` is never used
[INFO] [stdout]   --> src/tokenizer_tables.rs:47:12
[INFO] [stdout]    |
[INFO] [stdout] 13 | impl TokenTable{
[INFO] [stdout]    | --------------- method in this implementation
[INFO] [stdout] ...
[INFO] [stdout] 47 |     pub fn tokens(&self) -> &[Arc<str>]{
[INFO] [stdout]    |            ^^^^^^
[INFO] [stdout] 
[INFO] [stdout] 
[INFO] [stdout] warning: variable `Positions_vec` should have a snake case name
[INFO] [stdout]    --> src/bpe_tokenizer.rs:961:17
[INFO] [stdout]     |
[INFO] [stdout] 961 |         let mut Positions_vec:Vec<(usize,usize)>=positions.iter().cloned().collect();
[INFO] [stdout]     |                 ^^^^^^^^^^^^^ help: convert the identifier to snake case: `positions_vec`
[INFO] [stdout]     |
[INFO] [stdout]     = note: `#[warn(non_snake_case)]` (part of `#[warn(nonstandard_style)]`) on by default
[INFO] [stdout] 
[INFO] [stdout] 
[INFO] [stderr]     Finished `dev` profile [unoptimized + debuginfo] target(s) in 14.19s
[INFO] running `Command { std: "docker" "inspect" "917799d9da54ae0740160014a5197e3388f5293d73822a9ca4db4fc5d705a842", kill_on_drop: false }`
[INFO] running `Command { std: "docker" "rm" "-f" "917799d9da54ae0740160014a5197e3388f5293d73822a9ca4db4fc5d705a842", kill_on_drop: false }`
[INFO] [stdout] 917799d9da54ae0740160014a5197e3388f5293d73822a9ca4db4fc5d705a842
[INFO] running `Command { std: "docker" "create" "-v" "/var/lib/crater-agent-workspace/builds/worker-2-tc2/target:/opt/rustwide/target:rw,Z" "-v" "/var/lib/crater-agent-workspace/builds/worker-2-tc2/source:/opt/rustwide/workdir:ro,Z" "-v" "/var/lib/crater-agent-workspace/cargo-home:/opt/rustwide/cargo-home:ro,Z" "-v" "/var/lib/crater-agent-workspace/rustup-home:/opt/rustwide/rustup-home:ro,Z" "-e" "SOURCE_DIR=/opt/rustwide/workdir" "-e" "CARGO_TARGET_DIR=/opt/rustwide/target" "-e" "CARGO_INCREMENTAL=0" "-e" "RUST_BACKTRACE=full" "-e" "RUSTFLAGS=--cap-lints=warn" "-e" "RUSTDOCFLAGS=--cap-lints=warn" "-e" "CARGO_HOME=/opt/rustwide/cargo-home" "-e" "RUSTUP_HOME=/opt/rustwide/rustup-home" "-w" "/opt/rustwide/workdir" "-m" "1610612736" "--user" "0:0" "--network" "none" "ghcr.io/rust-lang/crates-build-env/linux@sha256:d429b63d4308055ea97f60fb1d3dfca48854a00942f1bd2ad806beaf015945ec" "/opt/rustwide/cargo-home/bin/cargo" "+beta-2026-03-05" "test" "--frozen" "--no-run" "--message-format=json", kill_on_drop: false }`
[INFO] [stdout] 21e41be4fdd258a720e80ae0cf21368c75a8d7a6e75f9f3c0bab8ba31db5d12a
[INFO] running `Command { std: "docker" "start" "-a" "21e41be4fdd258a720e80ae0cf21368c75a8d7a6e75f9f3c0bab8ba31db5d12a", kill_on_drop: false }`
[INFO] [stdout] warning: unused imports: `error` and `hash`
[INFO] [stdout]  --> src/bpe_tokenizer.rs:1:12
[INFO] [stdout]   |
[INFO] [stdout] 1 | use core::{error, hash};
[INFO] [stdout]   |            ^^^^^  ^^^^
[INFO] [stdout]   |
[INFO] [stdout]   = note: `#[warn(unused_imports)]` (part of `#[warn(unused)]`) on by default
[INFO] [stdout] 
[INFO] [stdout] 
[INFO] [stdout] warning: unused import: `fmt::format`
[INFO] [stdout]  --> src/bpe_tokenizer.rs:2:56
[INFO] [stdout]   |
[INFO] [stdout] 2 | use std::{collections::{BinaryHeap, HashMap, HashSet}, fmt::format, fs::File, u32, usize};
[INFO] [stdout]   |                                                        ^^^^^^^^^^^
[INFO] [stdout] 
[INFO] [stdout] 
[INFO] [stdout] warning: unused import: `std::io`
[INFO] [stdout]  --> src/bpe_tokenizer.rs:5:5
[INFO] [stdout]   |
[INFO] [stdout] 5 | use std::io;
[INFO] [stdout]   |     ^^^^^^^
[INFO] [stdout] 
[INFO] [stdout] 
[INFO] [stdout] warning: unused import: `std::rc::Rc`
[INFO] [stdout]  --> src/bpe_tokenizer.rs:7:5
[INFO] [stdout]   |
[INFO] [stdout] 7 | use std::rc::Rc;
[INFO] [stdout]   |     ^^^^^^^^^^^
[INFO] [stdout] 
[INFO] [stdout] 
[INFO] [stdout] warning: unnecessary parentheses around pattern
[INFO] [stdout]    --> src/bpe_tokenizer.rs:884:9
[INFO] [stdout]     |
[INFO] [stdout] 884 |     for (word) in vocab{
[INFO] [stdout]     |         ^    ^
[INFO] [stdout]     |
[INFO] [stdout]     = note: `#[warn(unused_parens)]` (part of `#[warn(unused)]`) on by default
[INFO] [stdout] help: remove these parentheses
[INFO] [stdout]     |
[INFO] [stdout] 884 -     for (word) in vocab{
[INFO] [stdout] 884 +     for word  in vocab{
[INFO] [stdout]     |
[INFO] [stdout] 
[INFO] [stdout] 
[INFO] [stdout] warning: unused import: `core::panic`
[INFO] [stdout]  --> src/tokenizer_tables.rs:1:5
[INFO] [stdout]   |
[INFO] [stdout] 1 | use core::panic;
[INFO] [stdout]   |     ^^^^^^^^^^^
[INFO] [stdout] 
[INFO] [stdout] 
[INFO] [stdout] warning: unused import: `std::rc::Rc`
[INFO] [stdout]  --> src/tokenizer_tables.rs:3:5
[INFO] [stdout]   |
[INFO] [stdout] 3 | use std::rc::Rc;
[INFO] [stdout]   |     ^^^^^^^^^^^
[INFO] [stdout] 
[INFO] [stdout] 
[INFO] [stdout] warning: value assigned to `cleaned_content_string` is never read
[INFO] [stdout]    --> src/bpe_tokenizer.rs:198:63
[INFO] [stdout]     |
[INFO] [stdout] 198 |                         let mut cleaned_content_string:String=String::new();
[INFO] [stdout]     |                                                               ^^^^^^^^^^^^^
[INFO] [stdout]     |
[INFO] [stdout]     = help: maybe it is overwritten before being read?
[INFO] [stdout]     = note: `#[warn(unused_assignments)]` (part of `#[warn(unused)]`) on by default
[INFO] [stdout] 
[INFO] [stdout] 
[INFO] [stdout] warning: unused variable: `indx`
[INFO] [stdout]    --> src/bpe_tokenizer.rs:797:10
[INFO] [stdout]     |
[INFO] [stdout] 797 |     for (indx,word )in pretokenized_text.iter().enumerate(){
[INFO] [stdout]     |          ^^^^ help: if this is intentional, prefix it with an underscore: `_indx`
[INFO] [stdout]     |
[INFO] [stdout]     = note: `#[warn(unused_variables)]` (part of `#[warn(unused)]`) on by default
[INFO] [stdout] 
[INFO] [stdout] 
[INFO] [stderr]    Compiling fibpetokenizer v0.1.0 (/opt/rustwide/workdir)
[INFO] [stdout] warning: method `get_merge_pair_from_pairs_to_merge_and_ranks` is never used
[INFO] [stdout]    --> src/bpe_tokenizer.rs:673:4
[INFO] [stdout]     |
[INFO] [stdout] 187 | impl BpeTokenizer {
[INFO] [stdout]     | ----------------- method in this implementation
[INFO] [stdout] ...
[INFO] [stdout] 673 | fn get_merge_pair_from_pairs_to_merge_and_ranks(&self,tuple:(u32,u32)){
[INFO] [stdout]     |    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[INFO] [stdout]     |
[INFO] [stdout]     = note: `#[warn(dead_code)]` (part of `#[warn(unused)]`) on by default
[INFO] [stdout] 
[INFO] [stdout] 
[INFO] [stdout] warning: variable `Positions_vec` should have a snake case name
[INFO] [stdout]    --> src/bpe_tokenizer.rs:961:17
[INFO] [stdout]     |
[INFO] [stdout] 961 |         let mut Positions_vec:Vec<(usize,usize)>=positions.iter().cloned().collect();
[INFO] [stdout]     |                 ^^^^^^^^^^^^^ help: convert the identifier to snake case: `positions_vec`
[INFO] [stdout]     |
[INFO] [stdout]     = note: `#[warn(non_snake_case)]` (part of `#[warn(nonstandard_style)]`) on by default
[INFO] [stdout] 
[INFO] [stdout] 
[INFO] [stdout] warning: unused import: `core::hash`
[INFO] [stdout]  --> src/main.rs:1:5
[INFO] [stdout]   |
[INFO] [stdout] 1 | use core::hash;
[INFO] [stdout]   |     ^^^^^^^^^^
[INFO] [stdout]   |
[INFO] [stdout]   = note: `#[warn(unused_imports)]` (part of `#[warn(unused)]`) on by default
[INFO] [stdout] 
[INFO] [stdout] 
[INFO] [stdout] warning: unused imports: `HashMap`, `HashSet`, `any::type_name`, `fmt::format`, and `usize`
[INFO] [stdout]  --> src/main.rs:2:11
[INFO] [stdout]   |
[INFO] [stdout] 2 | use std::{any::type_name, collections::{HashMap, HashSet}, fmt::format, usize};
[INFO] [stdout]   |           ^^^^^^^^^^^^^^                ^^^^^^^  ^^^^^^^   ^^^^^^^^^^^  ^^^^^
[INFO] [stdout] 
[INFO] [stdout] 
[INFO] [stdout] warning: unused import: `std::fs`
[INFO] [stdout]  --> src/main.rs:3:5
[INFO] [stdout]   |
[INFO] [stdout] 3 | use std::fs;
[INFO] [stdout]   |     ^^^^^^^
[INFO] [stdout] 
[INFO] [stdout] 
[INFO] [stdout] warning: unused import: `std::io`
[INFO] [stdout]  --> src/main.rs:4:5
[INFO] [stdout]   |
[INFO] [stdout] 4 | use std::io;
[INFO] [stdout]   |     ^^^^^^^
[INFO] [stdout] 
[INFO] [stdout] 
[INFO] [stdout] warning: unused imports: `error` and `hash`
[INFO] [stdout]  --> src/bpe_tokenizer.rs:1:12
[INFO] [stdout]   |
[INFO] [stdout] 1 | use core::{error, hash};
[INFO] [stdout]   |            ^^^^^  ^^^^
[INFO] [stdout] 
[INFO] [stdout] 
[INFO] [stdout] warning: unused import: `fmt::format`
[INFO] [stdout]  --> src/bpe_tokenizer.rs:2:56
[INFO] [stdout]   |
[INFO] [stdout] 2 | use std::{collections::{BinaryHeap, HashMap, HashSet}, fmt::format, fs::File, u32, usize};
[INFO] [stdout]   |                                                        ^^^^^^^^^^^
[INFO] [stdout] 
[INFO] [stdout] 
[INFO] [stdout] warning: unused import: `std::io`
[INFO] [stdout]  --> src/bpe_tokenizer.rs:5:5
[INFO] [stdout]   |
[INFO] [stdout] 5 | use std::io;
[INFO] [stdout]   |     ^^^^^^^
[INFO] [stdout] 
[INFO] [stdout] 
[INFO] [stdout] warning: unused import: `std::rc::Rc`
[INFO] [stdout]  --> src/bpe_tokenizer.rs:7:5
[INFO] [stdout]   |
[INFO] [stdout] 7 | use std::rc::Rc;
[INFO] [stdout]   |     ^^^^^^^^^^^
[INFO] [stdout] 
[INFO] [stdout] 
[INFO] [stdout] warning: unnecessary parentheses around pattern
[INFO] [stdout]    --> src/bpe_tokenizer.rs:884:9
[INFO] [stdout]     |
[INFO] [stdout] 884 |     for (word) in vocab{
[INFO] [stdout]     |         ^    ^
[INFO] [stdout]     |
[INFO] [stdout]     = note: `#[warn(unused_parens)]` (part of `#[warn(unused)]`) on by default
[INFO] [stdout] help: remove these parentheses
[INFO] [stdout]     |
[INFO] [stdout] 884 -     for (word) in vocab{
[INFO] [stdout] 884 +     for word  in vocab{
[INFO] [stdout]     |
[INFO] [stdout] 
[INFO] [stdout] 
[INFO] [stdout] warning: unused import: `core::panic`
[INFO] [stdout]  --> src/tokenizer_tables.rs:1:5
[INFO] [stdout]   |
[INFO] [stdout] 1 | use core::panic;
[INFO] [stdout]   |     ^^^^^^^^^^^
[INFO] [stdout] 
[INFO] [stdout] 
[INFO] [stdout] warning: unused import: `std::rc::Rc`
[INFO] [stdout]  --> src/tokenizer_tables.rs:3:5
[INFO] [stdout]   |
[INFO] [stdout] 3 | use std::rc::Rc;
[INFO] [stdout]   |     ^^^^^^^^^^^
[INFO] [stdout] 
[INFO] [stdout] 
[INFO] [stdout] warning: unused imports: `error` and `hash`
[INFO] [stdout]  --> src/bpe_tokenizer.rs:1:12
[INFO] [stdout]   |
[INFO] [stdout] 1 | use core::{error, hash};
[INFO] [stdout]   |            ^^^^^  ^^^^
[INFO] [stdout]   |
[INFO] [stdout]   = note: `#[warn(unused_imports)]` (part of `#[warn(unused)]`) on by default
[INFO] [stdout] 
[INFO] [stdout] 
[INFO] [stdout] warning: unused import: `fmt::format`
[INFO] [stdout]  --> src/bpe_tokenizer.rs:2:56
[INFO] [stdout]   |
[INFO] [stdout] 2 | use std::{collections::{BinaryHeap, HashMap, HashSet}, fmt::format, fs::File, u32, usize};
[INFO] [stdout]   |                                                        ^^^^^^^^^^^
[INFO] [stdout] 
[INFO] [stdout] 
[INFO] [stdout] warning: unused import: `std::io`
[INFO] [stdout]  --> src/bpe_tokenizer.rs:5:5
[INFO] [stdout]   |
[INFO] [stdout] 5 | use std::io;
[INFO] [stdout]   |     ^^^^^^^
[INFO] [stdout] 
[INFO] [stdout] 
[INFO] [stdout] warning: unused import: `std::rc::Rc`
[INFO] [stdout]  --> src/bpe_tokenizer.rs:7:5
[INFO] [stdout]   |
[INFO] [stdout] 7 | use std::rc::Rc;
[INFO] [stdout]   |     ^^^^^^^^^^^
[INFO] [stdout] 
[INFO] [stdout] 
[INFO] [stdout] warning: unnecessary parentheses around pattern
[INFO] [stdout]    --> src/bpe_tokenizer.rs:884:9
[INFO] [stdout]     |
[INFO] [stdout] 884 |     for (word) in vocab{
[INFO] [stdout]     |         ^    ^
[INFO] [stdout]     |
[INFO] [stdout]     = note: `#[warn(unused_parens)]` (part of `#[warn(unused)]`) on by default
[INFO] [stdout] help: remove these parentheses
[INFO] [stdout]     |
[INFO] [stdout] 884 -     for (word) in vocab{
[INFO] [stdout] 884 +     for word  in vocab{
[INFO] [stdout]     |
[INFO] [stdout] 
[INFO] [stdout] 
[INFO] [stdout] warning: unused import: `core::panic`
[INFO] [stdout]  --> src/tokenizer_tables.rs:1:5
[INFO] [stdout]   |
[INFO] [stdout] 1 | use core::panic;
[INFO] [stdout]   |     ^^^^^^^^^^^
[INFO] [stdout] 
[INFO] [stdout] 
[INFO] [stdout] warning: unused import: `std::rc::Rc`
[INFO] [stdout]  --> src/tokenizer_tables.rs:3:5
[INFO] [stdout]   |
[INFO] [stdout] 3 | use std::rc::Rc;
[INFO] [stdout]   |     ^^^^^^^^^^^
[INFO] [stdout] 
[INFO] [stdout] 
[INFO] [stdout] warning: value assigned to `cleaned_content_string` is never read
[INFO] [stdout]    --> src/bpe_tokenizer.rs:198:63
[INFO] [stdout]     |
[INFO] [stdout] 198 |                         let mut cleaned_content_string:String=String::new();
[INFO] [stdout]     |                                                               ^^^^^^^^^^^^^
[INFO] [stdout]     |
[INFO] [stdout]     = help: maybe it is overwritten before being read?
[INFO] [stdout]     = note: `#[warn(unused_assignments)]` (part of `#[warn(unused)]`) on by default
[INFO] [stdout] 
[INFO] [stdout] 
[INFO] [stdout] warning: unused variable: `indx`
[INFO] [stdout]    --> src/bpe_tokenizer.rs:797:10
[INFO] [stdout]     |
[INFO] [stdout] 797 |     for (indx,word )in pretokenized_text.iter().enumerate(){
[INFO] [stdout]     |          ^^^^ help: if this is intentional, prefix it with an underscore: `_indx`
[INFO] [stdout]     |
[INFO] [stdout]     = note: `#[warn(unused_variables)]` (part of `#[warn(unused)]`) on by default
[INFO] [stdout] 
[INFO] [stdout] 
[INFO] [stdout] warning: unused variable: `special_tokens2`
[INFO] [stdout]   --> src/main.rs:21:9
[INFO] [stdout]    |
[INFO] [stdout] 21 |     let special_tokens2:Vec<String>=Vec::new();
[INFO] [stdout]    |         ^^^^^^^^^^^^^^^ help: if this is intentional, prefix it with an underscore: `_special_tokens2`
[INFO] [stdout] 
[INFO] [stdout] 
[INFO] [stdout] warning: variants `JsonFileNotExists`, `DirNotExists`, and `IdNotExists` are never constructed
[INFO] [stdout]   --> src/bpe_tokenizer.rs:41:5
[INFO] [stdout]    |
[INFO] [stdout] 30 | pub enum TokenizerError {
[INFO] [stdout]    |          -------------- variants in this enum
[INFO] [stdout] ...
[INFO] [stdout] 41 |     JsonFileNotExists(String),
[INFO] [stdout]    |     ^^^^^^^^^^^^^^^^^
[INFO] [stdout] ...
[INFO] [stdout] 44 |     DirNotExists(String),
[INFO] [stdout]    |     ^^^^^^^^^^^^
[INFO] [stdout] ...
[INFO] [stdout] 50 |     IdNotExists(String),
[INFO] [stdout]    |     ^^^^^^^^^^^
[INFO] [stdout]    |
[INFO] [stdout]    = note: `TokenizerError` has a derived impl for the trait `Debug`, but this is intentionally ignored during dead code analysis
[INFO] [stdout]    = note: `#[warn(dead_code)]` (part of `#[warn(unused)]`) on by default
[INFO] [stdout] 
[INFO] [stdout] 
[INFO] [stdout] warning: associated function `new_from_pretrained` is never used
[INFO] [stdout]    --> src/bpe_tokenizer.rs:130:12
[INFO] [stdout]     |
[INFO] [stdout] 108 | impl BpeTokenizer {
[INFO] [stdout]     | ----------------- associated function in this implementation
[INFO] [stdout] ...
[INFO] [stdout] 130 |     pub fn new_from_pretrained(files_path:&str)->Self{
[INFO] [stdout]     |            ^^^^^^^^^^^^^^^^^^^
[INFO] [stdout] 
[INFO] [stdout] 
[INFO] [stdout] warning: methods `display`, `get_id_by_token`, `get_token_by_id`, and `get_merge_pair_from_pairs_to_merge_and_ranks` are never used
[INFO] [stdout]    --> src/bpe_tokenizer.rs:646:8
[INFO] [stdout]     |
[INFO] [stdout] 187 | impl BpeTokenizer {
[INFO] [stdout]     | ----------------- methods in this implementation
[INFO] [stdout] ...
[INFO] [stdout] 646 | pub fn display(& self){
[INFO] [stdout]     |        ^^^^^^^
[INFO] [stdout] ...
[INFO] [stdout] 654 | pub fn get_id_by_token(&self,token:String)->Result<u32,TokenizerError>{
[INFO] [stdout]     |        ^^^^^^^^^^^^^^^
[INFO] [stdout] ...
[INFO] [stdout] 663 | pub fn get_token_by_id(&self,id:u32)->Result<String,TokenizerError>{
[INFO] [stdout]     |        ^^^^^^^^^^^^^^^
[INFO] [stdout] ...
[INFO] [stdout] 673 | fn get_merge_pair_from_pairs_to_merge_and_ranks(&self,tuple:(u32,u32)){
[INFO] [stdout]     |    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[INFO] [stdout] 
[INFO] [stdout] 
[INFO] [stdout] warning: function `extract_from_filepath_json_and_populate_bpetokenizer_struct` is never used
[INFO] [stdout]    --> src/bpe_tokenizer.rs:702:4
[INFO] [stdout]     |
[INFO] [stdout] 702 | fn extract_from_filepath_json_and_populate_bpetokenizer_struct(files_path:&str)-> Result<(Config,TokenTable,Vec<(u32,u32)>,HashMap<...
[INFO] [stdout]     |    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[INFO] [stdout] 
[INFO] [stdout] 
[INFO] [stdout] warning: function `read_config` is never used
[INFO] [stdout]    --> src/bpe_tokenizer.rs:726:4
[INFO] [stdout]     |
[INFO] [stdout] 726 | fn read_config(files_path:&str)->Result<Config,TokenizerError>{
[INFO] [stdout]     |    ^^^^^^^^^^^
[INFO] [stdout] 
[INFO] [stdout] 
[INFO] [stdout] warning: function `read_token_table` is never used
[INFO] [stdout]    --> src/bpe_tokenizer.rs:733:8
[INFO] [stdout]     |
[INFO] [stdout] 733 |     fn read_token_table(files_path:&str)->Result<TokenTable,TokenizerError>{
[INFO] [stdout]     |        ^^^^^^^^^^^^^^^^
[INFO] [stdout] 
[INFO] [stdout] 
[INFO] [stdout] warning: function `read_merge_rules` is never used
[INFO] [stdout]    --> src/bpe_tokenizer.rs:740:8
[INFO] [stdout]     |
[INFO] [stdout] 740 |     fn read_merge_rules(files_path:&str)->Result<Vec<(u32,u32)>,TokenizerError>{
[INFO] [stdout]     |        ^^^^^^^^^^^^^^^^
[INFO] [stdout] 
[INFO] [stdout] 
[INFO] [stdout] warning: function `read_ranks` is never used
[INFO] [stdout]    --> src/bpe_tokenizer.rs:746:8
[INFO] [stdout]     |
[INFO] [stdout] 746 |     fn read_ranks(files_path:&str)-> Result<HashMap<(u32, u32), usize>,TokenizerError>{
[INFO] [stdout]     |        ^^^^^^^^^^
[INFO] [stdout] 
[INFO] [stdout] 
[INFO] [stdout] warning: function `read_pairs_to_merge` is never used
[INFO] [stdout]    --> src/bpe_tokenizer.rs:760:8
[INFO] [stdout]     |
[INFO] [stdout] 760 |     fn read_pairs_to_merge(files_path:&str)-> Result<HashMap<(u32, u32), u32>, TokenizerError>{
[INFO] [stdout]     |        ^^^^^^^^^^^^^^^^^^^
[INFO] [stdout] 
[INFO] [stdout] 
[INFO] [stdout] warning: function `vocab_builder` is never used
[INFO] [stdout]    --> src/bpe_tokenizer.rs:793:8
[INFO] [stdout]     |
[INFO] [stdout] 793 | pub fn vocab_builder(pretokenized_text:&[String],vocab:&mut Vec<Word>, table: &mut TokenTable,pair_occurrences: &mut HashMap<(u32, ...
[INFO] [stdout]     |        ^^^^^^^^^^^^^
[INFO] [stdout] 
[INFO] [stdout] 
[INFO] [stdout] warning: method `tokens` is never used
[INFO] [stdout]   --> src/tokenizer_tables.rs:47:12
[INFO] [stdout]    |
[INFO] [stdout] 13 | impl TokenTable{
[INFO] [stdout]    | --------------- method in this implementation
[INFO] [stdout] ...
[INFO] [stdout] 47 |     pub fn tokens(&self) -> &[Arc<str>]{
[INFO] [stdout]    |            ^^^^^^
[INFO] [stdout] 
[INFO] [stdout] 
[INFO] [stdout] warning: variable `Positions_vec` should have a snake case name
[INFO] [stdout]    --> src/bpe_tokenizer.rs:961:17
[INFO] [stdout]     |
[INFO] [stdout] 961 |         let mut Positions_vec:Vec<(usize,usize)>=positions.iter().cloned().collect();
[INFO] [stdout]     |                 ^^^^^^^^^^^^^ help: convert the identifier to snake case: `positions_vec`
[INFO] [stdout]     |
[INFO] [stdout]     = note: `#[warn(non_snake_case)]` (part of `#[warn(nonstandard_style)]`) on by default
[INFO] [stdout] 
[INFO] [stdout] 
[INFO] [stdout] warning: value assigned to `cleaned_content_string` is never read
[INFO] [stdout]    --> src/bpe_tokenizer.rs:198:63
[INFO] [stdout]     |
[INFO] [stdout] 198 |                         let mut cleaned_content_string:String=String::new();
[INFO] [stdout]     |                                                               ^^^^^^^^^^^^^
[INFO] [stdout]     |
[INFO] [stdout]     = help: maybe it is overwritten before being read?
[INFO] [stdout]     = note: `#[warn(unused_assignments)]` (part of `#[warn(unused)]`) on by default
[INFO] [stdout] 
[INFO] [stdout] 
[INFO] [stdout] warning: unused variable: `indx`
[INFO] [stdout]    --> src/bpe_tokenizer.rs:797:10
[INFO] [stdout]     |
[INFO] [stdout] 797 |     for (indx,word )in pretokenized_text.iter().enumerate(){
[INFO] [stdout]     |          ^^^^ help: if this is intentional, prefix it with an underscore: `_indx`
[INFO] [stdout]     |
[INFO] [stdout]     = note: `#[warn(unused_variables)]` (part of `#[warn(unused)]`) on by default
[INFO] [stdout] 
[INFO] [stdout] 
[INFO] [stdout] warning: method `get_merge_pair_from_pairs_to_merge_and_ranks` is never used
[INFO] [stdout]    --> src/bpe_tokenizer.rs:673:4
[INFO] [stdout]     |
[INFO] [stdout] 187 | impl BpeTokenizer {
[INFO] [stdout]     | ----------------- method in this implementation
[INFO] [stdout] ...
[INFO] [stdout] 673 | fn get_merge_pair_from_pairs_to_merge_and_ranks(&self,tuple:(u32,u32)){
[INFO] [stdout]     |    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[INFO] [stdout]     |
[INFO] [stdout]     = note: `#[warn(dead_code)]` (part of `#[warn(unused)]`) on by default
[INFO] [stdout] 
[INFO] [stdout] 
[INFO] [stdout] warning: variable `Positions_vec` should have a snake case name
[INFO] [stdout]    --> src/bpe_tokenizer.rs:961:17
[INFO] [stdout]     |
[INFO] [stdout] 961 |         let mut Positions_vec:Vec<(usize,usize)>=positions.iter().cloned().collect();
[INFO] [stdout]     |                 ^^^^^^^^^^^^^ help: convert the identifier to snake case: `positions_vec`
[INFO] [stdout]     |
[INFO] [stdout]     = note: `#[warn(non_snake_case)]` (part of `#[warn(nonstandard_style)]`) on by default
[INFO] [stdout] 
[INFO] [stdout] 
[INFO] [stderr]     Finished `test` profile [unoptimized + debuginfo] target(s) in 0.93s
[INFO] running `Command { std: "docker" "inspect" "21e41be4fdd258a720e80ae0cf21368c75a8d7a6e75f9f3c0bab8ba31db5d12a", kill_on_drop: false }`
[INFO] running `Command { std: "docker" "rm" "-f" "21e41be4fdd258a720e80ae0cf21368c75a8d7a6e75f9f3c0bab8ba31db5d12a", kill_on_drop: false }`
[INFO] [stdout] 21e41be4fdd258a720e80ae0cf21368c75a8d7a6e75f9f3c0bab8ba31db5d12a
[INFO] running `Command { std: "docker" "create" "-v" "/var/lib/crater-agent-workspace/builds/worker-2-tc2/target:/opt/rustwide/target:rw,Z" "-v" "/var/lib/crater-agent-workspace/builds/worker-2-tc2/source:/opt/rustwide/workdir:ro,Z" "-v" "/var/lib/crater-agent-workspace/cargo-home:/opt/rustwide/cargo-home:ro,Z" "-v" "/var/lib/crater-agent-workspace/rustup-home:/opt/rustwide/rustup-home:ro,Z" "-e" "SOURCE_DIR=/opt/rustwide/workdir" "-e" "CARGO_TARGET_DIR=/opt/rustwide/target" "-e" "CARGO_INCREMENTAL=0" "-e" "RUST_BACKTRACE=full" "-e" "RUSTFLAGS=--cap-lints=warn" "-e" "RUSTDOCFLAGS=--cap-lints=warn" "-e" "CARGO_HOME=/opt/rustwide/cargo-home" "-e" "RUSTUP_HOME=/opt/rustwide/rustup-home" "-w" "/opt/rustwide/workdir" "-m" "1610612736" "--user" "0:0" "--network" "none" "ghcr.io/rust-lang/crates-build-env/linux@sha256:d429b63d4308055ea97f60fb1d3dfca48854a00942f1bd2ad806beaf015945ec" "/opt/rustwide/cargo-home/bin/cargo" "+beta-2026-03-05" "test" "--frozen", kill_on_drop: false }`
[INFO] [stdout] bbeee354ecacfba6d8f6bdae76384fa51cacd4c6e9c04f4b6f228caaf5d6b076
[INFO] running `Command { std: "docker" "start" "-a" "bbeee354ecacfba6d8f6bdae76384fa51cacd4c6e9c04f4b6f228caaf5d6b076", kill_on_drop: false }`
[INFO] [stderr] warning: unused imports: `error` and `hash`
[INFO] [stderr]  --> src/bpe_tokenizer.rs:1:12
[INFO] [stderr]   |
[INFO] [stderr] 1 | use core::{error, hash};
[INFO] [stderr]   |            ^^^^^  ^^^^
[INFO] [stderr]   |
[INFO] [stderr]   = note: `#[warn(unused_imports)]` (part of `#[warn(unused)]`) on by default
[INFO] [stderr] 
[INFO] [stderr] warning: unused import: `fmt::format`
[INFO] [stderr]  --> src/bpe_tokenizer.rs:2:56
[INFO] [stderr]   |
[INFO] [stderr] 2 | use std::{collections::{BinaryHeap, HashMap, HashSet}, fmt::format, fs::File, u32, usize};
[INFO] [stderr]   |                                                        ^^^^^^^^^^^
[INFO] [stderr] 
[INFO] [stderr] warning: unused import: `std::io`
[INFO] [stderr]  --> src/bpe_tokenizer.rs:5:5
[INFO] [stderr]   |
[INFO] [stderr] 5 | use std::io;
[INFO] [stderr]   |     ^^^^^^^
[INFO] [stderr] 
[INFO] [stderr] warning: unused import: `std::rc::Rc`
[INFO] [stderr]  --> src/bpe_tokenizer.rs:7:5
[INFO] [stderr]   |
[INFO] [stderr] 7 | use std::rc::Rc;
[INFO] [stderr]   |     ^^^^^^^^^^^
[INFO] [stderr] 
[INFO] [stderr] warning: unnecessary parentheses around pattern
[INFO] [stderr]    --> src/bpe_tokenizer.rs:884:9
[INFO] [stderr]     |
[INFO] [stderr] 884 |     for (word) in vocab{
[INFO] [stderr]     |         ^    ^
[INFO] [stderr]     |
[INFO] [stderr]     = note: `#[warn(unused_parens)]` (part of `#[warn(unused)]`) on by default
[INFO] [stderr] help: remove these parentheses
[INFO] [stderr]     |
[INFO] [stderr] 884 -     for (word) in vocab{
[INFO] [stderr] 884 +     for word  in vocab{
[INFO] [stderr]     |
[INFO] [stderr] 
[INFO] [stderr] warning: unused import: `core::panic`
[INFO] [stderr]  --> src/tokenizer_tables.rs:1:5
[INFO] [stderr]   |
[INFO] [stderr] 1 | use core::panic;
[INFO] [stderr]   |     ^^^^^^^^^^^
[INFO] [stderr] 
[INFO] [stderr] warning: unused import: `std::rc::Rc`
[INFO] [stderr]  --> src/tokenizer_tables.rs:3:5
[INFO] [stderr]   |
[INFO] [stderr] 3 | use std::rc::Rc;
[INFO] [stderr]   |     ^^^^^^^^^^^
[INFO] [stderr] 
[INFO] [stderr] warning: value assigned to `cleaned_content_string` is never read
[INFO] [stderr]    --> src/bpe_tokenizer.rs:198:63
[INFO] [stderr]     |
[INFO] [stderr] 198 |                         let mut cleaned_content_string:String=String::new();
[INFO] [stderr]     |                                                               ^^^^^^^^^^^^^
[INFO] [stderr]     |
[INFO] [stderr]     = help: maybe it is overwritten before being read?
[INFO] [stderr]     = note: `#[warn(unused_assignments)]` (part of `#[warn(unused)]`) on by default
[INFO] [stderr] 
[INFO] [stderr] warning: unused variable: `indx`
[INFO] [stderr]    --> src/bpe_tokenizer.rs:797:10
[INFO] [stderr]     |
[INFO] [stderr] 797 |     for (indx,word )in pretokenized_text.iter().enumerate(){
[INFO] [stderr]     |          ^^^^ help: if this is intentional, prefix it with an underscore: `_indx`
[INFO] [stderr]     |
[INFO] [stderr]     = note: `#[warn(unused_variables)]` (part of `#[warn(unused)]`) on by default
[INFO] [stderr] 
[INFO] [stderr] warning: method `get_merge_pair_from_pairs_to_merge_and_ranks` is never used
[INFO] [stderr]    --> src/bpe_tokenizer.rs:673:4
[INFO] [stderr]     |
[INFO] [stderr] 187 | impl BpeTokenizer {
[INFO] [stderr]     | ----------------- method in this implementation
[INFO] [stderr] ...
[INFO] [stderr] 673 | fn get_merge_pair_from_pairs_to_merge_and_ranks(&self,tuple:(u32,u32)){
[INFO] [stderr]     |    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[INFO] [stderr]     |
[INFO] [stderr]     = note: `#[warn(dead_code)]` (part of `#[warn(unused)]`) on by default
[INFO] [stderr] 
[INFO] [stderr] warning: variable `Positions_vec` should have a snake case name
[INFO] [stderr]    --> src/bpe_tokenizer.rs:961:17
[INFO] [stderr]     |
[INFO] [stderr] 961 |         let mut Positions_vec:Vec<(usize,usize)>=positions.iter().cloned().collect();
[INFO] [stderr]     |                 ^^^^^^^^^^^^^ help: convert the identifier to snake case: `positions_vec`
[INFO] [stderr]     |
[INFO] [stderr]     = note: `#[warn(non_snake_case)]` (part of `#[warn(nonstandard_style)]`) on by default
[INFO] [stderr] 
[INFO] [stderr] warning: `fibpetokenizer` (lib) generated 11 warnings (run `cargo fix --lib -p fibpetokenizer` to apply 8 suggestions)
[INFO] [stderr] warning: unused import: `core::hash`
[INFO] [stderr]  --> src/main.rs:1:5
[INFO] [stderr]   |
[INFO] [stderr] 1 | use core::hash;
[INFO] [stderr]   |     ^^^^^^^^^^
[INFO] [stderr]   |
[INFO] [stderr]   = note: `#[warn(unused_imports)]` (part of `#[warn(unused)]`) on by default
[INFO] [stderr] 
[INFO] [stderr] warning: unused imports: `HashMap`, `HashSet`, `any::type_name`, `fmt::format`, and `usize`
[INFO] [stderr]  --> src/main.rs:2:11
[INFO] [stderr]   |
[INFO] [stderr] 2 | use std::{any::type_name, collections::{HashMap, HashSet}, fmt::format, usize};
[INFO] [stderr]   |           ^^^^^^^^^^^^^^                ^^^^^^^  ^^^^^^^   ^^^^^^^^^^^  ^^^^^
[INFO] [stderr] 
[INFO] [stderr] warning: unused import: `std::fs`
[INFO] [stderr]  --> src/main.rs:3:5
[INFO] [stderr]   |
[INFO] [stderr] 3 | use std::fs;
[INFO] [stderr]   |     ^^^^^^^
[INFO] [stderr] 
[INFO] [stderr] warning: unused import: `std::io`
[INFO] [stderr]  --> src/main.rs:4:5
[INFO] [stderr]   |
[INFO] [stderr] 4 | use std::io;
[INFO] [stderr]   |     ^^^^^^^
[INFO] [stderr] 
[INFO] [stderr] warning: unused imports: `error` and `hash`
[INFO] [stderr]  --> src/bpe_tokenizer.rs:1:12
[INFO] [stderr]   |
[INFO] [stderr] 1 | use core::{error, hash};
[INFO] [stderr]   |            ^^^^^  ^^^^
[INFO] [stderr] 
[INFO] [stderr] warning: unused variable: `special_tokens2`
[INFO] [stderr]   --> src/main.rs:21:9
[INFO] [stderr]    |
[INFO] [stderr] 21 |     let special_tokens2:Vec<String>=Vec::new();
[INFO] [stderr]    |         ^^^^^^^^^^^^^^^ help: if this is intentional, prefix it with an underscore: `_special_tokens2`
[INFO] [stderr] 
[INFO] [stderr] warning: variants `JsonFileNotExists`, `DirNotExists`, and `IdNotExists` are never constructed
[INFO] [stderr]   --> src/bpe_tokenizer.rs:41:5
[INFO] [stderr]    |
[INFO] [stderr] 30 | pub enum TokenizerError {
[INFO] [stderr]    |          -------------- variants in this enum
[INFO] [stderr] ...
[INFO] [stderr] 41 |     JsonFileNotExists(String),
[INFO] [stderr]    |     ^^^^^^^^^^^^^^^^^
[INFO] [stderr] ...
[INFO] [stderr] 44 |     DirNotExists(String),
[INFO] [stderr]    |     ^^^^^^^^^^^^
[INFO] [stderr] ...
[INFO] [stderr] 50 |     IdNotExists(String),
[INFO] [stderr]    |     ^^^^^^^^^^^
[INFO] [stderr]    |
[INFO] [stderr]    = note: `TokenizerError` has a derived impl for the trait `Debug`, but this is intentionally ignored during dead code analysis
[INFO] [stderr]    = note: `#[warn(dead_code)]` (part of `#[warn(unused)]`) on by default
[INFO] [stderr] 
[INFO] [stderr] warning: associated function `new_from_pretrained` is never used
[INFO] [stderr]    --> src/bpe_tokenizer.rs:130:12
[INFO] [stderr]     |
[INFO] [stderr] 108 | impl BpeTokenizer {
[INFO] [stderr]     | ----------------- associated function in this implementation
[INFO] [stderr] ...
[INFO] [stderr] 130 |     pub fn new_from_pretrained(files_path:&str)->Self{
[INFO] [stderr]     |            ^^^^^^^^^^^^^^^^^^^
[INFO] [stderr] 
[INFO] [stderr] warning: methods `display`, `get_id_by_token`, `get_token_by_id`, and `get_merge_pair_from_pairs_to_merge_and_ranks` are never used
[INFO] [stderr]    --> src/bpe_tokenizer.rs:646:8
[INFO] [stderr]     |
[INFO] [stderr] 187 | impl BpeTokenizer {
[INFO] [stderr]     | ----------------- methods in this implementation
[INFO] [stderr] ...
[INFO] [stderr] 646 | pub fn display(& self){
[INFO] [stderr]     |        ^^^^^^^
[INFO] [stderr] ...
[INFO] [stderr] 654 | pub fn get_id_by_token(&self,token:String)->Result<u32,TokenizerError>{
[INFO] [stderr]     |        ^^^^^^^^^^^^^^^
[INFO] [stderr] ...
[INFO] [stderr] 663 | pub fn get_token_by_id(&self,id:u32)->Result<String,TokenizerError>{
[INFO] [stderr]     |        ^^^^^^^^^^^^^^^
[INFO] [stderr] ...
[INFO] [stderr] 673 | fn get_merge_pair_from_pairs_to_merge_and_ranks(&self,tuple:(u32,u32)){
[INFO] [stderr]     |    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[INFO] [stderr] 
[INFO] [stderr] warning: function `extract_from_filepath_json_and_populate_bpetokenizer_struct` is never used
[INFO] [stderr]    --> src/bpe_tokenizer.rs:702:4
[INFO] [stderr]     |
[INFO] [stderr] 702 | fn extract_from_filepath_json_and_populate_bpetokenizer_struct(files_path:&str)-> Result<(Config,TokenTable,Vec<(u32,u32)>,HashMap<...
[INFO] [stderr]     |    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[INFO] [stderr] 
[INFO] [stderr] warning: function `read_config` is never used
[INFO] [stderr]    --> src/bpe_tokenizer.rs:726:4
[INFO] [stderr]     |
[INFO] [stderr] 726 | fn read_config(files_path:&str)->Result<Config,TokenizerError>{
[INFO] [stderr]     |    ^^^^^^^^^^^
[INFO] [stderr] 
[INFO] [stderr] warning: function `read_token_table` is never used
[INFO] [stderr]    --> src/bpe_tokenizer.rs:733:8
[INFO] [stderr]     |
[INFO] [stderr] 733 |     fn read_token_table(files_path:&str)->Result<TokenTable,TokenizerError>{
[INFO] [stderr]     |        ^^^^^^^^^^^^^^^^
[INFO] [stderr] 
[INFO] [stderr] warning: function `read_merge_rules` is never used
[INFO] [stderr]    --> src/bpe_tokenizer.rs:740:8
[INFO] [stderr]     |
[INFO] [stderr] 740 |     fn read_merge_rules(files_path:&str)->Result<Vec<(u32,u32)>,TokenizerError>{
[INFO] [stderr]     |        ^^^^^^^^^^^^^^^^
[INFO] [stderr] 
[INFO] [stderr] warning: function `read_ranks` is never used
[INFO] [stderr]    --> src/bpe_tokenizer.rs:746:8
[INFO] [stderr]     |
[INFO] [stderr] 746 |     fn read_ranks(files_path:&str)-> Result<HashMap<(u32, u32), usize>,TokenizerError>{
[INFO] [stderr]     |        ^^^^^^^^^^
[INFO] [stderr] 
[INFO] [stderr] warning: function `read_pairs_to_merge` is never used
[INFO] [stderr]    --> src/bpe_tokenizer.rs:760:8
[INFO] [stderr]     |
[INFO] [stderr] 760 |     fn read_pairs_to_merge(files_path:&str)-> Result<HashMap<(u32, u32), u32>, TokenizerError>{
[INFO] [stderr]     |        ^^^^^^^^^^^^^^^^^^^
[INFO] [stderr] 
[INFO] [stderr] warning: function `vocab_builder` is never used
[INFO] [stderr]    --> src/bpe_tokenizer.rs:793:8
[INFO] [stderr]     |
[INFO] [stderr] 793 | pub fn vocab_builder(pretokenized_text:&[String],vocab:&mut Vec<Word>, table: &mut TokenTable,pair_occurrences: &mut HashMap<(u32, ...
[INFO] [stderr]     |        ^^^^^^^^^^^^^
[INFO] [stderr] 
[INFO] [stderr] warning: method `tokens` is never used
[INFO] [stderr]   --> src/tokenizer_tables.rs:47:12
[INFO] [stderr]    |
[INFO] [stderr] 13 | impl TokenTable{
[INFO] [stderr]    | --------------- method in this implementation
[INFO] [stderr] ...
[INFO] [stderr] 47 |     pub fn tokens(&self) -> &[Arc<str>]{
[INFO] [stderr]    |            ^^^^^^
[INFO] [stderr] 
[INFO] [stderr] warning: `fibpetokenizer` (bin "fibpetokenizer" test) generated 26 warnings (9 duplicates) (run `cargo fix --bin "fibpetokenizer" -p fibpetokenizer --tests` to apply 6 suggestions)
[INFO] [stderr] warning: `fibpetokenizer` (lib test) generated 11 warnings (11 duplicates)
[INFO] [stderr]     Finished `test` profile [unoptimized + debuginfo] target(s) in 0.08s
[INFO] [stderr]      Running unittests src/lib.rs (/opt/rustwide/target/debug/deps/fibpetokenizer-7648d947b3123b37)
[INFO] [stdout] 
[INFO] [stdout] running 0 tests
[INFO] [stdout] 
[INFO] [stdout] test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s
[INFO] [stdout] 
[INFO] [stderr]      Running unittests src/main.rs (/opt/rustwide/target/debug/deps/fibpetokenizer-34522d5642f833f5)
[INFO] [stdout] 
[INFO] [stdout] running 0 tests
[INFO] [stdout] 
[INFO] [stdout] test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s
[INFO] [stdout] 
[INFO] [stderr]    Doc-tests fibpetokenizer
[INFO] [stderr] warning: unnecessary parentheses around pattern
[INFO] [stderr]    --> src/bpe_tokenizer.rs:884:9
[INFO] [stderr]     |
[INFO] [stderr] 884 |     for (word) in vocab{
[INFO] [stderr]     |         ^    ^
[INFO] [stderr]     |
[INFO] [stderr]     = note: `#[warn(unused_parens)]` (part of `#[warn(unused)]`) on by default
[INFO] [stderr] help: remove these parentheses
[INFO] [stderr]     |
[INFO] [stderr] 884 -     for (word) in vocab{
[INFO] [stderr] 884 +     for word  in vocab{
[INFO] [stderr]     |
[INFO] [stderr] 
[INFO] [stderr] warning: 1 warning emitted
[INFO] [stderr] 
[INFO] [stdout] 
[INFO] [stdout] running 1 test
[INFO] [stdout] test src/lib.rs - (line 14) ... FAILED
[INFO] [stdout] 
[INFO] [stdout] failures:
[INFO] [stdout] 
[INFO] [stdout] ---- src/lib.rs - (line 14) stdout ----
[INFO] [stdout] Test executable failed (exit status: 1).
[INFO] [stdout] 
[INFO] [stdout] stderr:
[INFO] [stdout] Error reading file: No such file or directory (os error 2)
[INFO] [stdout] 
[INFO] [stdout] 
[INFO] [stdout] 
[INFO] [stdout] failures:
[INFO] [stdout]     src/lib.rs - (line 14)
[INFO] [stdout] 
[INFO] [stdout] test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.35s
[INFO] [stdout] 
[INFO] [stderr] error: doctest failed, to rerun pass `--doc`
[INFO] running `Command { std: "docker" "inspect" "bbeee354ecacfba6d8f6bdae76384fa51cacd4c6e9c04f4b6f228caaf5d6b076", kill_on_drop: false }`
[INFO] running `Command { std: "docker" "rm" "-f" "bbeee354ecacfba6d8f6bdae76384fa51cacd4c6e9c04f4b6f228caaf5d6b076", kill_on_drop: false }`
[INFO] [stdout] bbeee354ecacfba6d8f6bdae76384fa51cacd4c6e9c04f4b6f228caaf5d6b076
