[INFO] cloning repository https://github.com/mklf/bert_tokenizer
[INFO] running `Command { std: "git" "-c" "credential.helper=" "-c" "credential.helper=/workspace/cargo-home/bin/git-credential-null" "clone" "--bare" "https://github.com/mklf/bert_tokenizer" "/workspace/cache/git-repos/https%3A%2F%2Fgithub.com%2Fmklf%2Fbert_tokenizer", kill_on_drop: false }`
[INFO] [stderr] Cloning into bare repository '/workspace/cache/git-repos/https%3A%2F%2Fgithub.com%2Fmklf%2Fbert_tokenizer'...
[INFO] running `Command { std: "git" "rev-parse" "HEAD", kill_on_drop: false }`
[INFO] [stdout] a3d1ecb983469aa4f976eb6e51475b55609d048d
[INFO] testing mklf/bert_tokenizer against beta-2026-01-20 for beta-1.94-1
[INFO] running `Command { std: "git" "clone" "/workspace/cache/git-repos/https%3A%2F%2Fgithub.com%2Fmklf%2Fbert_tokenizer" "/workspace/builds/worker-7-tc2/source", kill_on_drop: false }`
[INFO] [stderr] Cloning into '/workspace/builds/worker-7-tc2/source'...
[INFO] [stderr] done.
[INFO] started tweaking git repo https://github.com/mklf/bert_tokenizer
[INFO] finished tweaking git repo https://github.com/mklf/bert_tokenizer
[INFO] tweaked toml for git repo https://github.com/mklf/bert_tokenizer written to /workspace/builds/worker-7-tc2/source/Cargo.toml
[INFO] validating manifest of git repo https://github.com/mklf/bert_tokenizer on toolchain beta-2026-01-20
[INFO] running `Command { std: CARGO_HOME="/workspace/cargo-home" RUSTUP_HOME="/workspace/rustup-home" "/workspace/cargo-home/bin/cargo" "+beta-2026-01-20" "metadata" "--manifest-path" "Cargo.toml" "--no-deps", kill_on_drop: false }`
[INFO] crate git repo https://github.com/mklf/bert_tokenizer already has a lockfile, it will not be regenerated
[INFO] running `Command { std: CARGO_HOME="/workspace/cargo-home" RUSTUP_HOME="/workspace/rustup-home" "/workspace/cargo-home/bin/cargo" "+beta-2026-01-20" "fetch" "--manifest-path" "Cargo.toml", kill_on_drop: false }`
[INFO] [stderr] warning: no edition set: defaulting to the 2015 edition while the latest is 2024
[INFO] [stderr]     Blocking waiting for file lock on package cache
[INFO] [stderr]     Updating crates.io index
[INFO] [stderr]     Blocking waiting for file lock on package cache
[INFO] [stderr]  Downloading crates ...
[INFO] [stderr]   Downloaded indexmap v1.6.0
[INFO] [stderr]   Downloaded tinyvec v0.3.4
[INFO] [stderr]   Downloaded unicode_categories v0.1.1
[INFO] [stderr]   Downloaded hashbrown v0.9.0
[INFO] [stderr]   Downloaded unicode-normalization v0.1.13
[INFO] running `Command { std: "docker" "create" "-v" "/var/lib/crater-agent-workspace/builds/worker-7-tc2/target:/opt/rustwide/target:rw,Z" "-v" "/var/lib/crater-agent-workspace/builds/worker-7-tc2/source:/opt/rustwide/workdir:ro,Z" "-v" "/var/lib/crater-agent-workspace/cargo-home:/opt/rustwide/cargo-home:ro,Z" "-v" "/var/lib/crater-agent-workspace/rustup-home:/opt/rustwide/rustup-home:ro,Z" "-e" "SOURCE_DIR=/opt/rustwide/workdir" "-e" "CARGO_TARGET_DIR=/opt/rustwide/target" "-e" "CARGO_HOME=/opt/rustwide/cargo-home" "-e" "RUSTUP_HOME=/opt/rustwide/rustup-home" "-w" "/opt/rustwide/workdir" "-m" "1610612736" "--user" "0:0" "--network" "none" "ghcr.io/rust-lang/crates-build-env/linux@sha256:98afbf2d16093dec2546ff8915fddc74e65664aa03fc460b3712b1c2c54a33e4" "/opt/rustwide/cargo-home/bin/cargo" "+beta-2026-01-20" "metadata" "--no-deps" "--format-version=1", kill_on_drop: false }`
[INFO] [stdout] cdbf8f1371f3e2bbebc8a6c6449b978bb593460f3030cfb39f3f8c1345677c80
[INFO] running `Command { std: "docker" "start" "-a" "cdbf8f1371f3e2bbebc8a6c6449b978bb593460f3030cfb39f3f8c1345677c80", kill_on_drop: false }`
[INFO] running `Command { std: "docker" "inspect" "cdbf8f1371f3e2bbebc8a6c6449b978bb593460f3030cfb39f3f8c1345677c80", kill_on_drop: false }`
[INFO] running `Command { std: "docker" "rm" "-f" "cdbf8f1371f3e2bbebc8a6c6449b978bb593460f3030cfb39f3f8c1345677c80", kill_on_drop: false }`
[INFO] [stdout] cdbf8f1371f3e2bbebc8a6c6449b978bb593460f3030cfb39f3f8c1345677c80
[INFO] running `Command { std: "docker" "create" "-v" "/var/lib/crater-agent-workspace/builds/worker-7-tc2/target:/opt/rustwide/target:rw,Z" "-v" "/var/lib/crater-agent-workspace/builds/worker-7-tc2/source:/opt/rustwide/workdir:ro,Z" "-v" "/var/lib/crater-agent-workspace/cargo-home:/opt/rustwide/cargo-home:ro,Z" "-v" "/var/lib/crater-agent-workspace/rustup-home:/opt/rustwide/rustup-home:ro,Z" "-e" "SOURCE_DIR=/opt/rustwide/workdir" "-e" "CARGO_TARGET_DIR=/opt/rustwide/target" "-e" "CARGO_INCREMENTAL=0" "-e" "RUST_BACKTRACE=full" "-e" "RUSTFLAGS=--cap-lints=warn" "-e" "RUSTDOCFLAGS=--cap-lints=warn" "-e" "CARGO_HOME=/opt/rustwide/cargo-home" "-e" "RUSTUP_HOME=/opt/rustwide/rustup-home" "-w" "/opt/rustwide/workdir" "-m" "1610612736" "--user" "0:0" "--network" "none" "ghcr.io/rust-lang/crates-build-env/linux@sha256:98afbf2d16093dec2546ff8915fddc74e65664aa03fc460b3712b1c2c54a33e4" "/opt/rustwide/cargo-home/bin/cargo" "+beta-2026-01-20" "build" "--frozen" "--message-format=json", kill_on_drop: false }`
[INFO] [stdout] 4e7f8c0f407adc6d7d16ede08f12865c8d393ea91cfd388f3dafcc7cad1238a6
[INFO] running `Command { std: "docker" "start" "-a" "4e7f8c0f407adc6d7d16ede08f12865c8d393ea91cfd388f3dafcc7cad1238a6", kill_on_drop: false }`
[INFO] [stderr] warning: no edition set: defaulting to the 2015 edition while the latest is 2024
[INFO] [stderr]    Compiling autocfg v1.0.1
[INFO] [stderr]    Compiling hashbrown v0.9.0
[INFO] [stderr]    Compiling tinyvec v0.3.4
[INFO] [stderr]    Compiling unicode_categories v0.1.1
[INFO] [stderr]    Compiling unicode-normalization v0.1.13
[INFO] [stderr]    Compiling indexmap v1.6.0
[INFO] [stderr]    Compiling bert_tokenizer v0.1.0 (/opt/rustwide/workdir)
[INFO] [stdout] warning: unused return value of `Box::<T>::from_raw` that must be used
[INFO] [stdout]   --> src/ffi.rs:37:14
[INFO] [stdout]    |
[INFO] [stdout] 37 |     unsafe { Box::from_raw(tokenizer) };
[INFO] [stdout]    |              ^^^^^^^^^^^^^^^^^^^^^^^^
[INFO] [stdout]    |
[INFO] [stdout]    = note: call `drop(Box::from_raw(ptr))` if you intend to drop the `Box`
[INFO] [stdout]    = note: `#[warn(unused_must_use)]` (part of `#[warn(unused)]`) on by default
[INFO] [stdout] help: use `let _ = ...` to ignore the resulting value
[INFO] [stdout]    |
[INFO] [stdout] 37 |     unsafe { let _ = Box::from_raw(tokenizer); };
[INFO] [stdout]    |              +++++++                         +
[INFO] [stdout] 
[INFO] [stdout] 
[INFO] [stderr]     Finished `dev` profile [unoptimized + debuginfo] target(s) in 1.03s
[INFO] running `Command { std: "docker" "inspect" "4e7f8c0f407adc6d7d16ede08f12865c8d393ea91cfd388f3dafcc7cad1238a6", kill_on_drop: false }`
[INFO] running `Command { std: "docker" "rm" "-f" "4e7f8c0f407adc6d7d16ede08f12865c8d393ea91cfd388f3dafcc7cad1238a6", kill_on_drop: false }`
[INFO] [stdout] 4e7f8c0f407adc6d7d16ede08f12865c8d393ea91cfd388f3dafcc7cad1238a6
[INFO] running `Command { std: "docker" "create" "-v" "/var/lib/crater-agent-workspace/builds/worker-7-tc2/target:/opt/rustwide/target:rw,Z" "-v" "/var/lib/crater-agent-workspace/builds/worker-7-tc2/source:/opt/rustwide/workdir:ro,Z" "-v" "/var/lib/crater-agent-workspace/cargo-home:/opt/rustwide/cargo-home:ro,Z" "-v" "/var/lib/crater-agent-workspace/rustup-home:/opt/rustwide/rustup-home:ro,Z" "-e" "SOURCE_DIR=/opt/rustwide/workdir" "-e" "CARGO_TARGET_DIR=/opt/rustwide/target" "-e" "CARGO_INCREMENTAL=0" "-e" "RUST_BACKTRACE=full" "-e" "RUSTFLAGS=--cap-lints=warn" "-e" "RUSTDOCFLAGS=--cap-lints=warn" "-e" "CARGO_HOME=/opt/rustwide/cargo-home" "-e" "RUSTUP_HOME=/opt/rustwide/rustup-home" "-w" "/opt/rustwide/workdir" "-m" "1610612736" "--user" "0:0" "--network" "none" "ghcr.io/rust-lang/crates-build-env/linux@sha256:98afbf2d16093dec2546ff8915fddc74e65664aa03fc460b3712b1c2c54a33e4" "/opt/rustwide/cargo-home/bin/cargo" "+beta-2026-01-20" "test" "--frozen" "--no-run" "--message-format=json", kill_on_drop: false }`
[INFO] [stdout] 1e1993688e78c7addd3c1ec903135ec6336637a933709b0cc48690bf31a1b1e8
[INFO] running `Command { std: "docker" "start" "-a" "1e1993688e78c7addd3c1ec903135ec6336637a933709b0cc48690bf31a1b1e8", kill_on_drop: false }`
[INFO] [stderr] warning: no edition set: defaulting to the 2015 edition while the latest is 2024
[INFO] [stderr]    Compiling bert_tokenizer v0.1.0 (/opt/rustwide/workdir)
[INFO] [stdout] warning: unused return value of `Box::<T>::from_raw` that must be used
[INFO] [stdout]   --> src/ffi.rs:37:14
[INFO] [stdout]    |
[INFO] [stdout] 37 |     unsafe { Box::from_raw(tokenizer) };
[INFO] [stdout]    |              ^^^^^^^^^^^^^^^^^^^^^^^^
[INFO] [stdout]    |
[INFO] [stdout]    = note: call `drop(Box::from_raw(ptr))` if you intend to drop the `Box`
[INFO] [stdout]    = note: `#[warn(unused_must_use)]` (part of `#[warn(unused)]`) on by default
[INFO] [stdout] help: use `let _ = ...` to ignore the resulting value
[INFO] [stdout]    |
[INFO] [stdout] 37 |     unsafe { let _ = Box::from_raw(tokenizer); };
[INFO] [stdout]    |              +++++++                         +
[INFO] [stdout] 
[INFO] [stdout] 
[INFO] [stderr]     Finished `test` profile [unoptimized + debuginfo] target(s) in 0.92s
[INFO] running `Command { std: "docker" "inspect" "1e1993688e78c7addd3c1ec903135ec6336637a933709b0cc48690bf31a1b1e8", kill_on_drop: false }`
[INFO] running `Command { std: "docker" "rm" "-f" "1e1993688e78c7addd3c1ec903135ec6336637a933709b0cc48690bf31a1b1e8", kill_on_drop: false }`
[INFO] [stdout] 1e1993688e78c7addd3c1ec903135ec6336637a933709b0cc48690bf31a1b1e8
[INFO] running `Command { std: "docker" "create" "-v" "/var/lib/crater-agent-workspace/builds/worker-7-tc2/target:/opt/rustwide/target:rw,Z" "-v" "/var/lib/crater-agent-workspace/builds/worker-7-tc2/source:/opt/rustwide/workdir:ro,Z" "-v" "/var/lib/crater-agent-workspace/cargo-home:/opt/rustwide/cargo-home:ro,Z" "-v" "/var/lib/crater-agent-workspace/rustup-home:/opt/rustwide/rustup-home:ro,Z" "-e" "SOURCE_DIR=/opt/rustwide/workdir" "-e" "CARGO_TARGET_DIR=/opt/rustwide/target" "-e" "CARGO_INCREMENTAL=0" "-e" "RUST_BACKTRACE=full" "-e" "RUSTFLAGS=--cap-lints=warn" "-e" "RUSTDOCFLAGS=--cap-lints=warn" "-e" "CARGO_HOME=/opt/rustwide/cargo-home" "-e" "RUSTUP_HOME=/opt/rustwide/rustup-home" "-w" "/opt/rustwide/workdir" "-m" "1610612736" "--user" "0:0" "--network" "none" "ghcr.io/rust-lang/crates-build-env/linux@sha256:98afbf2d16093dec2546ff8915fddc74e65664aa03fc460b3712b1c2c54a33e4" "/opt/rustwide/cargo-home/bin/cargo" "+beta-2026-01-20" "test" "--frozen", kill_on_drop: false }`
[INFO] [stdout] 39945011a994bdac3bbec4802e42126cb56397d9eb5e39bf59685c52ae6f05f2
[INFO] running `Command { std: "docker" "start" "-a" "39945011a994bdac3bbec4802e42126cb56397d9eb5e39bf59685c52ae6f05f2", kill_on_drop: false }`
[INFO] [stderr] warning: no edition set: defaulting to the 2015 edition while the latest is 2024
[INFO] [stderr] warning: unused return value of `Box::<T>::from_raw` that must be used
[INFO] [stderr]   --> src/ffi.rs:37:14
[INFO] [stderr]    |
[INFO] [stderr] 37 |     unsafe { Box::from_raw(tokenizer) };
[INFO] [stderr]    |              ^^^^^^^^^^^^^^^^^^^^^^^^
[INFO] [stderr]    |
[INFO] [stderr]    = note: call `drop(Box::from_raw(ptr))` if you intend to drop the `Box`
[INFO] [stderr]    = note: `#[warn(unused_must_use)]` (part of `#[warn(unused)]`) on by default
[INFO] [stderr] help: use `let _ = ...` to ignore the resulting value
[INFO] [stderr]    |
[INFO] [stderr] 37 |     unsafe { let _ = Box::from_raw(tokenizer); };
[INFO] [stderr]    |              +++++++                         +
[INFO] [stderr] 
[INFO] [stderr] warning: `bert_tokenizer` (lib test) generated 1 warning
[INFO] [stderr]     Finished `test` profile [unoptimized + debuginfo] target(s) in 0.02s
[INFO] [stderr]      Running unittests src/lib.rs (/opt/rustwide/target/debug/deps/bert_tokenizer-8c3c891be1ec28f0)
[INFO] [stdout] 
[INFO] [stdout] running 10 tests
[INFO] [stdout] test tokenization::test::test_basic_tokenizer_no_lower ... ok
[INFO] [stdout] test tokenization::test::test_chinese ... ok
[INFO] [stdout] test tokenization::test::test_is_punctuation ... ok
[INFO] [stdout] test tokenization::test::test_convert_tokens_to_ids ... ok
[INFO] [stdout] test tokenization::test::test_is_control ... ok
[INFO] [stdout] test tokenization::test::test_is_whitespace ... ok
[INFO] [stdout] test tokenization::test::test_basic_tokenizer_lower ... ok
[INFO] [stdout] test tokenization::test::test_wordpiece_tokenizer ... ok
[INFO] [stdout] test tokenization::test::test_full_tokenizer ... ok
[INFO] [stdout] test ffi::test::pipeline ... ok
[INFO] [stdout] 
[INFO] [stdout] test result: ok. 10 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.10s
[INFO] [stdout] 
[INFO] running `Command { std: "docker" "inspect" "39945011a994bdac3bbec4802e42126cb56397d9eb5e39bf59685c52ae6f05f2", kill_on_drop: false }`
[INFO] running `Command { std: "docker" "rm" "-f" "39945011a994bdac3bbec4802e42126cb56397d9eb5e39bf59685c52ae6f05f2", kill_on_drop: false }`
[INFO] [stdout] 39945011a994bdac3bbec4802e42126cb56397d9eb5e39bf59685c52ae6f05f2
