[INFO] cloning repository https://github.com/IronBlood/cs336-rs [INFO] running `Command { std: "git" "-c" "credential.helper=" "-c" "credential.helper=/workspace/cargo-home/bin/git-credential-null" "clone" "--bare" "https://github.com/IronBlood/cs336-rs" "/workspace/cache/git-repos/https%3A%2F%2Fgithub.com%2FIronBlood%2Fcs336-rs", kill_on_drop: false }` [INFO] [stderr] Cloning into bare repository '/workspace/cache/git-repos/https%3A%2F%2Fgithub.com%2FIronBlood%2Fcs336-rs'... [INFO] running `Command { std: "git" "rev-parse" "HEAD", kill_on_drop: false }` [INFO] [stdout] 4fbeafafa976b18c5841c678ec439374d32ae023 [INFO] testing IronBlood/cs336-rs against 1.100.0-beta.1+cargoflags=--release for beta-release-1.100-2 [INFO] running `Command { std: "git" "clone" "/workspace/cache/git-repos/https%3A%2F%2Fgithub.com%2FIronBlood%2Fcs336-rs" "/workspace/builds/worker-6-tc2/source", kill_on_drop: false }` [INFO] [stderr] Cloning into '/workspace/builds/worker-6-tc2/source'... [INFO] [stderr] done. [INFO] started tweaking git repo https://github.com/IronBlood/cs336-rs [INFO] finished tweaking git repo https://github.com/IronBlood/cs336-rs [INFO] tweaked toml for git repo https://github.com/IronBlood/cs336-rs written to /workspace/builds/worker-6-tc2/source/Cargo.toml [INFO] validating manifest of git repo https://github.com/IronBlood/cs336-rs on toolchain 1.100.0-beta.1 [INFO] running `Command { std: CARGO_HOME="/workspace/cargo-home" RUSTUP_HOME="/workspace/rustup-home" "/workspace/cargo-home/bin/cargo" "+1.100.0-beta.1" "metadata" "--manifest-path" "Cargo.toml" "--no-deps", kill_on_drop: false }` [INFO] crate git repo https://github.com/IronBlood/cs336-rs already has a lockfile, it will not be regenerated [INFO] running `Command { std: CARGO_HOME="/workspace/cargo-home" RUSTUP_HOME="/workspace/rustup-home" "/workspace/cargo-home/bin/cargo" "+1.100.0-beta.1" "fetch" "--manifest-path" "Cargo.toml", kill_on_drop: false }` [INFO] [stderr] Blocking waiting for file lock on package cache [INFO] [stderr] Blocking waiting for file lock on package cache [INFO] running `Command { std: "docker" "create" "-v" "/var/lib/crater-agent-workspace/builds/worker-6-tc2/source:/opt/rustwide/workdir:ro,Z" "-v" "/var/lib/crater-agent-workspace/builds/worker-6-tc2/target:/opt/rustwide/target:rw,Z" "-v" "/var/lib/crater-agent-workspace/cargo-home:/opt/rustwide/cargo-home:ro,Z" "-v" "/var/lib/crater-agent-workspace/rustup-home:/opt/rustwide/rustup-home:ro,Z" "-m" "1610612736" "--network" "none" "ghcr.io/rust-lang/crates-build-env/linux@sha256:77db811e55d90add9212f6832c23229e3de5f1c1f9905e8cb07319ea15df73ac" "sleep" "infinity", kill_on_drop: false }` [INFO] [stdout] f51cb837820394ab7a9ab2fb1151b49d5b8d7e00ecffc2b1ef56a26ebd4ea8a9 [INFO] running `Command { std: "docker" "start" "f51cb837820394ab7a9ab2fb1151b49d5b8d7e00ecffc2b1ef56a26ebd4ea8a9", kill_on_drop: false }` [INFO] running `Command { std: "docker" "inspect" "f51cb837820394ab7a9ab2fb1151b49d5b8d7e00ecffc2b1ef56a26ebd4ea8a9", kill_on_drop: false }` [INFO] running `Command { std: "docker" "exec" "-e" "SOURCE_DIR=/opt/rustwide/workdir" "-e" "CARGO_HOME=/opt/rustwide/cargo-home" "-e" "RUSTUP_HOME=/opt/rustwide/rustup-home" "-e" "CARGO_TARGET_DIR=/opt/rustwide/target" "-w" "/opt/rustwide/workdir" "--user" "0:0" "f51cb837820394ab7a9ab2fb1151b49d5b8d7e00ecffc2b1ef56a26ebd4ea8a9" "/opt/rustwide/cargo-home/bin/cargo" "+1.100.0-beta.1" "metadata" "--no-deps" "--format-version=1", kill_on_drop: false }` [INFO] running `Command { std: "docker" "inspect" "f51cb837820394ab7a9ab2fb1151b49d5b8d7e00ecffc2b1ef56a26ebd4ea8a9", kill_on_drop: false }` [INFO] running `Command { std: "docker" "exec" "-e" "SOURCE_DIR=/opt/rustwide/workdir" "-e" "CARGO_HOME=/opt/rustwide/cargo-home" "-e" "RUSTUP_HOME=/opt/rustwide/rustup-home" "-e" "CARGO_TARGET_DIR=/opt/rustwide/target" "-e" "CARGO_INCREMENTAL=0" "-e" "RUST_BACKTRACE=full" "-e" "RUSTFLAGS=--cap-lints=warn" "-e" "RUSTDOCFLAGS=--cap-lints=warn" "-w" "/opt/rustwide/workdir" "--user" "0:0" "f51cb837820394ab7a9ab2fb1151b49d5b8d7e00ecffc2b1ef56a26ebd4ea8a9" "/opt/rustwide/cargo-home/bin/cargo" "+1.100.0-beta.1" "build" "--frozen" "--message-format=json" "--release", kill_on_drop: false }` [INFO] [stderr] Compiling cs336_rs v0.1.0 (/opt/rustwide/workdir) [INFO] [stderr] Finished `release` profile [optimized] target(s) in 3.89s [INFO] running `Command { std: "docker" "inspect" "f51cb837820394ab7a9ab2fb1151b49d5b8d7e00ecffc2b1ef56a26ebd4ea8a9", kill_on_drop: false }` [INFO] running `Command { std: "docker" "exec" "-e" "SOURCE_DIR=/opt/rustwide/workdir" "-e" "CARGO_HOME=/opt/rustwide/cargo-home" "-e" "RUSTUP_HOME=/opt/rustwide/rustup-home" "-e" "CARGO_TARGET_DIR=/opt/rustwide/target" "-e" "CARGO_INCREMENTAL=0" "-e" "RUST_BACKTRACE=full" "-e" "RUSTFLAGS=--cap-lints=warn" "-e" "RUSTDOCFLAGS=--cap-lints=warn" "-w" "/opt/rustwide/workdir" "--user" "0:0" "f51cb837820394ab7a9ab2fb1151b49d5b8d7e00ecffc2b1ef56a26ebd4ea8a9" "/opt/rustwide/cargo-home/bin/cargo" "+1.100.0-beta.1" "test" "--frozen" "--no-run" "--message-format=json" "--release", kill_on_drop: false }` [INFO] [stderr] Compiling serde_core v1.0.228 [INFO] [stderr] Compiling serde_json v1.0.150 [INFO] [stderr] Compiling memchr v2.8.2 [INFO] [stderr] Compiling cs336_rs v0.1.0 (/opt/rustwide/workdir) [INFO] [stderr] Finished `release` profile [optimized] target(s) in 8.94s [INFO] running `Command { std: "docker" "inspect" "f51cb837820394ab7a9ab2fb1151b49d5b8d7e00ecffc2b1ef56a26ebd4ea8a9", kill_on_drop: false }` [INFO] running `Command { std: "docker" "exec" "-e" "SOURCE_DIR=/opt/rustwide/workdir" "-e" "CARGO_HOME=/opt/rustwide/cargo-home" "-e" "RUSTUP_HOME=/opt/rustwide/rustup-home" "-e" "CARGO_TARGET_DIR=/opt/rustwide/target" "-e" "CARGO_INCREMENTAL=0" "-e" "RUST_BACKTRACE=full" "-e" "RUSTFLAGS=--cap-lints=warn" "-e" "RUSTDOCFLAGS=--cap-lints=warn" "-w" "/opt/rustwide/workdir" "--user" "0:0" "f51cb837820394ab7a9ab2fb1151b49d5b8d7e00ecffc2b1ef56a26ebd4ea8a9" "/opt/rustwide/cargo-home/bin/cargo" "+1.100.0-beta.1" "test" "--frozen" "--release", kill_on_drop: false }` [INFO] [stderr] Finished `release` profile [optimized] target(s) in 0.03s [INFO] [stderr] Running unittests src/lib.rs (/opt/rustwide/target/release/build/cs336_rs/5c79d6cd135fe23f/out/cs336_rs-5c79d6cd135fe23f) [INFO] [stdout] [INFO] [stdout] running 21 tests [INFO] [stdout] test file::test::test_serialize_merges ... ok [INFO] [stdout] test utils::tests::test_find_chunk_boundaries_0_chunk_should_fall_to_1 ... ok [INFO] [stdout] test encode::test::test_bs ... ok [INFO] [stdout] test encode::test::test_cs ... ok [INFO] [stdout] test utils::tests::test_find_special_tokens_should_return_the_smallest_occurance ... ok [INFO] [stdout] test file::test::test_serialize_vocab ... ok [INFO] [stdout] test file::test::test_parse_merges ... ok [INFO] [stdout] test file::test::test_parse_vocab ... ok [INFO] [stdout] test utils::tests::test_find_special_tokens_with_empty_special_tokens ... ok [INFO] [stdout] test utils::tests::test_find_chunk_boundaries_1_chunk_should_return_start_and_end ... ok [INFO] [stdout] test regex::tests::escape_literal_should_quote_regex_metacharacters ... ok [INFO] [stdout] test regex::tests::unicode_properties_should_match_utf8_codepoints ... ok [INFO] [stdout] test utils::tests::test_cmp_tuple ... ok [INFO] [stdout] test utils::tests::test_find_chunk_boundaries_should_return_the_correct_location ... ok [INFO] [stdout] test utils::tests::test_replace_pair_in_token ... ok [INFO] [stdout] test utils::tests::test_largest_pair ... ok [INFO] [stdout] test regex::tests::regex_test ... ok [INFO] [stdout] test utils::tests::test_find_special_tokens_should_return_none ... ok [INFO] [stdout] test utils::tests::test_find_pretoken_spans ... ok [INFO] [stdout] test utils::tests::test_find_token_freq_map ... ok [INFO] [stderr] Running unittests src/main.rs (/opt/rustwide/target/release/build/cs336_rs/8f69e420070fb46d/out/cs336_rs-8f69e420070fb46d) [INFO] [stdout] test utils::tests::test_train_bpe ... ok [INFO] [stdout] [INFO] [stdout] test result: ok. 21 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.01s [INFO] [stdout] [INFO] [stdout] [INFO] [stdout] running 0 tests [INFO] [stdout] [INFO] [stdout] test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s [INFO] [stdout] [INFO] [stderr] Running unittests src/bin/decode_file.rs (/opt/rustwide/target/release/build/cs336_rs/738d1531d1f7aba6/out/decode_file-738d1531d1f7aba6) [INFO] [stdout] [INFO] [stdout] running 0 tests [INFO] [stdout] [INFO] [stdout] test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s [INFO] [stdout] [INFO] [stderr] Running unittests src/bin/encode_file.rs (/opt/rustwide/target/release/build/cs336_rs/3fabd4e1e356ee61/out/encode_file-3fabd4e1e356ee61) [INFO] [stdout] [INFO] [stdout] running 0 tests [INFO] [stdout] [INFO] [stdout] test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s [INFO] [stdout] [INFO] [stderr] Running unittests src/bin/tokenize_samples.rs (/opt/rustwide/target/release/build/cs336_rs/56e8bfb033bab7de/out/tokenize_samples-56e8bfb033bab7de) [INFO] [stdout] [INFO] [stdout] running 0 tests [INFO] [stdout] [INFO] [stdout] test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s [INFO] [stdout] [INFO] [stderr] Running unittests src/bin/write_u16_sample.rs (/opt/rustwide/target/release/build/cs336_rs/1fa72ca99ba54887/out/write_u16_sample-1fa72ca99ba54887) [INFO] [stdout] [INFO] [stdout] running 0 tests [INFO] [stdout] [INFO] [stdout] test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s [INFO] [stdout] [INFO] [stderr] Running tests/bpe_integration.rs (/opt/rustwide/target/release/build/cs336_rs/7701bfa70ec42e6a/out/bpe_integration-7701bfa70ec42e6a) [INFO] [stdout] [INFO] [stdout] running 2 tests [INFO] [stdout] test test_train_bpe ... ok [INFO] [stdout] test test_train_bpe_speed ... ok [INFO] [stdout] [INFO] [stdout] test result: ok. 2 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 1.45s [INFO] [stdout] [INFO] [stderr] Running tests/common.rs (/opt/rustwide/target/release/build/cs336_rs/07933f8940b0ac16/out/common-07933f8940b0ac16) [INFO] [stdout] [INFO] [stdout] running 0 tests [INFO] [stderr] Running tests/tokenizer_integration.rs (/opt/rustwide/target/release/build/cs336_rs/df567f19272eb1af/out/tokenizer_integration-df567f19272eb1af) [INFO] [stdout] [INFO] [stdout] test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s [INFO] [stdout] [INFO] [stdout] [INFO] [stdout] running 16 tests [INFO] [stdout] test test_continues ... ok [INFO] [stdout] test test_roundtrip_single_character ... ok [INFO] [stdout] test test_roundtrip_single_unicode_character ... ok [INFO] [stdout] test test_german_roundtrip ... ok [INFO] [stdout] test test_roundtrip_empty ... ok [INFO] [stdout] test test_tinystories_sample_roundtrip ... ok [INFO] [stdout] test test_encode_special_token_trailing_newlines ... ok [INFO] [stdout] test test_encode_special_token_double_newline_non_whitespace ... ok [INFO] [stdout] test test_address_roundtrip ... ok [INFO] [stdout] test test_overlapping_special_tokens ... ok [INFO] [stdout] test test_tinystories_sample_debug ... ok [INFO] [stdout] test test_roundtrip_unicode_string ... ok [INFO] [stdout] test test_roundtrip_unicode_string_with_special_tokens ... ok [INFO] [stdout] test test_roundtrip_ascii_string ... ok [INFO] [stdout] test test_roundtrip_special_tokens ... ok [INFO] [stdout] test test_tinystories_5m_roundtrip ... ok [INFO] [stdout] [INFO] [stdout] test result: ok. 16 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 1.62s [INFO] [stdout] [INFO] [stderr] Doc-tests cs336_rs [INFO] [stdout] [INFO] [stdout] running 0 tests [INFO] [stdout] [INFO] [stdout] test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s [INFO] [stdout] [INFO] running `Command { std: "docker" "inspect" "f51cb837820394ab7a9ab2fb1151b49d5b8d7e00ecffc2b1ef56a26ebd4ea8a9", kill_on_drop: false }` [INFO] running `Command { std: "docker" "rm" "-f" "f51cb837820394ab7a9ab2fb1151b49d5b8d7e00ecffc2b1ef56a26ebd4ea8a9", kill_on_drop: false }` [INFO] [stdout] f51cb837820394ab7a9ab2fb1151b49d5b8d7e00ecffc2b1ef56a26ebd4ea8a9