[INFO] cloning repository https://github.com/errorhugo/gpt-from-scratch-in-rust [INFO] running `Command { std: "git" "-c" "credential.helper=" "-c" "credential.helper=/workspace/cargo-home/bin/git-credential-null" "clone" "--bare" "https://github.com/errorhugo/gpt-from-scratch-in-rust" "/workspace/cache/git-repos/https%3A%2F%2Fgithub.com%2Ferrorhugo%2Fgpt-from-scratch-in-rust", kill_on_drop: false }` [INFO] [stderr] Cloning into bare repository '/workspace/cache/git-repos/https%3A%2F%2Fgithub.com%2Ferrorhugo%2Fgpt-from-scratch-in-rust'... [INFO] running `Command { std: "git" "rev-parse" "HEAD", kill_on_drop: false }` [INFO] [stdout] 7b10a2d92926a305a1b996a1e5a713262c435c0c [INFO] testing errorhugo/gpt-from-scratch-in-rust against master#cdb45c87e2cd43495379f7e867e3cc15dcee9f93 for pr-145838-1 [INFO] running `Command { std: "git" "clone" "/workspace/cache/git-repos/https%3A%2F%2Fgithub.com%2Ferrorhugo%2Fgpt-from-scratch-in-rust" "/workspace/builds/worker-4-tc1/source", kill_on_drop: false }` [INFO] [stderr] Cloning into '/workspace/builds/worker-4-tc1/source'... [INFO] [stderr] done. [INFO] started tweaking git repo https://github.com/errorhugo/gpt-from-scratch-in-rust [INFO] finished tweaking git repo https://github.com/errorhugo/gpt-from-scratch-in-rust [INFO] tweaked toml for git repo https://github.com/errorhugo/gpt-from-scratch-in-rust written to /workspace/builds/worker-4-tc1/source/Cargo.toml [INFO] validating manifest of git repo https://github.com/errorhugo/gpt-from-scratch-in-rust on toolchain cdb45c87e2cd43495379f7e867e3cc15dcee9f93 [INFO] running `Command { std: CARGO_HOME="/workspace/cargo-home" RUSTUP_HOME="/workspace/rustup-home" "/workspace/cargo-home/bin/cargo" "+cdb45c87e2cd43495379f7e867e3cc15dcee9f93" "metadata" "--manifest-path" "Cargo.toml" "--no-deps", kill_on_drop: false }` [INFO] crate git repo https://github.com/errorhugo/gpt-from-scratch-in-rust already has a lockfile, it will not be regenerated [INFO] running `Command { std: CARGO_HOME="/workspace/cargo-home" RUSTUP_HOME="/workspace/rustup-home" "/workspace/cargo-home/bin/cargo" "+cdb45c87e2cd43495379f7e867e3cc15dcee9f93" "fetch" "--manifest-path" "Cargo.toml", kill_on_drop: false }` [INFO] [stderr] Updating crates.io index [INFO] [stderr] Downloading crates ... [INFO] [stderr] Downloaded nichi v0.5.1 [INFO] [stderr] Downloaded twelf v0.15.0 [INFO] [stderr] Downloaded config-derive v0.15.0 [INFO] [stderr] Downloaded readable v0.16.0 [INFO] [stderr] Downloaded tokenizers v0.21.2 [INFO] running `Command { std: "docker" "create" "-v" "/var/lib/crater-agent-workspace/builds/worker-4-tc1/target:/opt/rustwide/target:rw,Z" "-v" "/var/lib/crater-agent-workspace/builds/worker-4-tc1/source:/opt/rustwide/workdir:ro,Z" "-v" "/var/lib/crater-agent-workspace/cargo-home:/opt/rustwide/cargo-home:ro,Z" "-v" "/var/lib/crater-agent-workspace/rustup-home:/opt/rustwide/rustup-home:ro,Z" "-e" "SOURCE_DIR=/opt/rustwide/workdir" "-e" "CARGO_TARGET_DIR=/opt/rustwide/target" "-e" "CARGO_HOME=/opt/rustwide/cargo-home" "-e" "RUSTUP_HOME=/opt/rustwide/rustup-home" "-w" "/opt/rustwide/workdir" "-m" "1610612736" "--user" "0:0" "--network" "none" "ghcr.io/rust-lang/crates-build-env/linux@sha256:7ad1b28ee6f5f7f699f6cf7015098d6ccdd96d6f2d78dd06228f5b4c9faf309c" "/opt/rustwide/cargo-home/bin/cargo" "+cdb45c87e2cd43495379f7e867e3cc15dcee9f93" "metadata" "--no-deps" "--format-version=1", kill_on_drop: false }` [INFO] [stdout] e4aa405d6b951a405911562fd5557292dfc78119573c11eb75037f2776dfa2cc [INFO] running `Command { std: "docker" "start" "-a" "e4aa405d6b951a405911562fd5557292dfc78119573c11eb75037f2776dfa2cc", kill_on_drop: false }` [INFO] running `Command { std: "docker" "inspect" "e4aa405d6b951a405911562fd5557292dfc78119573c11eb75037f2776dfa2cc", kill_on_drop: false }` [INFO] running `Command { std: "docker" "rm" "-f" "e4aa405d6b951a405911562fd5557292dfc78119573c11eb75037f2776dfa2cc", kill_on_drop: false }` [INFO] [stdout] e4aa405d6b951a405911562fd5557292dfc78119573c11eb75037f2776dfa2cc [INFO] running `Command { std: "docker" "create" "-v" "/var/lib/crater-agent-workspace/builds/worker-4-tc1/target:/opt/rustwide/target:rw,Z" "-v" "/var/lib/crater-agent-workspace/builds/worker-4-tc1/source:/opt/rustwide/workdir:ro,Z" "-v" "/var/lib/crater-agent-workspace/cargo-home:/opt/rustwide/cargo-home:ro,Z" "-v" "/var/lib/crater-agent-workspace/rustup-home:/opt/rustwide/rustup-home:ro,Z" "-e" "SOURCE_DIR=/opt/rustwide/workdir" "-e" "CARGO_TARGET_DIR=/opt/rustwide/target" "-e" "CARGO_INCREMENTAL=0" "-e" "RUST_BACKTRACE=full" "-e" "RUSTFLAGS=--cap-lints=forbid" "-e" "RUSTDOCFLAGS=--cap-lints=forbid" "-e" "CARGO_HOME=/opt/rustwide/cargo-home" "-e" "RUSTUP_HOME=/opt/rustwide/rustup-home" "-w" "/opt/rustwide/workdir" "-m" "1610612736" "--user" "0:0" "--network" "none" "ghcr.io/rust-lang/crates-build-env/linux@sha256:7ad1b28ee6f5f7f699f6cf7015098d6ccdd96d6f2d78dd06228f5b4c9faf309c" "/opt/rustwide/cargo-home/bin/cargo" "+cdb45c87e2cd43495379f7e867e3cc15dcee9f93" "build" "--frozen" "--message-format=json", kill_on_drop: false }` [INFO] [stdout] b314648dc7049db3d6e1570f64c97c71f260a1ed2cb9270e93c0bc40b082b02c [INFO] running `Command { std: "docker" "start" "-a" "b314648dc7049db3d6e1570f64c97c71f260a1ed2cb9270e93c0bc40b082b02c", kill_on_drop: false }` [INFO] [stderr] Compiling syn v2.0.104 [INFO] [stderr] Compiling libc v0.2.174 [INFO] [stderr] Compiling cfg-if v1.0.1 [INFO] [stderr] Compiling autocfg v1.5.0 [INFO] [stderr] Compiling libm v0.2.15 [INFO] [stderr] Compiling zerocopy v0.8.26 [INFO] [stderr] Compiling getrandom v0.3.3 [INFO] [stderr] Compiling cc v1.2.30 [INFO] [stderr] Compiling crossbeam-epoch v0.9.18 [INFO] [stderr] Compiling crossbeam-deque v0.8.6 [INFO] [stderr] Compiling serde v1.0.219 [INFO] [stderr] Compiling stable_deref_trait v1.2.0 [INFO] [stderr] Compiling rayon-core v1.12.1 [INFO] [stderr] Compiling reborrow v0.5.5 [INFO] [stderr] Compiling num-traits v0.2.19 [INFO] [stderr] Compiling seq-macro v0.3.6 [INFO] [stderr] Compiling memchr v2.7.5 [INFO] [stderr] Compiling bitflags v2.9.1 [INFO] [stderr] Compiling rand_core v0.9.3 [INFO] [stderr] Compiling rayon v1.10.0 [INFO] [stderr] Compiling crc32fast v1.5.0 [INFO] [stderr] Compiling pulp v0.21.5 [INFO] [stderr] Compiling rustversion v1.0.21 [INFO] [stderr] Compiling raw-cpuid v11.5.0 [INFO] [stderr] Compiling litemap v0.8.0 [INFO] [stderr] Compiling writeable v0.6.1 [INFO] [stderr] Compiling serde_json v1.0.141 [INFO] [stderr] Compiling raw-cpuid v10.7.0 [INFO] [stderr] Compiling icu_properties_data v2.0.1 [INFO] [stderr] Compiling icu_normalizer_data v2.0.0 [INFO] [stderr] Compiling adler2 v2.0.1 [INFO] [stderr] Compiling ppv-lite86 v0.2.21 [INFO] [stderr] Compiling ring v0.17.14 [INFO] [stderr] Compiling miniz_oxide v0.8.9 [INFO] [stderr] Compiling log v0.4.27 [INFO] [stderr] Compiling rand_chacha v0.9.0 [INFO] [stderr] Compiling option-ext v0.2.0 [INFO] [stderr] Compiling hashbrown v0.15.4 [INFO] [stderr] Compiling smallvec v1.15.1 [INFO] [stderr] Compiling flate2 v1.1.2 [INFO] [stderr] Compiling rand v0.9.2 [INFO] [stderr] Compiling dirs-sys v0.5.0 [INFO] [stderr] Compiling rustls-pki-types v1.12.0 [INFO] [stderr] Compiling indexmap v2.10.0 [INFO] [stderr] Compiling num-integer v0.1.46 [INFO] [stderr] Compiling libloading v0.8.8 [INFO] [stderr] Compiling toml_datetime v0.6.11 [INFO] [stderr] Compiling winnow v0.7.12 [INFO] [stderr] Compiling dirs v6.0.0 [INFO] [stderr] Compiling rand_distr v0.5.1 [INFO] [stderr] Compiling getrandom v0.2.16 [INFO] [stderr] Compiling unicode-width v0.2.1 [INFO] [stderr] Compiling portable-atomic v1.11.1 [INFO] [stderr] Compiling num-bigint v0.4.6 [INFO] [stderr] Compiling pathfinder_simd v0.5.5 [INFO] [stderr] Compiling freetype-sys v0.20.1 [INFO] [stderr] Compiling yeslogic-fontconfig-sys v6.0.0 [INFO] [stderr] Compiling aho-corasick v1.1.3 [INFO] [stderr] Compiling rustls v0.23.29 [INFO] [stderr] Compiling percent-encoding v2.3.1 [INFO] [stderr] Compiling anstyle-parse v0.2.7 [INFO] [stderr] Compiling form_urlencoded v1.2.1 [INFO] [stderr] Compiling num-iter v0.1.45 [INFO] [stderr] Compiling webpki-roots v1.0.2 [INFO] [stderr] Compiling castaway v0.2.4 [INFO] [stderr] Compiling fdeflate v0.3.7 [INFO] [stderr] Compiling onig_sys v69.9.1 [INFO] [stderr] Compiling synstructure v0.13.2 [INFO] [stderr] Compiling darling_core v0.20.11 [INFO] [stderr] Compiling num-rational v0.4.2 [INFO] [stderr] Compiling indexmap v1.9.3 [INFO] [stderr] Compiling regex-automata v0.4.9 [INFO] [stderr] Compiling tracing-core v0.1.34 [INFO] [stderr] Compiling toml_edit v0.22.27 [INFO] [stderr] Compiling colorchoice v1.0.4 [INFO] [stderr] Compiling anstyle-query v1.1.3 [INFO] [stderr] Compiling anstyle v1.0.11 [INFO] [stderr] Compiling virtue v0.0.18 [INFO] [stderr] Compiling color_quant v1.1.0 [INFO] [stderr] Compiling zip v1.1.4 [INFO] [stderr] Compiling subtle v2.6.1 [INFO] [stderr] Compiling bytemuck_derive v1.10.0 [INFO] [stderr] Compiling zerofrom-derive v0.1.6 [INFO] [stderr] Compiling serde_derive v1.0.219 [INFO] [stderr] Compiling displaydoc v0.2.5 [INFO] [stderr] Compiling yoke-derive v0.8.0 [INFO] [stderr] Compiling bytemuck v1.23.1 [INFO] [stderr] Compiling zerovec-derive v0.11.1 [INFO] [stderr] Compiling zerofrom v0.1.6 [INFO] [stderr] Compiling num-complex v0.4.6 [INFO] [stderr] Compiling half v2.6.0 [INFO] [stderr] Compiling dyn-stack v0.13.0 [INFO] [stderr] Compiling dyn-stack v0.10.0 [INFO] [stderr] Compiling thiserror-impl v1.0.69 [INFO] [stderr] Compiling yoke v0.8.0 [INFO] [stderr] Compiling pulp v0.18.22 [INFO] [stderr] Compiling zerotrie v0.2.2 [INFO] [stderr] Compiling proc-macro-crate v3.3.0 [INFO] [stderr] Compiling zerovec v0.11.2 [INFO] [stderr] Compiling darling_macro v0.20.11 [INFO] [stderr] Compiling darling v0.20.11 [INFO] [stderr] Compiling rustls-webpki v0.103.4 [INFO] [stderr] Compiling num_enum_derive v0.7.4 [INFO] [stderr] Compiling thiserror v1.0.69 [INFO] [stderr] Compiling tracing-attributes v0.1.30 [INFO] [stderr] Compiling tinystr v0.8.1 [INFO] [stderr] Compiling potential_utf v0.1.2 [INFO] [stderr] Compiling icu_locale_core v2.0.0 [INFO] [stderr] Compiling gemm-common v0.17.1 [INFO] [stderr] Compiling icu_collections v2.0.0 [INFO] [stderr] Compiling yoke-derive v0.7.5 [INFO] [stderr] Compiling gemm-f32 v0.17.1 [INFO] [stderr] Compiling gemm-f64 v0.17.1 [INFO] [stderr] Compiling gemm-c32 v0.17.1 [INFO] [stderr] Compiling tracing v0.1.41 [INFO] [stderr] Compiling icu_provider v2.0.0 [INFO] [stderr] Compiling gemm-common v0.18.2 [INFO] [stderr] Compiling yoke v0.7.5 [INFO] [stderr] Compiling icu_properties v2.0.1 [INFO] [stderr] Compiling icu_normalizer v2.0.0 [INFO] [stderr] Compiling gemm-f16 v0.17.1 [INFO] [stderr] Compiling gemm-f32 v0.18.2 [INFO] [stderr] Compiling gemm-c64 v0.18.2 [INFO] [stderr] Compiling idna_adapter v1.2.1 [INFO] [stderr] Compiling idna v1.0.3 [INFO] [stderr] Compiling gemm-f16 v0.18.2 [INFO] [stderr] Compiling gemm-c32 v0.18.2 [INFO] [stderr] Compiling gemm-f64 v0.18.2 [INFO] [stderr] Compiling url v2.5.4 [INFO] [stderr] Compiling gemm-c64 v0.17.1 [INFO] [stderr] Compiling gemm v0.18.2 [INFO] [stderr] Compiling num_enum v0.7.4 [INFO] [stderr] Compiling safetensors v0.4.5 [INFO] [stderr] Compiling derive_builder_core v0.20.2 [INFO] [stderr] Compiling num v0.4.3 [INFO] [stderr] Compiling regex v1.11.1 [INFO] [stderr] Compiling bincode_derive v2.0.1 [INFO] [stderr] Compiling thiserror-impl v2.0.12 [INFO] [stderr] Compiling anstream v0.6.19 [INFO] [stderr] Compiling png v0.17.16 [INFO] [stderr] Compiling webpki-roots v0.26.11 [INFO] [stderr] Compiling console v0.15.11 [INFO] [stderr] Compiling dlib v0.5.2 [INFO] [stderr] Compiling socks v0.3.4 [INFO] [stderr] Compiling esaxx-rs v0.1.10 [INFO] [stderr] Compiling memmap2 v0.9.7 [INFO] [stderr] Compiling num_cpus v1.17.0 [INFO] [stderr] Compiling ahash v0.8.12 [INFO] [stderr] Compiling clap_lex v0.7.5 [INFO] [stderr] Compiling number_prefix v0.4.0 [INFO] [stderr] Compiling convert_case v0.4.0 [INFO] [stderr] Compiling unty v0.0.4 [INFO] [stderr] Compiling font-kit v0.14.3 [INFO] [stderr] Compiling plotters-backend v0.3.7 [INFO] [stderr] Compiling weezl v0.1.10 [INFO] [stderr] Compiling jpeg-decoder v0.3.2 [INFO] [stderr] Compiling derive_more v0.99.20 [INFO] [stderr] Compiling nom v7.1.3 [INFO] [stderr] Compiling gif v0.12.0 [INFO] [stderr] Compiling clap_derive v4.5.41 [INFO] [stderr] Compiling yaml-rust v0.4.5 [INFO] [stderr] Compiling image v0.24.9 [INFO] [stderr] Compiling ureq v2.12.1 [INFO] [stderr] Compiling bincode v2.0.1 [INFO] [stderr] Compiling chrono v0.4.41 [INFO] [stderr] Compiling indicatif v0.17.11 [INFO] [stderr] Compiling clap_builder v4.5.41 [INFO] [stderr] Compiling ug v0.4.0 [INFO] [stderr] Compiling pathfinder_geometry v0.5.1 [INFO] [stderr] Compiling thiserror v2.0.12 [INFO] [stderr] Compiling derive_builder_macro v0.20.2 [INFO] [stderr] Compiling syn v1.0.109 [INFO] [stderr] Compiling gemm v0.17.1 [INFO] [stderr] Compiling monostate-impl v0.1.14 [INFO] [stderr] Compiling itertools v0.14.0 [INFO] [stderr] Compiling macro_rules_attribute-proc_macro v0.2.2 [INFO] [stderr] Compiling float-ord v0.3.2 [INFO] [stderr] Compiling spm_precompiled v0.1.4 [INFO] [stderr] Compiling candle-core v0.9.1 [INFO] [stderr] Compiling clap v4.5.41 [INFO] [stderr] Compiling macro_rules_attribute v0.2.2 [INFO] [stderr] Compiling monostate v0.1.14 [INFO] [stderr] Compiling hf-hub v0.4.3 [INFO] [stderr] Compiling derive_builder v0.20.2 [INFO] [stderr] Compiling serde_yaml v0.8.26 [INFO] [stderr] Compiling plotters-bitmap v0.3.7 [INFO] [stderr] Compiling rayon-cond v0.4.0 [INFO] [stderr] Compiling nichi v0.5.1 [INFO] [stderr] Compiling onig v6.5.1 [INFO] [stderr] Compiling plotters-svg v0.3.7 [INFO] [stderr] Compiling dary_heap v0.3.7 [INFO] [stderr] Compiling compact_str v0.7.1 [INFO] [stderr] Compiling compact_str v0.9.0 [INFO] [stderr] Compiling envy v0.4.2 [INFO] [stderr] Compiling console v0.16.0 [INFO] [stderr] Compiling shellexpand v3.1.1 [INFO] [stderr] Compiling unicode-normalization-alignments v0.1.12 [INFO] [stderr] Compiling ttf-parser v0.20.0 [INFO] [stderr] Compiling unit-prefix v0.5.1 [INFO] [stderr] Compiling readable v0.16.0 [INFO] [stderr] Compiling indicatif v0.18.0 [INFO] [stderr] Compiling colored v3.0.0 [INFO] [stderr] Compiling tokenizers v0.21.2 [INFO] [stderr] Compiling config-derive v0.15.0 [INFO] [stderr] Compiling twelf v0.15.0 [INFO] [stderr] Compiling plotters v0.3.7 [INFO] [stderr] Compiling candle-nn v0.9.1 [INFO] [stderr] Compiling mgpt v0.1.0 (/opt/rustwide/workdir) [INFO] [stderr] Finished `dev` profile [unoptimized + debuginfo] target(s) in 4m 08s [INFO] running `Command { std: "docker" "inspect" "b314648dc7049db3d6e1570f64c97c71f260a1ed2cb9270e93c0bc40b082b02c", kill_on_drop: false }` [INFO] running `Command { std: "docker" "rm" "-f" "b314648dc7049db3d6e1570f64c97c71f260a1ed2cb9270e93c0bc40b082b02c", kill_on_drop: false }` [INFO] [stdout] b314648dc7049db3d6e1570f64c97c71f260a1ed2cb9270e93c0bc40b082b02c [INFO] running `Command { std: "docker" "create" "-v" "/var/lib/crater-agent-workspace/builds/worker-4-tc1/target:/opt/rustwide/target:rw,Z" "-v" "/var/lib/crater-agent-workspace/builds/worker-4-tc1/source:/opt/rustwide/workdir:ro,Z" "-v" "/var/lib/crater-agent-workspace/cargo-home:/opt/rustwide/cargo-home:ro,Z" "-v" "/var/lib/crater-agent-workspace/rustup-home:/opt/rustwide/rustup-home:ro,Z" "-e" "SOURCE_DIR=/opt/rustwide/workdir" "-e" "CARGO_TARGET_DIR=/opt/rustwide/target" "-e" "CARGO_INCREMENTAL=0" "-e" "RUST_BACKTRACE=full" "-e" "RUSTFLAGS=--cap-lints=forbid" "-e" "RUSTDOCFLAGS=--cap-lints=forbid" "-e" "CARGO_HOME=/opt/rustwide/cargo-home" "-e" "RUSTUP_HOME=/opt/rustwide/rustup-home" "-w" "/opt/rustwide/workdir" "-m" "1610612736" "--user" "0:0" "--network" "none" "ghcr.io/rust-lang/crates-build-env/linux@sha256:7ad1b28ee6f5f7f699f6cf7015098d6ccdd96d6f2d78dd06228f5b4c9faf309c" "/opt/rustwide/cargo-home/bin/cargo" "+cdb45c87e2cd43495379f7e867e3cc15dcee9f93" "test" "--frozen" "--no-run" "--message-format=json", kill_on_drop: false }` [INFO] [stdout] 3704e474d8125fcf69ad47f75914f84030a28e6c330b0a38c9d86ac230c5da2c [INFO] running `Command { std: "docker" "start" "-a" "3704e474d8125fcf69ad47f75914f84030a28e6c330b0a38c9d86ac230c5da2c", kill_on_drop: false }` [INFO] [stderr] Compiling mgpt v0.1.0 (/opt/rustwide/workdir) [INFO] [stderr] Finished `test` profile [unoptimized + debuginfo] target(s) in 8.48s [INFO] running `Command { std: "docker" "inspect" "3704e474d8125fcf69ad47f75914f84030a28e6c330b0a38c9d86ac230c5da2c", kill_on_drop: false }` [INFO] running `Command { std: "docker" "rm" "-f" "3704e474d8125fcf69ad47f75914f84030a28e6c330b0a38c9d86ac230c5da2c", kill_on_drop: false }` [INFO] [stdout] 3704e474d8125fcf69ad47f75914f84030a28e6c330b0a38c9d86ac230c5da2c [INFO] running `Command { std: "docker" "create" "-v" "/var/lib/crater-agent-workspace/builds/worker-4-tc1/target:/opt/rustwide/target:rw,Z" "-v" "/var/lib/crater-agent-workspace/builds/worker-4-tc1/source:/opt/rustwide/workdir:ro,Z" "-v" "/var/lib/crater-agent-workspace/cargo-home:/opt/rustwide/cargo-home:ro,Z" "-v" "/var/lib/crater-agent-workspace/rustup-home:/opt/rustwide/rustup-home:ro,Z" "-e" "SOURCE_DIR=/opt/rustwide/workdir" "-e" "CARGO_TARGET_DIR=/opt/rustwide/target" "-e" "CARGO_INCREMENTAL=0" "-e" "RUST_BACKTRACE=full" "-e" "RUSTFLAGS=--cap-lints=forbid" "-e" "RUSTDOCFLAGS=--cap-lints=forbid" "-e" "CARGO_HOME=/opt/rustwide/cargo-home" "-e" "RUSTUP_HOME=/opt/rustwide/rustup-home" "-w" "/opt/rustwide/workdir" "-m" "1610612736" "--user" "0:0" "--network" "none" "ghcr.io/rust-lang/crates-build-env/linux@sha256:7ad1b28ee6f5f7f699f6cf7015098d6ccdd96d6f2d78dd06228f5b4c9faf309c" "/opt/rustwide/cargo-home/bin/cargo" "+cdb45c87e2cd43495379f7e867e3cc15dcee9f93" "test" "--frozen", kill_on_drop: false }` [INFO] [stdout] 6c04e3b829b04a87c90c0fa69bc852d1706fe273aaf9907e201c4cddd747d8e5 [INFO] running `Command { std: "docker" "start" "-a" "6c04e3b829b04a87c90c0fa69bc852d1706fe273aaf9907e201c4cddd747d8e5", kill_on_drop: false }` [INFO] [stderr] Finished `test` profile [unoptimized + debuginfo] target(s) in 0.35s [INFO] [stderr] Running unittests src/lib.rs (/opt/rustwide/target/debug/deps/mgpt-38dd3fe9065cd5dc) [INFO] [stdout] [INFO] [stdout] running 38 tests [INFO] [stdout] test datasets::pretrain_dataset_loader::tests::test_decode_line_valid_and_invalid_json ... ok [INFO] [stdout] test config::tests::test_load_config ... ok [INFO] [stdout] test llm::feedforward::tests::test_mlp_forward ... ok [INFO] [stdout] test datasets::pretrain_dataset_loader::tests::test_iter_overall_loader ... FAILED [INFO] [stdout] test datasets::pretrain_dataset_loader::tests::test_make_result_valid_input ... FAILED [INFO] [stdout] test datasets::pretrain_dataset_loader::tests::test_process_line_padding_and_truncation ... FAILED [INFO] [stdout] test llm::inference::tests::test_batch_inputs_with_padding ... FAILED [INFO] [stdout] test datasets::pretrain_dataset_loader::tests::test_make_result_empty_input ... FAILED [INFO] [stdout] test llm::inference::tests::test_inference ... FAILED [INFO] [stdout] test datasets::pretrain_dataset_loader::tests::test_load_tokenizer ... FAILED [INFO] [stdout] test datasets::pretrain_dataset_loader::tests::test_prtrain_dataset_loader ... FAILED [INFO] [stdout] test llm::mgpt::tests::test_llm_generate_for_batch_inputs ... FAILED [INFO] [stdout] test llm::inference::tests::test_model_params ... FAILED [INFO] [stdout] test llm::mgpt::tests::test_llm_generate_with_tokenizer ... FAILED [INFO] [stdout] test llm::nn_utils::tests::test_masked_cross_entropy_basic ... ok [INFO] [stdout] test llm::multi_head_attention::tests::test_tril ... ok [INFO] [stdout] test llm::nn_utils::tests::test_multinomial_sample_batch_2_vocab_4_n3 ... ok [INFO] [stdout] test llm::nn_utils::tests::test_multinomial_sample_invalid_input ... ok [INFO] [stdout] test llm::multi_head_attention::tests::test_multihead_attn_forward ... ok [INFO] [stdout] test llm::nn_utils::tests::test_draw_loss_series ... FAILED [INFO] [stdout] test llm::positional_encoding::tests::test_sinusoidial_position_encoding ... ok [INFO] [stdout] test llm::nn_utils::tests::test_multinomial_sample_zero_prob ... ok [INFO] [stdout] test llm::pretrain::tests::test_pretrain_llm ... FAILED [INFO] [stdout] test llm::scheduler::tests::test_lr_scheduler_cosine ... ok [INFO] [stdout] test processor::logits_processor_list::tests::test_logits_processor_list ... ok [INFO] [stdout] test processor::repetition_penalty_processor::tests::test_repetition_penalty_processor ... ok [INFO] [stdout] test processor::temperature_processor::tests::test_temperature_processor ... ok [INFO] [stdout] test processor::topk_processor::tests::test_top_k_processor ... ok [INFO] [stdout] test processor::topp_processor::tests::test_top_p_processor ... ok [INFO] [stdout] test processor::utils::tests::test_normalized_random_vec ... ok [INFO] [stdout] test llm::mgpt::tests::test_llm_forward ... ok [INFO] [stdout] test llm::transformer_block::tests::test_transformer_block ... ok [INFO] [stdout] test llm::mgpt::tests::test_llm_generate ... ok [INFO] [stdout] test llm::mgpt::tests::test_llm_generate_with_generation_config ... ok [INFO] [stdout] test llm::mgpt::tests::test_llm_generate_with_1batch ... ok [INFO] [stdout] test tokenizer::bert_tokenizer::tests::test_custom_tokenizer ... FAILED [INFO] [stdout] test llm::scheduler::tests::test_plot_schedule ... ok [INFO] [stdout] test llm::pretrain::tests::test_pretrain_llm_with_manual_data ... ok [INFO] [stdout] [INFO] [stdout] failures: [INFO] [stdout] [INFO] [stdout] ---- datasets::pretrain_dataset_loader::tests::test_iter_overall_loader stdout ---- [INFO] [stdout] start to load tokenizer from: ./out/tokenizer/tokenizer.json [INFO] [stdout] Error: No such file or directory (os error 2) [INFO] [stdout] [INFO] [stdout] Stack backtrace: [INFO] [stdout] 0: anyhow::error:: for anyhow::Error>::from [INFO] [stdout] at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/anyhow-1.0.98/src/backtrace.rs:27:14 [INFO] [stdout] 1: as core::ops::try_trait::FromResidual>>::from_residual [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/core/src/result.rs:2174:27 [INFO] [stdout] 2: mgpt::datasets::pretrain_dataset_loader::tests::test_iter_overall_loader [INFO] [stdout] at ./src/datasets/pretrain_dataset_loader.rs:367:25 [INFO] [stdout] 3: mgpt::datasets::pretrain_dataset_loader::tests::test_iter_overall_loader::{{closure}} [INFO] [stdout] at ./src/datasets/pretrain_dataset_loader.rs:366:38 [INFO] [stdout] 4: core::ops::function::FnOnce::call_once [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/core/src/ops/function.rs:253:5 [INFO] [stdout] 5: core::ops::function::FnOnce::call_once [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/core/src/ops/function.rs:253:5 [INFO] [stdout] 6: test::__rust_begin_short_backtrace [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/test/src/lib.rs:663:18 [INFO] [stdout] 7: test::run_test_in_process::{{closure}} [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/test/src/lib.rs:686:74 [INFO] [stdout] 8: as core::ops::function::FnOnce<()>>::call_once [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/core/src/panic/unwind_safe.rs:272:9 [INFO] [stdout] 9: std::panicking::catch_unwind::do_call [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/panicking.rs:590:40 [INFO] [stdout] 10: std::panicking::catch_unwind [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/panicking.rs:553:19 [INFO] [stdout] 11: std::panic::catch_unwind [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/panic.rs:359:14 [INFO] [stdout] 12: test::run_test_in_process [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/test/src/lib.rs:686:27 [INFO] [stdout] 13: test::run_test::{{closure}} [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/test/src/lib.rs:607:43 [INFO] [stdout] 14: test::run_test::{{closure}} [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/test/src/lib.rs:637:41 [INFO] [stdout] 15: std::sys::backtrace::__rust_begin_short_backtrace [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/sys/backtrace.rs:158:18 [INFO] [stdout] 16: std::thread::Builder::spawn_unchecked_::{{closure}}::{{closure}} [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/thread/mod.rs:559:17 [INFO] [stdout] 17: as core::ops::function::FnOnce<()>>::call_once [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/core/src/panic/unwind_safe.rs:272:9 [INFO] [stdout] 18: std::panicking::catch_unwind::do_call [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/panicking.rs:590:40 [INFO] [stdout] 19: std::panicking::catch_unwind [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/panicking.rs:553:19 [INFO] [stdout] 20: std::panic::catch_unwind [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/panic.rs:359:14 [INFO] [stdout] 21: std::thread::Builder::spawn_unchecked_::{{closure}} [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/thread/mod.rs:557:30 [INFO] [stdout] 22: core::ops::function::FnOnce::call_once{{vtable.shim}} [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/core/src/ops/function.rs:253:5 [INFO] [stdout] 23: as core::ops::function::FnOnce>::call_once [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/alloc/src/boxed.rs:1985:9 [INFO] [stdout] 24: std::sys::pal::unix::thread::Thread::new::thread_start [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/sys/pal/unix/thread.rs:118:17 [INFO] [stdout] 25: [INFO] [stdout] 26: clone [INFO] [stdout] [INFO] [stdout] ---- datasets::pretrain_dataset_loader::tests::test_make_result_valid_input stdout ---- [INFO] [stdout] start to load tokenizer from: ./out/tokenizer/tokenizer.json [INFO] [stdout] [INFO] [stdout] thread 'datasets::pretrain_dataset_loader::tests::test_make_result_valid_input' (22) panicked at src/datasets/pretrain_dataset_loader.rs:343:74: [INFO] [stdout] called `Result::unwrap()` on an `Err` value: TokenizersError(Os { code: 2, kind: NotFound, message: "No such file or directory" }) [INFO] [stdout] stack backtrace: [INFO] [stdout] 0: 0x5b3fffe0a0a2 - std::backtrace_rs::backtrace::libunwind::trace::h7cddb8376417e7cc [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/../../backtrace/src/backtrace/libunwind.rs:117:9 [INFO] [stdout] 1: 0x5b3fffe0a0a2 - std::backtrace_rs::backtrace::trace_unsynchronized::hb4e41acf4b349ff1 [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/../../backtrace/src/backtrace/mod.rs:66:14 [INFO] [stdout] 2: 0x5b3fffe0a0a2 - std::sys::backtrace::_print_fmt::h1222b80910ba6eb5 [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/sys/backtrace.rs:66:9 [INFO] [stdout] 3: 0x5b3fffe0a0a2 - ::fmt::h5bb8a979ba5db788 [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/sys/backtrace.rs:39:26 [INFO] [stdout] 4: 0x5b3fffe1deaf - core::fmt::rt::Argument::fmt::h2ee2c138a50a7796 [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/core/src/fmt/rt.rs:173:76 [INFO] [stdout] 5: 0x5b3fffe1deaf - core::fmt::write::h1e0dbf07fe3990bd [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/core/src/fmt/mod.rs:1468:25 [INFO] [stdout] 6: 0x5b3fffdd4a43 - std::io::default_write_fmt::h7d7ad5ed6a883d81 [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/io/mod.rs:639:11 [INFO] [stdout] 7: 0x5b3fffdd4a43 - std::io::Write::write_fmt::hc477d9325b345ece [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/io/mod.rs:1954:13 [INFO] [stdout] 8: 0x5b3fffde0ef2 - std::sys::backtrace::BacktraceLock::print::h788d486777205086 [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/sys/backtrace.rs:42:9 [INFO] [stdout] 9: 0x5b3fffde67cf - std::panicking::default_hook::{{closure}}::ha7bdfeb5949fc0fa [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/panicking.rs:301:27 [INFO] [stdout] 10: 0x5b3fffde6661 - std::panicking::default_hook::h6dad75ec721846f4 [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/panicking.rs:325:9 [INFO] [stdout] 11: 0x5b3ffe6a6ece - as core::ops::function::Fn>::call::h33799d04b0b96146 [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/alloc/src/boxed.rs:1999:9 [INFO] [stdout] 12: 0x5b3ffe6a6ece - test::test_main_with_exit_callback::{{closure}}::h880e31c829a2eb5d [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/test/src/lib.rs:145:21 [INFO] [stdout] 13: 0x5b3fffde6f1e - as core::ops::function::Fn>::call::h5e1b8c652ea49180 [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/alloc/src/boxed.rs:1999:9 [INFO] [stdout] 14: 0x5b3fffde6f1e - std::panicking::panic_with_hook::hba00e869ada17676 [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/panicking.rs:842:13 [INFO] [stdout] 15: 0x5b3fffde6c3a - std::panicking::panic_handler::{{closure}}::h3d21149c258e5ceb [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/panicking.rs:707:13 [INFO] [stdout] 16: 0x5b3fffde1029 - std::sys::backtrace::__rust_end_short_backtrace::h4f5d9b9dfb3e6ec1 [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/sys/backtrace.rs:174:18 [INFO] [stdout] 17: 0x5b3fffdc7b8d - __rustc[9a7a9f9af7564de1]::rust_begin_unwind [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/panicking.rs:698:5 [INFO] [stdout] 18: 0x5b3fffe28690 - core::panicking::panic_fmt::h78e817a90331d98b [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/core/src/panicking.rs:75:14 [INFO] [stdout] 19: 0x5b3fffe276c6 - core::result::unwrap_failed::h3ce108722ad88f1e [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/core/src/result.rs:1852:5 [INFO] [stdout] 20: 0x5b3ffe56c85a - core::result::Result::unwrap::ha2f4e594bcd0c114 [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/core/src/result.rs:1223:23 [INFO] [stdout] 21: 0x5b3ffe56c85a - mgpt::datasets::pretrain_dataset_loader::tests::test_make_result_valid_input::h41ceffcef4a7320c [INFO] [stdout] at /opt/rustwide/workdir/src/datasets/pretrain_dataset_loader.rs:343:74 [INFO] [stdout] 22: 0x5b3ffe56d3d7 - mgpt::datasets::pretrain_dataset_loader::tests::test_make_result_valid_input::{{closure}}::h16cb1a331bb74376 [INFO] [stdout] at /opt/rustwide/workdir/src/datasets/pretrain_dataset_loader.rs:342:38 [INFO] [stdout] 23: 0x5b3ffe60fa36 - core::ops::function::FnOnce::call_once::hc0e1cc676da7741a [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/core/src/ops/function.rs:253:5 [INFO] [stdout] 24: 0x5b3ffe6a6ccb - core::ops::function::FnOnce::call_once::h7f4b4fba903e39d5 [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/core/src/ops/function.rs:253:5 [INFO] [stdout] 25: 0x5b3ffe6a6ccb - test::__rust_begin_short_backtrace::h9277cb6a2ccfc000 [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/test/src/lib.rs:663:18 [INFO] [stdout] 26: 0x5b3ffe6bc775 - test::run_test_in_process::{{closure}}::h9aea5ca90d1f4423 [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/test/src/lib.rs:686:74 [INFO] [stdout] 27: 0x5b3ffe6bc775 - as core::ops::function::FnOnce<()>>::call_once::h364f5fe6cc8afa85 [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/core/src/panic/unwind_safe.rs:272:9 [INFO] [stdout] 28: 0x5b3ffe6bc775 - std::panicking::catch_unwind::do_call::h2b2bec3317fe53ec [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/panicking.rs:590:40 [INFO] [stdout] 29: 0x5b3ffe6bc775 - std::panicking::catch_unwind::hc3763734156da4af [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/panicking.rs:553:19 [INFO] [stdout] 30: 0x5b3ffe6bc775 - std::panic::catch_unwind::h28038391e867eabc [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/panic.rs:359:14 [INFO] [stdout] 31: 0x5b3ffe6bc775 - test::run_test_in_process::had0273166695a036 [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/test/src/lib.rs:686:27 [INFO] [stdout] 32: 0x5b3ffe6bc775 - test::run_test::{{closure}}::h743d09d4bb476605 [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/test/src/lib.rs:607:43 [INFO] [stdout] 33: 0x5b3ffe693194 - test::run_test::{{closure}}::he950b8f9118d37e2 [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/test/src/lib.rs:637:41 [INFO] [stdout] 34: 0x5b3ffe693194 - std::sys::backtrace::__rust_begin_short_backtrace::hbfaffa6539f6abb7 [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/sys/backtrace.rs:158:18 [INFO] [stdout] 35: 0x5b3ffe696a7a - std::thread::Builder::spawn_unchecked_::{{closure}}::{{closure}}::hcce3c2c65b9c3b20 [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/thread/mod.rs:559:17 [INFO] [stdout] 36: 0x5b3ffe696a7a - as core::ops::function::FnOnce<()>>::call_once::hb66b30b6d37985a5 [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/core/src/panic/unwind_safe.rs:272:9 [INFO] [stdout] 37: 0x5b3ffe696a7a - std::panicking::catch_unwind::do_call::h992bbe2c32dc1d79 [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/panicking.rs:590:40 [INFO] [stdout] 38: 0x5b3ffe696a7a - std::panicking::catch_unwind::h412991d5237de610 [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/panicking.rs:553:19 [INFO] [stdout] 39: 0x5b3ffe696a7a - std::panic::catch_unwind::ha82b139b3eb5840a [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/panic.rs:359:14 [INFO] [stdout] 40: 0x5b3ffe696a7a - std::thread::Builder::spawn_unchecked_::{{closure}}::ha823b36f5114938e [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/thread/mod.rs:557:30 [INFO] [stdout] 41: 0x5b3ffe696a7a - core::ops::function::FnOnce::call_once{{vtable.shim}}::hf120e7b1db22ac07 [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/core/src/ops/function.rs:253:5 [INFO] [stdout] 42: 0x5b3fffddafaf - as core::ops::function::FnOnce>::call_once::h3e049222c99298ac [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/alloc/src/boxed.rs:1985:9 [INFO] [stdout] 43: 0x5b3fffddafaf - std::sys::pal::unix::thread::Thread::new::thread_start::h942e336943ad5963 [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/sys/pal/unix/thread.rs:118:17 [INFO] [stdout] 44: 0x76862176faa4 - [INFO] [stdout] 45: 0x7686217fca34 - clone [INFO] [stdout] 46: 0x0 - [INFO] [stdout] [INFO] [stdout] ---- datasets::pretrain_dataset_loader::tests::test_process_line_padding_and_truncation stdout ---- [INFO] [stdout] start to load tokenizer from: ./out/tokenizer/tokenizer.json [INFO] [stdout] Error: No such file or directory (os error 2) [INFO] [stdout] [INFO] [stdout] Stack backtrace: [INFO] [stdout] 0: anyhow::error:: for anyhow::Error>::from [INFO] [stdout] at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/anyhow-1.0.98/src/backtrace.rs:27:14 [INFO] [stdout] 1: as core::ops::try_trait::FromResidual>>::from_residual [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/core/src/result.rs:2174:27 [INFO] [stdout] 2: mgpt::datasets::pretrain_dataset_loader::tests::test_process_line_padding_and_truncation [INFO] [stdout] at ./src/datasets/pretrain_dataset_loader.rs:270:25 [INFO] [stdout] 3: mgpt::datasets::pretrain_dataset_loader::tests::test_process_line_padding_and_truncation::{{closure}} [INFO] [stdout] at ./src/datasets/pretrain_dataset_loader.rs:269:54 [INFO] [stdout] 4: core::ops::function::FnOnce::call_once [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/core/src/ops/function.rs:253:5 [INFO] [stdout] 5: core::ops::function::FnOnce::call_once [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/core/src/ops/function.rs:253:5 [INFO] [stdout] 6: test::__rust_begin_short_backtrace [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/test/src/lib.rs:663:18 [INFO] [stdout] 7: test::run_test_in_process::{{closure}} [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/test/src/lib.rs:686:74 [INFO] [stdout] 8: as core::ops::function::FnOnce<()>>::call_once [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/core/src/panic/unwind_safe.rs:272:9 [INFO] [stdout] 9: std::panicking::catch_unwind::do_call [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/panicking.rs:590:40 [INFO] [stdout] 10: std::panicking::catch_unwind [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/panicking.rs:553:19 [INFO] [stdout] 11: std::panic::catch_unwind [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/panic.rs:359:14 [INFO] [stdout] 12: test::run_test_in_process [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/test/src/lib.rs:686:27 [INFO] [stdout] 13: test::run_test::{{closure}} [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/test/src/lib.rs:607:43 [INFO] [stdout] 14: test::run_test::{{closure}} [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/test/src/lib.rs:637:41 [INFO] [stdout] 15: std::sys::backtrace::__rust_begin_short_backtrace [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/sys/backtrace.rs:158:18 [INFO] [stdout] 16: std::thread::Builder::spawn_unchecked_::{{closure}}::{{closure}} [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/thread/mod.rs:559:17 [INFO] [stdout] 17: as core::ops::function::FnOnce<()>>::call_once [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/core/src/panic/unwind_safe.rs:272:9 [INFO] [stdout] 18: std::panicking::catch_unwind::do_call [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/panicking.rs:590:40 [INFO] [stdout] 19: std::panicking::catch_unwind [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/panicking.rs:553:19 [INFO] [stdout] 20: std::panic::catch_unwind [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/panic.rs:359:14 [INFO] [stdout] 21: std::thread::Builder::spawn_unchecked_::{{closure}} [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/thread/mod.rs:557:30 [INFO] [stdout] 22: core::ops::function::FnOnce::call_once{{vtable.shim}} [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/core/src/ops/function.rs:253:5 [INFO] [stdout] 23: as core::ops::function::FnOnce>::call_once [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/alloc/src/boxed.rs:1985:9 [INFO] [stdout] 24: std::sys::pal::unix::thread::Thread::new::thread_start [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/sys/pal/unix/thread.rs:118:17 [INFO] [stdout] 25: [INFO] [stdout] 26: clone [INFO] [stdout] [INFO] [stdout] ---- llm::inference::tests::test_batch_inputs_with_padding stdout ---- [INFO] [stdout] start to load tokenizer from: ./out/tokenizer/tokenizer.json [INFO] [stdout] Error: No such file or directory (os error 2) [INFO] [stdout] [INFO] [stdout] Stack backtrace: [INFO] [stdout] 0: anyhow::error:: for anyhow::Error>::from [INFO] [stdout] at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/anyhow-1.0.98/src/backtrace.rs:27:14 [INFO] [stdout] 1: as core::ops::try_trait::FromResidual>>::from_residual [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/core/src/result.rs:2174:27 [INFO] [stdout] 2: mgpt::llm::inference::tests::test_batch_inputs_with_padding [INFO] [stdout] at ./src/llm/inference.rs:142:25 [INFO] [stdout] 3: mgpt::llm::inference::tests::test_batch_inputs_with_padding::{{closure}} [INFO] [stdout] at ./src/llm/inference.rs:141:44 [INFO] [stdout] 4: core::ops::function::FnOnce::call_once [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/core/src/ops/function.rs:253:5 [INFO] [stdout] 5: core::ops::function::FnOnce::call_once [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/core/src/ops/function.rs:253:5 [INFO] [stdout] 6: test::__rust_begin_short_backtrace [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/test/src/lib.rs:663:18 [INFO] [stdout] 7: test::run_test_in_process::{{closure}} [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/test/src/lib.rs:686:74 [INFO] [stdout] 8: as core::ops::function::FnOnce<()>>::call_once [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/core/src/panic/unwind_safe.rs:272:9 [INFO] [stdout] 9: std::panicking::catch_unwind::do_call [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/panicking.rs:590:40 [INFO] [stdout] 10: std::panicking::catch_unwind [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/panicking.rs:553:19 [INFO] [stdout] 11: std::panic::catch_unwind [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/panic.rs:359:14 [INFO] [stdout] 12: test::run_test_in_process [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/test/src/lib.rs:686:27 [INFO] [stdout] 13: test::run_test::{{closure}} [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/test/src/lib.rs:607:43 [INFO] [stdout] 14: test::run_test::{{closure}} [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/test/src/lib.rs:637:41 [INFO] [stdout] 15: std::sys::backtrace::__rust_begin_short_backtrace [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/sys/backtrace.rs:158:18 [INFO] [stdout] 16: std::thread::Builder::spawn_unchecked_::{{closure}}::{{closure}} [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/thread/mod.rs:559:17 [INFO] [stdout] 17: as core::ops::function::FnOnce<()>>::call_once [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/core/src/panic/unwind_safe.rs:272:9 [INFO] [stdout] 18: std::panicking::catch_unwind::do_call [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/panicking.rs:590:40 [INFO] [stdout] 19: std::panicking::catch_unwind [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/panicking.rs:553:19 [INFO] [stdout] 20: std::panic::catch_unwind [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/panic.rs:359:14 [INFO] [stdout] 21: std::thread::Builder::spawn_unchecked_::{{closure}} [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/thread/mod.rs:557:30 [INFO] [stdout] 22: core::ops::function::FnOnce::call_once{{vtable.shim}} [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/core/src/ops/function.rs:253:5 [INFO] [stdout] 23: as core::ops::function::FnOnce>::call_once [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/alloc/src/boxed.rs:1985:9 [INFO] [stdout] 24: std::sys::pal::unix::thread::Thread::new::thread_start [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/sys/pal/unix/thread.rs:118:17 [INFO] [stdout] 25: [INFO] [stdout] 26: clone [INFO] [stdout] [INFO] [stdout] ---- datasets::pretrain_dataset_loader::tests::test_make_result_empty_input stdout ---- [INFO] [stdout] start to load tokenizer from: ./out/tokenizer/tokenizer.json [INFO] [stdout] [INFO] [stdout] thread 'datasets::pretrain_dataset_loader::tests::test_make_result_empty_input' (21) panicked at src/datasets/pretrain_dataset_loader.rs:323:74: [INFO] [stdout] called `Result::unwrap()` on an `Err` value: TokenizersError(Os { code: 2, kind: NotFound, message: "No such file or directory" }) [INFO] [stdout] stack backtrace: [INFO] [stdout] 0: 0x5b3fffe0a0a2 - std::backtrace_rs::backtrace::libunwind::trace::h7cddb8376417e7cc [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/../../backtrace/src/backtrace/libunwind.rs:117:9 [INFO] [stdout] 1: 0x5b3fffe0a0a2 - std::backtrace_rs::backtrace::trace_unsynchronized::hb4e41acf4b349ff1 [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/../../backtrace/src/backtrace/mod.rs:66:14 [INFO] [stdout] 2: 0x5b3fffe0a0a2 - std::sys::backtrace::_print_fmt::h1222b80910ba6eb5 [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/sys/backtrace.rs:66:9 [INFO] [stdout] 3: 0x5b3fffe0a0a2 - ::fmt::h5bb8a979ba5db788 [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/sys/backtrace.rs:39:26 [INFO] [stdout] 4: 0x5b3fffe1deaf - core::fmt::rt::Argument::fmt::h2ee2c138a50a7796 [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/core/src/fmt/rt.rs:173:76 [INFO] [stdout] 5: 0x5b3fffe1deaf - core::fmt::write::h1e0dbf07fe3990bd [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/core/src/fmt/mod.rs:1468:25 [INFO] [stdout] 6: 0x5b3fffdd4a43 - std::io::default_write_fmt::h7d7ad5ed6a883d81 [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/io/mod.rs:639:11 [INFO] [stdout] 7: 0x5b3fffdd4a43 - std::io::Write::write_fmt::hc477d9325b345ece [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/io/mod.rs:1954:13 [INFO] [stdout] 8: 0x5b3fffde0ef2 - std::sys::backtrace::BacktraceLock::print::h788d486777205086 [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/sys/backtrace.rs:42:9 [INFO] [stdout] 9: 0x5b3fffde67cf - std::panicking::default_hook::{{closure}}::ha7bdfeb5949fc0fa [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/panicking.rs:301:27 [INFO] [stdout] 10: 0x5b3fffde6661 - std::panicking::default_hook::h6dad75ec721846f4 [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/panicking.rs:325:9 [INFO] [stdout] 11: 0x5b3ffe6a6ece - as core::ops::function::Fn>::call::h33799d04b0b96146 [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/alloc/src/boxed.rs:1999:9 [INFO] [stdout] 12: 0x5b3ffe6a6ece - test::test_main_with_exit_callback::{{closure}}::h880e31c829a2eb5d [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/test/src/lib.rs:145:21 [INFO] [stdout] 13: 0x5b3fffde6f1e - as core::ops::function::Fn>::call::h5e1b8c652ea49180 [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/alloc/src/boxed.rs:1999:9 [INFO] [stdout] 14: 0x5b3fffde6f1e - std::panicking::panic_with_hook::hba00e869ada17676 [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/panicking.rs:842:13 [INFO] [stdout] 15: 0x5b3fffde6c3a - std::panicking::panic_handler::{{closure}}::h3d21149c258e5ceb [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/panicking.rs:707:13 [INFO] [stdout] 16: 0x5b3fffde1029 - std::sys::backtrace::__rust_end_short_backtrace::h4f5d9b9dfb3e6ec1 [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/sys/backtrace.rs:174:18 [INFO] [stdout] 17: 0x5b3fffdc7b8d - __rustc[9a7a9f9af7564de1]::rust_begin_unwind [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/panicking.rs:698:5 [INFO] [stdout] 18: 0x5b3fffe28690 - core::panicking::panic_fmt::h78e817a90331d98b [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/core/src/panicking.rs:75:14 [INFO] [stdout] 19: 0x5b3fffe276c6 - core::result::unwrap_failed::h3ce108722ad88f1e [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/core/src/result.rs:1852:5 [INFO] [stdout] 20: 0x5b3ffe56c382 - core::result::Result::unwrap::ha2f4e594bcd0c114 [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/core/src/result.rs:1223:23 [INFO] [stdout] 21: 0x5b3ffe56c382 - mgpt::datasets::pretrain_dataset_loader::tests::test_make_result_empty_input::h01951c0e00f717df [INFO] [stdout] at /opt/rustwide/workdir/src/datasets/pretrain_dataset_loader.rs:323:74 [INFO] [stdout] 22: 0x5b3ffe56c767 - mgpt::datasets::pretrain_dataset_loader::tests::test_make_result_empty_input::{{closure}}::h746994f6c959f37e [INFO] [stdout] at /opt/rustwide/workdir/src/datasets/pretrain_dataset_loader.rs:322:38 [INFO] [stdout] 23: 0x5b3ffe60fb76 - core::ops::function::FnOnce::call_once::hc5450fd3eefccf98 [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/core/src/ops/function.rs:253:5 [INFO] [stdout] 24: 0x5b3ffe6a6ccb - core::ops::function::FnOnce::call_once::h7f4b4fba903e39d5 [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/core/src/ops/function.rs:253:5 [INFO] [stdout] 25: 0x5b3ffe6a6ccb - test::__rust_begin_short_backtrace::h9277cb6a2ccfc000 [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/test/src/lib.rs:663:18 [INFO] [stdout] 26: 0x5b3ffe6bc775 - test::run_test_in_process::{{closure}}::h9aea5ca90d1f4423 [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/test/src/lib.rs:686:74 [INFO] [stdout] 27: 0x5b3ffe6bc775 - as core::ops::function::FnOnce<()>>::call_once::h364f5fe6cc8afa85 [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/core/src/panic/unwind_safe.rs:272:9 [INFO] [stdout] 28: 0x5b3ffe6bc775 - std::panicking::catch_unwind::do_call::h2b2bec3317fe53ec [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/panicking.rs:590:40 [INFO] [stdout] 29: 0x5b3ffe6bc775 - std::panicking::catch_unwind::hc3763734156da4af [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/panicking.rs:553:19 [INFO] [stdout] 30: 0x5b3ffe6bc775 - std::panic::catch_unwind::h28038391e867eabc [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/panic.rs:359:14 [INFO] [stdout] 31: 0x5b3ffe6bc775 - test::run_test_in_process::had0273166695a036 [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/test/src/lib.rs:686:27 [INFO] [stdout] 32: 0x5b3ffe6bc775 - test::run_test::{{closure}}::h743d09d4bb476605 [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/test/src/lib.rs:607:43 [INFO] [stdout] 33: 0x5b3ffe693194 - test::run_test::{{closure}}::he950b8f9118d37e2 [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/test/src/lib.rs:637:41 [INFO] [stdout] 34: 0x5b3ffe693194 - std::sys::backtrace::__rust_begin_short_backtrace::hbfaffa6539f6abb7 [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/sys/backtrace.rs:158:18 [INFO] [stdout] 35: 0x5b3ffe696a7a - std::thread::Builder::spawn_unchecked_::{{closure}}::{{closure}}::hcce3c2c65b9c3b20 [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/thread/mod.rs:559:17 [INFO] [stdout] 36: 0x5b3ffe696a7a - as core::ops::function::FnOnce<()>>::call_once::hb66b30b6d37985a5 [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/core/src/panic/unwind_safe.rs:272:9 [INFO] [stdout] 37: 0x5b3ffe696a7a - std::panicking::catch_unwind::do_call::h992bbe2c32dc1d79 [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/panicking.rs:590:40 [INFO] [stdout] 38: 0x5b3ffe696a7a - std::panicking::catch_unwind::h412991d5237de610 [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/panicking.rs:553:19 [INFO] [stdout] 39: 0x5b3ffe696a7a - std::panic::catch_unwind::ha82b139b3eb5840a [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/panic.rs:359:14 [INFO] [stdout] 40: 0x5b3ffe696a7a - std::thread::Builder::spawn_unchecked_::{{closure}}::ha823b36f5114938e [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/thread/mod.rs:557:30 [INFO] [stdout] 41: 0x5b3ffe696a7a - core::ops::function::FnOnce::call_once{{vtable.shim}}::hf120e7b1db22ac07 [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/core/src/ops/function.rs:253:5 [INFO] [stdout] 42: 0x5b3fffddafaf - as core::ops::function::FnOnce>::call_once::h3e049222c99298ac [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/alloc/src/boxed.rs:1985:9 [INFO] [stdout] 43: 0x5b3fffddafaf - std::sys::pal::unix::thread::Thread::new::thread_start::h942e336943ad5963 [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/sys/pal/unix/thread.rs:118:17 [INFO] [stdout] 44: 0x76862176faa4 - [INFO] [stdout] 45: 0x7686217fca34 - clone [INFO] [stdout] 46: 0x0 - [INFO] [stdout] [INFO] [stdout] ---- llm::inference::tests::test_inference stdout ---- [INFO] [stdout] start to load tokenizer from: ./out/tokenizer/tokenizer.json [INFO] [stdout] Error: No such file or directory (os error 2) [INFO] [stdout] [INFO] [stdout] Stack backtrace: [INFO] [stdout] 0: anyhow::error:: for anyhow::Error>::from [INFO] [stdout] at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/anyhow-1.0.98/src/backtrace.rs:27:14 [INFO] [stdout] 1: as core::ops::try_trait::FromResidual>>::from_residual [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/core/src/result.rs:2174:27 [INFO] [stdout] 2: mgpt::llm::inference::tests::test_inference [INFO] [stdout] at ./src/llm/inference.rs:105:25 [INFO] [stdout] 3: mgpt::llm::inference::tests::test_inference::{{closure}} [INFO] [stdout] at ./src/llm/inference.rs:103:28 [INFO] [stdout] 4: core::ops::function::FnOnce::call_once [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/core/src/ops/function.rs:253:5 [INFO] [stdout] 5: core::ops::function::FnOnce::call_once [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/core/src/ops/function.rs:253:5 [INFO] [stdout] 6: test::__rust_begin_short_backtrace [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/test/src/lib.rs:663:18 [INFO] [stdout] 7: test::run_test_in_process::{{closure}} [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/test/src/lib.rs:686:74 [INFO] [stdout] 8: as core::ops::function::FnOnce<()>>::call_once [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/core/src/panic/unwind_safe.rs:272:9 [INFO] [stdout] 9: std::panicking::catch_unwind::do_call [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/panicking.rs:590:40 [INFO] [stdout] 10: std::panicking::catch_unwind [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/panicking.rs:553:19 [INFO] [stdout] 11: std::panic::catch_unwind [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/panic.rs:359:14 [INFO] [stdout] 12: test::run_test_in_process [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/test/src/lib.rs:686:27 [INFO] [stdout] 13: test::run_test::{{closure}} [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/test/src/lib.rs:607:43 [INFO] [stdout] 14: test::run_test::{{closure}} [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/test/src/lib.rs:637:41 [INFO] [stdout] 15: std::sys::backtrace::__rust_begin_short_backtrace [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/sys/backtrace.rs:158:18 [INFO] [stdout] 16: std::thread::Builder::spawn_unchecked_::{{closure}}::{{closure}} [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/thread/mod.rs:559:17 [INFO] [stdout] 17: as core::ops::function::FnOnce<()>>::call_once [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/core/src/panic/unwind_safe.rs:272:9 [INFO] [stdout] 18: std::panicking::catch_unwind::do_call [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/panicking.rs:590:40 [INFO] [stdout] 19: std::panicking::catch_unwind [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/panicking.rs:553:19 [INFO] [stdout] 20: std::panic::catch_unwind [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/panic.rs:359:14 [INFO] [stdout] 21: std::thread::Builder::spawn_unchecked_::{{closure}} [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/thread/mod.rs:557:30 [INFO] [stdout] 22: core::ops::function::FnOnce::call_once{{vtable.shim}} [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/core/src/ops/function.rs:253:5 [INFO] [stdout] 23: as core::ops::function::FnOnce>::call_once [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/alloc/src/boxed.rs:1985:9 [INFO] [stdout] 24: std::sys::pal::unix::thread::Thread::new::thread_start [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/sys/pal/unix/thread.rs:118:17 [INFO] [stdout] 25: [INFO] [stdout] 26: clone [INFO] [stdout] [INFO] [stdout] ---- datasets::pretrain_dataset_loader::tests::test_load_tokenizer stdout ---- [INFO] [stdout] start to load tokenizer from: ./out/tokenizer/tokenizer.json [INFO] [stdout] Error: No such file or directory (os error 2) [INFO] [stdout] [INFO] [stdout] Stack backtrace: [INFO] [stdout] 0: anyhow::error:: for anyhow::Error>::from [INFO] [stdout] at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/anyhow-1.0.98/src/backtrace.rs:27:14 [INFO] [stdout] 1: as core::ops::try_trait::FromResidual>>::from_residual [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/core/src/result.rs:2174:27 [INFO] [stdout] 2: mgpt::datasets::pretrain_dataset_loader::tests::test_load_tokenizer [INFO] [stdout] at ./src/datasets/pretrain_dataset_loader.rs:239:25 [INFO] [stdout] 3: mgpt::datasets::pretrain_dataset_loader::tests::test_load_tokenizer::{{closure}} [INFO] [stdout] at ./src/datasets/pretrain_dataset_loader.rs:238:33 [INFO] [stdout] 4: core::ops::function::FnOnce::call_once [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/core/src/ops/function.rs:253:5 [INFO] [stdout] 5: core::ops::function::FnOnce::call_once [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/core/src/ops/function.rs:253:5 [INFO] [stdout] 6: test::__rust_begin_short_backtrace [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/test/src/lib.rs:663:18 [INFO] [stdout] 7: test::run_test_in_process::{{closure}} [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/test/src/lib.rs:686:74 [INFO] [stdout] 8: as core::ops::function::FnOnce<()>>::call_once [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/core/src/panic/unwind_safe.rs:272:9 [INFO] [stdout] 9: std::panicking::catch_unwind::do_call [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/panicking.rs:590:40 [INFO] [stdout] 10: std::panicking::catch_unwind [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/panicking.rs:553:19 [INFO] [stdout] 11: std::panic::catch_unwind [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/panic.rs:359:14 [INFO] [stdout] 12: test::run_test_in_process [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/test/src/lib.rs:686:27 [INFO] [stdout] 13: test::run_test::{{closure}} [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/test/src/lib.rs:607:43 [INFO] [stdout] 14: test::run_test::{{closure}} [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/test/src/lib.rs:637:41 [INFO] [stdout] 15: std::sys::backtrace::__rust_begin_short_backtrace [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/sys/backtrace.rs:158:18 [INFO] [stdout] 16: std::thread::Builder::spawn_unchecked_::{{closure}}::{{closure}} [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/thread/mod.rs:559:17 [INFO] [stdout] 17: as core::ops::function::FnOnce<()>>::call_once [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/core/src/panic/unwind_safe.rs:272:9 [INFO] [stdout] 18: std::panicking::catch_unwind::do_call [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/panicking.rs:590:40 [INFO] [stdout] 19: std::panicking::catch_unwind [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/panicking.rs:553:19 [INFO] [stdout] 20: std::panic::catch_unwind [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/panic.rs:359:14 [INFO] [stdout] 21: std::thread::Builder::spawn_unchecked_::{{closure}} [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/thread/mod.rs:557:30 [INFO] [stdout] 22: core::ops::function::FnOnce::call_once{{vtable.shim}} [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/core/src/ops/function.rs:253:5 [INFO] [stdout] 23: as core::ops::function::FnOnce>::call_once [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/alloc/src/boxed.rs:1985:9 [INFO] [stdout] 24: std::sys::pal::unix::thread::Thread::new::thread_start [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/sys/pal/unix/thread.rs:118:17 [INFO] [stdout] 25: [INFO] [stdout] 26: clone [INFO] [stdout] [INFO] [stdout] ---- datasets::pretrain_dataset_loader::tests::test_prtrain_dataset_loader stdout ---- [INFO] [stdout] start to load tokenizer from: ./out/tokenizer/tokenizer.json [INFO] [stdout] Error: No such file or directory (os error 2) [INFO] [stdout] [INFO] [stdout] Stack backtrace: [INFO] [stdout] 0: anyhow::error:: for anyhow::Error>::from [INFO] [stdout] at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/anyhow-1.0.98/src/backtrace.rs:27:14 [INFO] [stdout] 1: as core::ops::try_trait::FromResidual>>::from_residual [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/core/src/result.rs:2174:27 [INFO] [stdout] 2: mgpt::datasets::pretrain_dataset_loader::tests::test_prtrain_dataset_loader [INFO] [stdout] at ./src/datasets/pretrain_dataset_loader.rs:246:25 [INFO] [stdout] 3: mgpt::datasets::pretrain_dataset_loader::tests::test_prtrain_dataset_loader::{{closure}} [INFO] [stdout] at ./src/datasets/pretrain_dataset_loader.rs:245:41 [INFO] [stdout] 4: core::ops::function::FnOnce::call_once [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/core/src/ops/function.rs:253:5 [INFO] [stdout] 5: core::ops::function::FnOnce::call_once [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/core/src/ops/function.rs:253:5 [INFO] [stdout] 6: test::__rust_begin_short_backtrace [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/test/src/lib.rs:663:18 [INFO] [stdout] 7: test::run_test_in_process::{{closure}} [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/test/src/lib.rs:686:74 [INFO] [stdout] 8: as core::ops::function::FnOnce<()>>::call_once [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/core/src/panic/unwind_safe.rs:272:9 [INFO] [stdout] 9: std::panicking::catch_unwind::do_call [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/panicking.rs:590:40 [INFO] [stdout] 10: std::panicking::catch_unwind [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/panicking.rs:553:19 [INFO] [stdout] 11: std::panic::catch_unwind [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/panic.rs:359:14 [INFO] [stdout] 12: test::run_test_in_process [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/test/src/lib.rs:686:27 [INFO] [stdout] 13: test::run_test::{{closure}} [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/test/src/lib.rs:607:43 [INFO] [stdout] 14: test::run_test::{{closure}} [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/test/src/lib.rs:637:41 [INFO] [stdout] 15: std::sys::backtrace::__rust_begin_short_backtrace [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/sys/backtrace.rs:158:18 [INFO] [stdout] 16: std::thread::Builder::spawn_unchecked_::{{closure}}::{{closure}} [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/thread/mod.rs:559:17 [INFO] [stdout] 17: as core::ops::function::FnOnce<()>>::call_once [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/core/src/panic/unwind_safe.rs:272:9 [INFO] [stdout] 18: std::panicking::catch_unwind::do_call [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/panicking.rs:590:40 [INFO] [stdout] 19: std::panicking::catch_unwind [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/panicking.rs:553:19 [INFO] [stdout] 20: std::panic::catch_unwind [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/panic.rs:359:14 [INFO] [stdout] 21: std::thread::Builder::spawn_unchecked_::{{closure}} [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/thread/mod.rs:557:30 [INFO] [stdout] 22: core::ops::function::FnOnce::call_once{{vtable.shim}} [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/core/src/ops/function.rs:253:5 [INFO] [stdout] 23: as core::ops::function::FnOnce>::call_once [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/alloc/src/boxed.rs:1985:9 [INFO] [stdout] 24: std::sys::pal::unix::thread::Thread::new::thread_start [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/sys/pal/unix/thread.rs:118:17 [INFO] [stdout] 25: [INFO] [stdout] 26: clone [INFO] [stdout] [INFO] [stdout] ---- llm::mgpt::tests::test_llm_generate_for_batch_inputs stdout ---- [INFO] [stdout] start to load tokenizer from: ./out/tokenizer/tokenizer.json [INFO] [stdout] Error: No such file or directory (os error 2) [INFO] [stdout] [INFO] [stdout] Stack backtrace: [INFO] [stdout] 0: anyhow::error:: for anyhow::Error>::from [INFO] [stdout] at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/anyhow-1.0.98/src/backtrace.rs:27:14 [INFO] [stdout] 1: as core::ops::try_trait::FromResidual>>::from_residual [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/core/src/result.rs:2174:27 [INFO] [stdout] 2: mgpt::llm::mgpt::tests::test_llm_generate_for_batch_inputs [INFO] [stdout] at ./src/llm/mgpt.rs:464:25 [INFO] [stdout] 3: mgpt::llm::mgpt::tests::test_llm_generate_for_batch_inputs::{{closure}} [INFO] [stdout] at ./src/llm/mgpt.rs:457:48 [INFO] [stdout] 4: core::ops::function::FnOnce::call_once [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/core/src/ops/function.rs:253:5 [INFO] [stdout] 5: core::ops::function::FnOnce::call_once [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/core/src/ops/function.rs:253:5 [INFO] [stdout] 6: test::__rust_begin_short_backtrace [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/test/src/lib.rs:663:18 [INFO] [stdout] 7: test::run_test_in_process::{{closure}} [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/test/src/lib.rs:686:74 [INFO] [stdout] 8: as core::ops::function::FnOnce<()>>::call_once [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/core/src/panic/unwind_safe.rs:272:9 [INFO] [stdout] 9: std::panicking::catch_unwind::do_call [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/panicking.rs:590:40 [INFO] [stdout] 10: std::panicking::catch_unwind [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/panicking.rs:553:19 [INFO] [stdout] 11: std::panic::catch_unwind [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/panic.rs:359:14 [INFO] [stdout] 12: test::run_test_in_process [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/test/src/lib.rs:686:27 [INFO] [stdout] 13: test::run_test::{{closure}} [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/test/src/lib.rs:607:43 [INFO] [stdout] 14: test::run_test::{{closure}} [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/test/src/lib.rs:637:41 [INFO] [stdout] 15: std::sys::backtrace::__rust_begin_short_backtrace [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/sys/backtrace.rs:158:18 [INFO] [stdout] 16: std::thread::Builder::spawn_unchecked_::{{closure}}::{{closure}} [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/thread/mod.rs:559:17 [INFO] [stdout] 17: as core::ops::function::FnOnce<()>>::call_once [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/core/src/panic/unwind_safe.rs:272:9 [INFO] [stdout] 18: std::panicking::catch_unwind::do_call [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/panicking.rs:590:40 [INFO] [stdout] 19: std::panicking::catch_unwind [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/panicking.rs:553:19 [INFO] [stdout] 20: std::panic::catch_unwind [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/panic.rs:359:14 [INFO] [stdout] 21: std::thread::Builder::spawn_unchecked_::{{closure}} [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/thread/mod.rs:557:30 [INFO] [stdout] 22: core::ops::function::FnOnce::call_once{{vtable.shim}} [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/core/src/ops/function.rs:253:5 [INFO] [stdout] 23: as core::ops::function::FnOnce>::call_once [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/alloc/src/boxed.rs:1985:9 [INFO] [stdout] 24: std::sys::pal::unix::thread::Thread::new::thread_start [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/sys/pal/unix/thread.rs:118:17 [INFO] [stdout] 25: [INFO] [stdout] 26: clone [INFO] [stdout] [INFO] [stdout] ---- llm::inference::tests::test_model_params stdout ---- [INFO] [stdout] Error: No such file or directory (os error 2) [INFO] [stdout] [INFO] [stdout] Stack backtrace: [INFO] [stdout] 0: anyhow::error:: for anyhow::Error>::from [INFO] [stdout] at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/anyhow-1.0.98/src/backtrace.rs:27:14 [INFO] [stdout] 1: as core::ops::try_trait::FromResidual>>::from_residual [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/core/src/result.rs:2174:27 [INFO] [stdout] 2: mgpt::llm::inference::tests::test_model_params [INFO] [stdout] at ./src/llm/inference.rs:170:13 [INFO] [stdout] 3: mgpt::llm::inference::tests::test_model_params::{{closure}} [INFO] [stdout] at ./src/llm/inference.rs:168:31 [INFO] [stdout] 4: core::ops::function::FnOnce::call_once [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/core/src/ops/function.rs:253:5 [INFO] [stdout] 5: core::ops::function::FnOnce::call_once [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/core/src/ops/function.rs:253:5 [INFO] [stdout] 6: test::__rust_begin_short_backtrace [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/test/src/lib.rs:663:18 [INFO] [stdout] 7: test::run_test_in_process::{{closure}} [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/test/src/lib.rs:686:74 [INFO] [stdout] 8: as core::ops::function::FnOnce<()>>::call_once [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/core/src/panic/unwind_safe.rs:272:9 [INFO] [stdout] 9: std::panicking::catch_unwind::do_call [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/panicking.rs:590:40 [INFO] [stdout] 10: std::panicking::catch_unwind [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/panicking.rs:553:19 [INFO] [stdout] 11: std::panic::catch_unwind [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/panic.rs:359:14 [INFO] [stdout] 12: test::run_test_in_process [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/test/src/lib.rs:686:27 [INFO] [stdout] 13: test::run_test::{{closure}} [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/test/src/lib.rs:607:43 [INFO] [stdout] 14: test::run_test::{{closure}} [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/test/src/lib.rs:637:41 [INFO] [stdout] 15: std::sys::backtrace::__rust_begin_short_backtrace [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/sys/backtrace.rs:158:18 [INFO] [stdout] 16: std::thread::Builder::spawn_unchecked_::{{closure}}::{{closure}} [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/thread/mod.rs:559:17 [INFO] [stdout] 17: as core::ops::function::FnOnce<()>>::call_once [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/core/src/panic/unwind_safe.rs:272:9 [INFO] [stdout] 18: std::panicking::catch_unwind::do_call [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/panicking.rs:590:40 [INFO] [stdout] 19: std::panicking::catch_unwind [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/panicking.rs:553:19 [INFO] [stdout] 20: std::panic::catch_unwind [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/panic.rs:359:14 [INFO] [stdout] 21: std::thread::Builder::spawn_unchecked_::{{closure}} [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/thread/mod.rs:557:30 [INFO] [stdout] 22: core::ops::function::FnOnce::call_once{{vtable.shim}} [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/core/src/ops/function.rs:253:5 [INFO] [stdout] 23: as core::ops::function::FnOnce>::call_once [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/alloc/src/boxed.rs:1985:9 [INFO] [stdout] 24: std::sys::pal::unix::thread::Thread::new::thread_start [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/sys/pal/unix/thread.rs:118:17 [INFO] [stdout] 25: [INFO] [stdout] 26: clone [INFO] [stdout] [INFO] [stdout] ---- llm::mgpt::tests::test_llm_generate_with_tokenizer stdout ---- [INFO] [stdout] start to load tokenizer from: ./out/tokenizer/tokenizer.json [INFO] [stdout] Error: No such file or directory (os error 2) [INFO] [stdout] [INFO] [stdout] Stack backtrace: [INFO] [stdout] 0: anyhow::error:: for anyhow::Error>::from [INFO] [stdout] at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/anyhow-1.0.98/src/backtrace.rs:27:14 [INFO] [stdout] 1: as core::ops::try_trait::FromResidual>>::from_residual [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/core/src/result.rs:2174:27 [INFO] [stdout] 2: mgpt::llm::mgpt::tests::test_llm_generate_with_tokenizer [INFO] [stdout] at ./src/llm/mgpt.rs:423:25 [INFO] [stdout] 3: mgpt::llm::mgpt::tests::test_llm_generate_with_tokenizer::{{closure}} [INFO] [stdout] at ./src/llm/mgpt.rs:416:46 [INFO] [stdout] 4: core::ops::function::FnOnce::call_once [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/core/src/ops/function.rs:253:5 [INFO] [stdout] 5: core::ops::function::FnOnce::call_once [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/core/src/ops/function.rs:253:5 [INFO] [stdout] 6: test::__rust_begin_short_backtrace [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/test/src/lib.rs:663:18 [INFO] [stdout] 7: test::run_test_in_process::{{closure}} [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/test/src/lib.rs:686:74 [INFO] [stdout] 8: as core::ops::function::FnOnce<()>>::call_once [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/core/src/panic/unwind_safe.rs:272:9 [INFO] [stdout] 9: std::panicking::catch_unwind::do_call [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/panicking.rs:590:40 [INFO] [stdout] 10: std::panicking::catch_unwind [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/panicking.rs:553:19 [INFO] [stdout] 11: std::panic::catch_unwind [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/panic.rs:359:14 [INFO] [stdout] 12: test::run_test_in_process [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/test/src/lib.rs:686:27 [INFO] [stdout] 13: test::run_test::{{closure}} [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/test/src/lib.rs:607:43 [INFO] [stdout] 14: test::run_test::{{closure}} [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/test/src/lib.rs:637:41 [INFO] [stdout] 15: std::sys::backtrace::__rust_begin_short_backtrace [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/sys/backtrace.rs:158:18 [INFO] [stdout] 16: std::thread::Builder::spawn_unchecked_::{{closure}}::{{closure}} [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/thread/mod.rs:559:17 [INFO] [stdout] 17: as core::ops::function::FnOnce<()>>::call_once [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/core/src/panic/unwind_safe.rs:272:9 [INFO] [stdout] 18: std::panicking::catch_unwind::do_call [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/panicking.rs:590:40 [INFO] [stdout] 19: std::panicking::catch_unwind [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/panicking.rs:553:19 [INFO] [stdout] 20: std::panic::catch_unwind [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/panic.rs:359:14 [INFO] [stdout] 21: std::thread::Builder::spawn_unchecked_::{{closure}} [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/thread/mod.rs:557:30 [INFO] [stdout] 22: core::ops::function::FnOnce::call_once{{vtable.shim}} [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/core/src/ops/function.rs:253:5 [INFO] [stdout] 23: as core::ops::function::FnOnce>::call_once [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/alloc/src/boxed.rs:1985:9 [INFO] [stdout] 24: std::sys::pal::unix::thread::Thread::new::thread_start [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/sys/pal/unix/thread.rs:118:17 [INFO] [stdout] 25: [INFO] [stdout] 26: clone [INFO] [stdout] [INFO] [stdout] ---- llm::nn_utils::tests::test_draw_loss_series stdout ---- [INFO] [stdout] Error: Read-only file system (os error 30) [INFO] [stdout] [INFO] [stdout] Stack backtrace: [INFO] [stdout] 0: anyhow::error:: for anyhow::Error>::from [INFO] [stdout] at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/anyhow-1.0.98/src/backtrace.rs:27:14 [INFO] [stdout] 1: as core::ops::try_trait::FromResidual>>::from_residual [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/core/src/result.rs:2174:27 [INFO] [stdout] 2: mgpt::llm::nn_utils::tests::test_draw_loss_series [INFO] [stdout] at ./src/llm/nn_utils.rs:297:9 [INFO] [stdout] 3: mgpt::llm::nn_utils::tests::test_draw_loss_series::{{closure}} [INFO] [stdout] at ./src/llm/nn_utils.rs:286:35 [INFO] [stdout] 4: core::ops::function::FnOnce::call_once [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/core/src/ops/function.rs:253:5 [INFO] [stdout] 5: core::ops::function::FnOnce::call_once [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/core/src/ops/function.rs:253:5 [INFO] [stdout] 6: test::__rust_begin_short_backtrace [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/test/src/lib.rs:663:18 [INFO] [stdout] 7: test::run_test_in_process::{{closure}} [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/test/src/lib.rs:686:74 [INFO] [stdout] 8: as core::ops::function::FnOnce<()>>::call_once [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/core/src/panic/unwind_safe.rs:272:9 [INFO] [stdout] 9: std::panicking::catch_unwind::do_call [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/panicking.rs:590:40 [INFO] [stdout] 10: std::panicking::catch_unwind [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/panicking.rs:553:19 [INFO] [stdout] 11: std::panic::catch_unwind [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/panic.rs:359:14 [INFO] [stdout] 12: test::run_test_in_process [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/test/src/lib.rs:686:27 [INFO] [stdout] 13: test::run_test::{{closure}} [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/test/src/lib.rs:607:43 [INFO] [stdout] 14: test::run_test::{{closure}} [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/test/src/lib.rs:637:41 [INFO] [stdout] 15: std::sys::backtrace::__rust_begin_short_backtrace [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/sys/backtrace.rs:158:18 [INFO] [stdout] 16: std::thread::Builder::spawn_unchecked_::{{closure}}::{{closure}} [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/thread/mod.rs:559:17 [INFO] [stdout] 17: as core::ops::function::FnOnce<()>>::call_once [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/core/src/panic/unwind_safe.rs:272:9 [INFO] [stdout] 18: std::panicking::catch_unwind::do_call [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/panicking.rs:590:40 [INFO] [stdout] 19: std::panicking::catch_unwind [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/panicking.rs:553:19 [INFO] [stdout] 20: std::panic::catch_unwind [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/panic.rs:359:14 [INFO] [stdout] 21: std::thread::Builder::spawn_unchecked_::{{closure}} [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/thread/mod.rs:557:30 [INFO] [stdout] 22: core::ops::function::FnOnce::call_once{{vtable.shim}} [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/core/src/ops/function.rs:253:5 [INFO] [stdout] 23: as core::ops::function::FnOnce>::call_once [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/alloc/src/boxed.rs:1985:9 [INFO] [stdout] 24: std::sys::pal::unix::thread::Thread::new::thread_start [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/sys/pal/unix/thread.rs:118:17 [INFO] [stdout] 25: [INFO] [stdout] 26: clone [INFO] [stdout] [INFO] [stdout] ---- llm::pretrain::tests::test_pretrain_llm stdout ---- [INFO] [stdout] [INFO] [stdout] thread 'llm::pretrain::tests::test_pretrain_llm' (43) panicked at src/llm/pretrain.rs:303:80: [INFO] [stdout] called `Result::unwrap()` on an `Err` value: Os { code: 2, kind: NotFound, message: "No such file or directory" } [INFO] [stdout] stack backtrace: [INFO] [stdout] 0: 0x5b3fffe0a0a2 - std::backtrace_rs::backtrace::libunwind::trace::h7cddb8376417e7cc [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/../../backtrace/src/backtrace/libunwind.rs:117:9 [INFO] [stdout] 1: 0x5b3fffe0a0a2 - std::backtrace_rs::backtrace::trace_unsynchronized::hb4e41acf4b349ff1 [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/../../backtrace/src/backtrace/mod.rs:66:14 [INFO] [stdout] 2: 0x5b3fffe0a0a2 - std::sys::backtrace::_print_fmt::h1222b80910ba6eb5 [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/sys/backtrace.rs:66:9 [INFO] [stdout] 3: 0x5b3fffe0a0a2 - ::fmt::h5bb8a979ba5db788 [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/sys/backtrace.rs:39:26 [INFO] [stdout] 4: 0x5b3fffe1deaf - core::fmt::rt::Argument::fmt::h2ee2c138a50a7796 [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/core/src/fmt/rt.rs:173:76 [INFO] [stdout] 5: 0x5b3fffe1deaf - core::fmt::write::h1e0dbf07fe3990bd [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/core/src/fmt/mod.rs:1468:25 [INFO] [stdout] 6: 0x5b3fffdd4a43 - std::io::default_write_fmt::h7d7ad5ed6a883d81 [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/io/mod.rs:639:11 [INFO] [stdout] 7: 0x5b3fffdd4a43 - std::io::Write::write_fmt::hc477d9325b345ece [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/io/mod.rs:1954:13 [INFO] [stdout] 8: 0x5b3fffde0ef2 - std::sys::backtrace::BacktraceLock::print::h788d486777205086 [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/sys/backtrace.rs:42:9 [INFO] [stdout] 9: 0x5b3fffde67cf - std::panicking::default_hook::{{closure}}::ha7bdfeb5949fc0fa [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/panicking.rs:301:27 [INFO] [stdout] 10: 0x5b3fffde6661 - std::panicking::default_hook::h6dad75ec721846f4 [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/panicking.rs:325:9 [INFO] [stdout] 11: 0x5b3ffe6a6ece - as core::ops::function::Fn>::call::h33799d04b0b96146 [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/alloc/src/boxed.rs:1999:9 [INFO] [stdout] 12: 0x5b3ffe6a6ece - test::test_main_with_exit_callback::{{closure}}::h880e31c829a2eb5d [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/test/src/lib.rs:145:21 [INFO] [stdout] 13: 0x5b3fffde6f1e - as core::ops::function::Fn>::call::h5e1b8c652ea49180 [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/alloc/src/boxed.rs:1999:9 [INFO] [stdout] 14: 0x5b3fffde6f1e - std::panicking::panic_with_hook::hba00e869ada17676 [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/panicking.rs:842:13 [INFO] [stdout] 15: 0x5b3fffde6c3a - std::panicking::panic_handler::{{closure}}::h3d21149c258e5ceb [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/panicking.rs:707:13 [INFO] [stdout] 16: 0x5b3fffde1029 - std::sys::backtrace::__rust_end_short_backtrace::h4f5d9b9dfb3e6ec1 [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/sys/backtrace.rs:174:18 [INFO] [stdout] 17: 0x5b3fffdc7b8d - __rustc[9a7a9f9af7564de1]::rust_begin_unwind [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/panicking.rs:698:5 [INFO] [stdout] 18: 0x5b3fffe28690 - core::panicking::panic_fmt::h78e817a90331d98b [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/core/src/panicking.rs:75:14 [INFO] [stdout] 19: 0x5b3fffe276c6 - core::result::unwrap_failed::h3ce108722ad88f1e [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/core/src/result.rs:1852:5 [INFO] [stdout] 20: 0x5b3ffe5491e4 - core::result::Result::unwrap::h03fda1b00703199f [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/core/src/result.rs:1223:23 [INFO] [stdout] 21: 0x5b3ffe5491e4 - mgpt::llm::pretrain::tests::test_pretrain_llm::h1513d4b8abc15e7a [INFO] [stdout] at /opt/rustwide/workdir/src/llm/pretrain.rs:303:80 [INFO] [stdout] 22: 0x5b3ffe54abf7 - mgpt::llm::pretrain::tests::test_pretrain_llm::{{closure}}::hd5e8f8a9e8503420 [INFO] [stdout] at /opt/rustwide/workdir/src/llm/pretrain.rs:301:31 [INFO] [stdout] 23: 0x5b3ffe60fdd6 - core::ops::function::FnOnce::call_once::hf660962598b4c62b [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/core/src/ops/function.rs:253:5 [INFO] [stdout] 24: 0x5b3ffe6a6ccb - core::ops::function::FnOnce::call_once::h7f4b4fba903e39d5 [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/core/src/ops/function.rs:253:5 [INFO] [stdout] 25: 0x5b3ffe6a6ccb - test::__rust_begin_short_backtrace::h9277cb6a2ccfc000 [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/test/src/lib.rs:663:18 [INFO] [stdout] 26: 0x5b3ffe6bc775 - test::run_test_in_process::{{closure}}::h9aea5ca90d1f4423 [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/test/src/lib.rs:686:74 [INFO] [stdout] 27: 0x5b3ffe6bc775 - as core::ops::function::FnOnce<()>>::call_once::h364f5fe6cc8afa85 [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/core/src/panic/unwind_safe.rs:272:9 [INFO] [stdout] 28: 0x5b3ffe6bc775 - std::panicking::catch_unwind::do_call::h2b2bec3317fe53ec [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/panicking.rs:590:40 [INFO] [stdout] 29: 0x5b3ffe6bc775 - std::panicking::catch_unwind::hc3763734156da4af [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/panicking.rs:553:19 [INFO] [stdout] 30: 0x5b3ffe6bc775 - std::panic::catch_unwind::h28038391e867eabc [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/panic.rs:359:14 [INFO] [stdout] 31: 0x5b3ffe6bc775 - test::run_test_in_process::had0273166695a036 [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/test/src/lib.rs:686:27 [INFO] [stdout] 32: 0x5b3ffe6bc775 - test::run_test::{{closure}}::h743d09d4bb476605 [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/test/src/lib.rs:607:43 [INFO] [stdout] 33: 0x5b3ffe693194 - test::run_test::{{closure}}::he950b8f9118d37e2 [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/test/src/lib.rs:637:41 [INFO] [stdout] 34: 0x5b3ffe693194 - std::sys::backtrace::__rust_begin_short_backtrace::hbfaffa6539f6abb7 [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/sys/backtrace.rs:158:18 [INFO] [stdout] 35: 0x5b3ffe696a7a - std::thread::Builder::spawn_unchecked_::{{closure}}::{{closure}}::hcce3c2c65b9c3b20 [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/thread/mod.rs:559:17 [INFO] [stdout] 36: 0x5b3ffe696a7a - as core::ops::function::FnOnce<()>>::call_once::hb66b30b6d37985a5 [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/core/src/panic/unwind_safe.rs:272:9 [INFO] [stdout] 37: 0x5b3ffe696a7a - std::panicking::catch_unwind::do_call::h992bbe2c32dc1d79 [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/panicking.rs:590:40 [INFO] [stdout] 38: 0x5b3ffe696a7a - std::panicking::catch_unwind::h412991d5237de610 [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/panicking.rs:553:19 [INFO] [stdout] 39: 0x5b3ffe696a7a - std::panic::catch_unwind::ha82b139b3eb5840a [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/panic.rs:359:14 [INFO] [stdout] 40: 0x5b3ffe696a7a - std::thread::Builder::spawn_unchecked_::{{closure}}::ha823b36f5114938e [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/thread/mod.rs:557:30 [INFO] [stdout] 41: 0x5b3ffe696a7a - core::ops::function::FnOnce::call_once{{vtable.shim}}::hf120e7b1db22ac07 [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/core/src/ops/function.rs:253:5 [INFO] [stdout] 42: 0x5b3fffddafaf - as core::ops::function::FnOnce>::call_once::h3e049222c99298ac [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/alloc/src/boxed.rs:1985:9 [INFO] [stdout] 43: 0x5b3fffddafaf - std::sys::pal::unix::thread::Thread::new::thread_start::h942e336943ad5963 [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/sys/pal/unix/thread.rs:118:17 [INFO] [stdout] 44: 0x76862176faa4 - [INFO] [stdout] 45: 0x7686217fca34 - clone [INFO] [stdout] 46: 0x0 - [INFO] [stdout] [INFO] [stdout] ---- tokenizer::bert_tokenizer::tests::test_custom_tokenizer stdout ---- [INFO] [stdout] Error: Read-only file system (os error 30) [INFO] [stdout] [INFO] [stdout] Stack backtrace: [INFO] [stdout] 0: anyhow::error:: for anyhow::Error>::from [INFO] [stdout] at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/anyhow-1.0.98/src/backtrace.rs:27:14 [INFO] [stdout] 1: as core::ops::try_trait::FromResidual>>::from_residual [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/core/src/result.rs:2174:27 [INFO] [stdout] 2: mgpt::tokenizer::bert_tokenizer::tests::test_custom_tokenizer [INFO] [stdout] at ./src/tokenizer/bert_tokenizer.rs:57:9 [INFO] [stdout] 3: mgpt::tokenizer::bert_tokenizer::tests::test_custom_tokenizer::{{closure}} [INFO] [stdout] at ./src/tokenizer/bert_tokenizer.rs:53:35 [INFO] [stdout] 4: core::ops::function::FnOnce::call_once [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/core/src/ops/function.rs:253:5 [INFO] [stdout] 5: core::ops::function::FnOnce::call_once [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/core/src/ops/function.rs:253:5 [INFO] [stdout] 6: test::__rust_begin_short_backtrace [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/test/src/lib.rs:663:18 [INFO] [stdout] 7: test::run_test_in_process::{{closure}} [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/test/src/lib.rs:686:74 [INFO] [stdout] 8: as core::ops::function::FnOnce<()>>::call_once [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/core/src/panic/unwind_safe.rs:272:9 [INFO] [stdout] 9: std::panicking::catch_unwind::do_call [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/panicking.rs:590:40 [INFO] [stdout] 10: std::panicking::catch_unwind [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/panicking.rs:553:19 [INFO] [stdout] 11: std::panic::catch_unwind [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/panic.rs:359:14 [INFO] [stdout] 12: test::run_test_in_process [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/test/src/lib.rs:686:27 [INFO] [stdout] 13: test::run_test::{{closure}} [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/test/src/lib.rs:607:43 [INFO] [stdout] 14: test::run_test::{{closure}} [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/test/src/lib.rs:637:41 [INFO] [stdout] 15: std::sys::backtrace::__rust_begin_short_backtrace [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/sys/backtrace.rs:158:18 [INFO] [stdout] 16: std::thread::Builder::spawn_unchecked_::{{closure}}::{{closure}} [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/thread/mod.rs:559:17 [INFO] [stdout] 17: as core::ops::function::FnOnce<()>>::call_once [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/core/src/panic/unwind_safe.rs:272:9 [INFO] [stdout] 18: std::panicking::catch_unwind::do_call [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/panicking.rs:590:40 [INFO] [stdout] 19: std::panicking::catch_unwind [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/panicking.rs:553:19 [INFO] [stdout] 20: std::panic::catch_unwind [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/panic.rs:359:14 [INFO] [stdout] 21: std::thread::Builder::spawn_unchecked_::{{closure}} [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/thread/mod.rs:557:30 [INFO] [stdout] 22: core::ops::function::FnOnce::call_once{{vtable.shim}} [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/core/src/ops/function.rs:253:5 [INFO] [stdout] 23: as core::ops::function::FnOnce>::call_once [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/alloc/src/boxed.rs:1985:9 [INFO] [stdout] 24: std::sys::pal::unix::thread::Thread::new::thread_start [INFO] [stdout] at /rustc/cdb45c87e2cd43495379f7e867e3cc15dcee9f93/library/std/src/sys/pal/unix/thread.rs:118:17 [INFO] [stdout] 25: [INFO] [stdout] 26: clone [INFO] [stdout] [INFO] [stdout] [INFO] [stdout] failures: [INFO] [stdout] datasets::pretrain_dataset_loader::tests::test_iter_overall_loader [INFO] [stdout] datasets::pretrain_dataset_loader::tests::test_load_tokenizer [INFO] [stdout] datasets::pretrain_dataset_loader::tests::test_make_result_empty_input [INFO] [stdout] datasets::pretrain_dataset_loader::tests::test_make_result_valid_input [INFO] [stdout] datasets::pretrain_dataset_loader::tests::test_process_line_padding_and_truncation [INFO] [stdout] datasets::pretrain_dataset_loader::tests::test_prtrain_dataset_loader [INFO] [stdout] llm::inference::tests::test_batch_inputs_with_padding [INFO] [stdout] llm::inference::tests::test_inference [INFO] [stdout] llm::inference::tests::test_model_params [INFO] [stdout] llm::mgpt::tests::test_llm_generate_for_batch_inputs [INFO] [stdout] llm::mgpt::tests::test_llm_generate_with_tokenizer [INFO] [stdout] llm::nn_utils::tests::test_draw_loss_series [INFO] [stdout] llm::pretrain::tests::test_pretrain_llm [INFO] [stdout] tokenizer::bert_tokenizer::tests::test_custom_tokenizer [INFO] [stdout] [INFO] [stdout] test result: FAILED. 24 passed; 14 failed; 0 ignored; 0 measured; 0 filtered out; finished in 4.23s [INFO] [stdout] [INFO] [stderr] error: test failed, to rerun pass `--lib` [INFO] running `Command { std: "docker" "inspect" "6c04e3b829b04a87c90c0fa69bc852d1706fe273aaf9907e201c4cddd747d8e5", kill_on_drop: false }` [INFO] running `Command { std: "docker" "rm" "-f" "6c04e3b829b04a87c90c0fa69bc852d1706fe273aaf9907e201c4cddd747d8e5", kill_on_drop: false }` [INFO] [stdout] 6c04e3b829b04a87c90c0fa69bc852d1706fe273aaf9907e201c4cddd747d8e5