[INFO] cloning repository https://github.com/errorhugo/gpt-from-scratch-in-rust [INFO] running `Command { std: "git" "-c" "credential.helper=" "-c" "credential.helper=/workspace/cargo-home/bin/git-credential-null" "clone" "--bare" "https://github.com/errorhugo/gpt-from-scratch-in-rust" "/workspace/cache/git-repos/https%3A%2F%2Fgithub.com%2Ferrorhugo%2Fgpt-from-scratch-in-rust", kill_on_drop: false }` [INFO] [stderr] Cloning into bare repository '/workspace/cache/git-repos/https%3A%2F%2Fgithub.com%2Ferrorhugo%2Fgpt-from-scratch-in-rust'... [INFO] running `Command { std: "git" "rev-parse" "HEAD", kill_on_drop: false }` [INFO] [stdout] 7b10a2d92926a305a1b996a1e5a713262c435c0c [INFO] testing errorhugo/gpt-from-scratch-in-rust against try#16fbf98f22730e073c09e7b03f0eafb87a295545 for pr-145628 [INFO] running `Command { std: "git" "clone" "/workspace/cache/git-repos/https%3A%2F%2Fgithub.com%2Ferrorhugo%2Fgpt-from-scratch-in-rust" "/workspace/builds/worker-5-tc2/source", kill_on_drop: false }` [INFO] [stderr] Cloning into '/workspace/builds/worker-5-tc2/source'... [INFO] [stderr] done. [INFO] started tweaking git repo https://github.com/errorhugo/gpt-from-scratch-in-rust [INFO] finished tweaking git repo https://github.com/errorhugo/gpt-from-scratch-in-rust [INFO] tweaked toml for git repo https://github.com/errorhugo/gpt-from-scratch-in-rust written to /workspace/builds/worker-5-tc2/source/Cargo.toml [INFO] validating manifest of git repo https://github.com/errorhugo/gpt-from-scratch-in-rust on toolchain 16fbf98f22730e073c09e7b03f0eafb87a295545 [INFO] running `Command { std: CARGO_HOME="/workspace/cargo-home" RUSTUP_HOME="/workspace/rustup-home" "/workspace/cargo-home/bin/cargo" "+16fbf98f22730e073c09e7b03f0eafb87a295545" "metadata" "--manifest-path" "Cargo.toml" "--no-deps", kill_on_drop: false }` [INFO] crate git repo https://github.com/errorhugo/gpt-from-scratch-in-rust already has a lockfile, it will not be regenerated [INFO] running `Command { std: CARGO_HOME="/workspace/cargo-home" RUSTUP_HOME="/workspace/rustup-home" "/workspace/cargo-home/bin/cargo" "+16fbf98f22730e073c09e7b03f0eafb87a295545" "fetch" "--manifest-path" "Cargo.toml", kill_on_drop: false }` [INFO] running `Command { std: "docker" "create" "-v" "/var/lib/crater-agent-workspace/builds/worker-5-tc2/target:/opt/rustwide/target:rw,Z" "-v" "/var/lib/crater-agent-workspace/builds/worker-5-tc2/source:/opt/rustwide/workdir:ro,Z" "-v" "/var/lib/crater-agent-workspace/cargo-home:/opt/rustwide/cargo-home:ro,Z" "-v" "/var/lib/crater-agent-workspace/rustup-home:/opt/rustwide/rustup-home:ro,Z" "-e" "SOURCE_DIR=/opt/rustwide/workdir" "-e" "CARGO_TARGET_DIR=/opt/rustwide/target" "-e" "CARGO_HOME=/opt/rustwide/cargo-home" "-e" "RUSTUP_HOME=/opt/rustwide/rustup-home" "-w" "/opt/rustwide/workdir" "-m" "1610612736" "--user" "0:0" "--network" "none" "ghcr.io/rust-lang/crates-build-env/linux@sha256:7ad1b28ee6f5f7f699f6cf7015098d6ccdd96d6f2d78dd06228f5b4c9faf309c" "/opt/rustwide/cargo-home/bin/cargo" "+16fbf98f22730e073c09e7b03f0eafb87a295545" "metadata" "--no-deps" "--format-version=1", kill_on_drop: false }` [INFO] [stdout] 6ebd668b3255fa9aa8d3f05aec75fac8b7665091c55ec3e7cd2b20a8d4b85d0a [INFO] running `Command { std: "docker" "start" "-a" "6ebd668b3255fa9aa8d3f05aec75fac8b7665091c55ec3e7cd2b20a8d4b85d0a", kill_on_drop: false }` [INFO] running `Command { std: "docker" "inspect" "6ebd668b3255fa9aa8d3f05aec75fac8b7665091c55ec3e7cd2b20a8d4b85d0a", kill_on_drop: false }` [INFO] running `Command { std: "docker" "rm" "-f" "6ebd668b3255fa9aa8d3f05aec75fac8b7665091c55ec3e7cd2b20a8d4b85d0a", kill_on_drop: false }` [INFO] [stdout] 6ebd668b3255fa9aa8d3f05aec75fac8b7665091c55ec3e7cd2b20a8d4b85d0a [INFO] running `Command { std: "docker" "create" "-v" "/var/lib/crater-agent-workspace/builds/worker-5-tc2/target:/opt/rustwide/target:rw,Z" "-v" "/var/lib/crater-agent-workspace/builds/worker-5-tc2/source:/opt/rustwide/workdir:ro,Z" "-v" "/var/lib/crater-agent-workspace/cargo-home:/opt/rustwide/cargo-home:ro,Z" "-v" "/var/lib/crater-agent-workspace/rustup-home:/opt/rustwide/rustup-home:ro,Z" "-e" "SOURCE_DIR=/opt/rustwide/workdir" "-e" "CARGO_TARGET_DIR=/opt/rustwide/target" "-e" "CARGO_INCREMENTAL=0" "-e" "RUST_BACKTRACE=full" "-e" "RUSTFLAGS=--cap-lints=forbid" "-e" "RUSTDOCFLAGS=--cap-lints=forbid" "-e" "CARGO_HOME=/opt/rustwide/cargo-home" "-e" "RUSTUP_HOME=/opt/rustwide/rustup-home" "-w" "/opt/rustwide/workdir" "-m" "1610612736" "--user" "0:0" "--network" "none" "ghcr.io/rust-lang/crates-build-env/linux@sha256:7ad1b28ee6f5f7f699f6cf7015098d6ccdd96d6f2d78dd06228f5b4c9faf309c" "/opt/rustwide/cargo-home/bin/cargo" "+16fbf98f22730e073c09e7b03f0eafb87a295545" "build" "--frozen" "--message-format=json", kill_on_drop: false }` [INFO] [stdout] 321451fbdd1b8ba58280c9e83ec8c498cdcddfc307139aadfa7dd9413d6f18d1 [INFO] running `Command { std: "docker" "start" "-a" "321451fbdd1b8ba58280c9e83ec8c498cdcddfc307139aadfa7dd9413d6f18d1", kill_on_drop: false }` [INFO] [stderr] Compiling libm v0.2.15 [INFO] [stderr] Compiling num-traits v0.2.19 [INFO] [stderr] Compiling cc v1.2.30 [INFO] [stderr] Compiling syn v2.0.104 [INFO] [stderr] Compiling getrandom v0.3.3 [INFO] [stderr] Compiling reborrow v0.5.5 [INFO] [stderr] Compiling crc32fast v1.5.0 [INFO] [stderr] Compiling pulp v0.21.5 [INFO] [stderr] Compiling rustversion v1.0.21 [INFO] [stderr] Compiling rand_core v0.9.3 [INFO] [stderr] Compiling raw-cpuid v11.5.0 [INFO] [stderr] Compiling serde_json v1.0.141 [INFO] [stderr] Compiling rand_chacha v0.9.0 [INFO] [stderr] Compiling raw-cpuid v10.7.0 [INFO] [stderr] Compiling adler2 v2.0.1 [INFO] [stderr] Compiling miniz_oxide v0.8.9 [INFO] [stderr] Compiling rand v0.9.2 [INFO] [stderr] Compiling smallvec v1.15.1 [INFO] [stderr] Compiling hashbrown v0.15.4 [INFO] [stderr] Compiling ring v0.17.14 [INFO] [stderr] Compiling zeroize v1.8.1 [INFO] [stderr] Compiling dirs-sys v0.5.0 [INFO] [stderr] Compiling indexmap v2.10.0 [INFO] [stderr] Compiling rustls-pki-types v1.12.0 [INFO] [stderr] Compiling flate2 v1.1.2 [INFO] [stderr] Compiling libloading v0.8.8 [INFO] [stderr] Compiling winnow v0.7.12 [INFO] [stderr] Compiling rand_distr v0.5.1 [INFO] [stderr] Compiling num-integer v0.1.46 [INFO] [stderr] Compiling dirs v6.0.0 [INFO] [stderr] Compiling getrandom v0.2.16 [INFO] [stderr] Compiling unicode-width v0.2.1 [INFO] [stderr] Compiling portable-atomic v1.11.1 [INFO] [stderr] Compiling pathfinder_simd v0.5.5 [INFO] [stderr] Compiling num-bigint v0.4.6 [INFO] [stderr] Compiling freetype-sys v0.20.1 [INFO] [stderr] Compiling yeslogic-fontconfig-sys v6.0.0 [INFO] [stderr] Compiling aho-corasick v1.1.3 [INFO] [stderr] Compiling percent-encoding v2.3.1 [INFO] [stderr] Compiling rustls v0.23.29 [INFO] [stderr] Compiling form_urlencoded v1.2.1 [INFO] [stderr] Compiling anstyle-parse v0.2.7 [INFO] [stderr] Compiling num-iter v0.1.45 [INFO] [stderr] Compiling webpki-roots v1.0.2 [INFO] [stderr] Compiling onig_sys v69.9.1 [INFO] [stderr] Compiling castaway v0.2.4 [INFO] [stderr] Compiling tracing-core v0.1.34 [INFO] [stderr] Compiling indexmap v1.9.3 [INFO] [stderr] Compiling anstyle v1.0.11 [INFO] [stderr] Compiling subtle v2.6.1 [INFO] [stderr] Compiling anstyle-query v1.1.3 [INFO] [stderr] Compiling virtue v0.0.18 [INFO] [stderr] Compiling synstructure v0.13.2 [INFO] [stderr] Compiling toml_edit v0.22.27 [INFO] [stderr] Compiling darling_core v0.20.11 [INFO] [stderr] Compiling regex-automata v0.4.9 [INFO] [stderr] Compiling num-rational v0.4.2 [INFO] [stderr] Compiling colorchoice v1.0.4 [INFO] [stderr] Compiling zip v1.1.4 [INFO] [stderr] Compiling anstream v0.6.19 [INFO] [stderr] Compiling bincode_derive v2.0.1 [INFO] [stderr] Compiling png v0.17.16 [INFO] [stderr] Compiling bytemuck_derive v1.10.0 [INFO] [stderr] Compiling zerofrom-derive v0.1.6 [INFO] [stderr] Compiling serde_derive v1.0.219 [INFO] [stderr] Compiling bytemuck v1.23.1 [INFO] [stderr] Compiling displaydoc v0.2.5 [INFO] [stderr] Compiling yoke-derive v0.8.0 [INFO] [stderr] Compiling zerofrom v0.1.6 [INFO] [stderr] Compiling zerovec-derive v0.11.1 [INFO] [stderr] Compiling num-complex v0.4.6 [INFO] [stderr] Compiling half v2.6.0 [INFO] [stderr] Compiling dyn-stack v0.13.0 [INFO] [stderr] Compiling pulp v0.18.22 [INFO] [stderr] Compiling dyn-stack v0.10.0 [INFO] [stderr] Compiling thiserror-impl v1.0.69 [INFO] [stderr] Compiling yoke v0.8.0 [INFO] [stderr] Compiling zerovec v0.11.2 [INFO] [stderr] Compiling zerotrie v0.2.2 [INFO] [stderr] Compiling darling_macro v0.20.11 [INFO] [stderr] Compiling gemm-common v0.17.1 [INFO] [stderr] Compiling proc-macro-crate v3.3.0 [INFO] [stderr] Compiling darling v0.20.11 [INFO] [stderr] Compiling rustls-webpki v0.103.4 [INFO] [stderr] Compiling thiserror v1.0.69 [INFO] [stderr] Compiling yoke-derive v0.7.5 [INFO] [stderr] Compiling tracing-attributes v0.1.30 [INFO] [stderr] Compiling tinystr v0.8.1 [INFO] [stderr] Compiling potential_utf v0.1.2 [INFO] [stderr] Compiling icu_locale_core v2.0.0 [INFO] [stderr] Compiling icu_collections v2.0.0 [INFO] [stderr] Compiling num_enum_derive v0.7.4 [INFO] [stderr] Compiling serde v1.0.219 [INFO] [stderr] Compiling gemm-f32 v0.17.1 [INFO] [stderr] Compiling gemm-common v0.18.2 [INFO] [stderr] Compiling icu_provider v2.0.0 [INFO] [stderr] Compiling yoke v0.7.5 [INFO] [stderr] Compiling num_enum v0.7.4 [INFO] [stderr] Compiling icu_normalizer v2.0.0 [INFO] [stderr] Compiling icu_properties v2.0.1 [INFO] [stderr] Compiling tracing v0.1.41 [INFO] [stderr] Compiling gemm-f32 v0.18.2 [INFO] [stderr] Compiling gemm-f64 v0.18.2 [INFO] [stderr] Compiling gemm-c32 v0.18.2 [INFO] [stderr] Compiling gemm-f16 v0.18.2 [INFO] [stderr] Compiling idna_adapter v1.2.1 [INFO] [stderr] Compiling gemm-c64 v0.18.2 [INFO] [stderr] Compiling gemm-f16 v0.17.1 [INFO] [stderr] Compiling idna v1.0.3 [INFO] [stderr] Compiling url v2.5.4 [INFO] [stderr] Compiling gemm-f64 v0.17.1 [INFO] [stderr] Compiling gemm v0.18.2 [INFO] [stderr] Compiling gemm-c32 v0.17.1 [INFO] [stderr] Compiling gemm-c64 v0.17.1 [INFO] [stderr] Compiling derive_builder_core v0.20.2 [INFO] [stderr] Compiling safetensors v0.4.5 [INFO] [stderr] Compiling num v0.4.3 [INFO] [stderr] Compiling regex v1.11.1 [INFO] [stderr] Compiling thiserror-impl v2.0.12 [INFO] [stderr] Compiling webpki-roots v0.26.11 [INFO] [stderr] Compiling console v0.15.11 [INFO] [stderr] Compiling dlib v0.5.2 [INFO] [stderr] Compiling esaxx-rs v0.1.10 [INFO] [stderr] Compiling socks v0.3.4 [INFO] [stderr] Compiling ahash v0.8.12 [INFO] [stderr] Compiling memmap2 v0.9.7 [INFO] [stderr] Compiling num_cpus v1.17.0 [INFO] [stderr] Compiling weezl v0.1.10 [INFO] [stderr] Compiling plotters-backend v0.3.7 [INFO] [stderr] Compiling font-kit v0.14.3 [INFO] [stderr] Compiling unty v0.0.4 [INFO] [stderr] Compiling jpeg-decoder v0.3.2 [INFO] [stderr] Compiling clap_lex v0.7.5 [INFO] [stderr] Compiling clap_derive v4.5.41 [INFO] [stderr] Compiling chrono v0.4.41 [INFO] [stderr] Compiling clap_builder v4.5.41 [INFO] [stderr] Compiling image v0.24.9 [INFO] [stderr] Compiling bincode v2.0.1 [INFO] [stderr] Compiling gif v0.12.0 [INFO] [stderr] Compiling ureq v2.12.1 [INFO] [stderr] Compiling derive_more v0.99.20 [INFO] [stderr] Compiling indicatif v0.17.11 [INFO] [stderr] Compiling ug v0.4.0 [INFO] [stderr] Compiling nom v7.1.3 [INFO] [stderr] Compiling thiserror v2.0.12 [INFO] [stderr] Compiling pathfinder_geometry v0.5.1 [INFO] [stderr] Compiling derive_builder_macro v0.20.2 [INFO] [stderr] Compiling gemm v0.17.1 [INFO] [stderr] Compiling monostate-impl v0.1.14 [INFO] [stderr] Compiling syn v1.0.109 [INFO] [stderr] Compiling itertools v0.14.0 [INFO] [stderr] Compiling float-ord v0.3.2 [INFO] [stderr] Compiling macro_rules_attribute-proc_macro v0.2.2 [INFO] [stderr] Compiling spm_precompiled v0.1.4 [INFO] [stderr] Compiling nichi v0.5.1 [INFO] [stderr] Compiling macro_rules_attribute v0.2.2 [INFO] [stderr] Compiling candle-core v0.9.1 [INFO] [stderr] Compiling hf-hub v0.4.3 [INFO] [stderr] Compiling monostate v0.1.14 [INFO] [stderr] Compiling onig v6.5.1 [INFO] [stderr] Compiling derive_builder v0.20.2 [INFO] [stderr] Compiling plotters-bitmap v0.3.7 [INFO] [stderr] Compiling rayon-cond v0.4.0 [INFO] [stderr] Compiling clap v4.5.41 [INFO] [stderr] Compiling serde_yaml v0.8.26 [INFO] [stderr] Compiling plotters-svg v0.3.7 [INFO] [stderr] Compiling compact_str v0.7.1 [INFO] [stderr] Compiling compact_str v0.9.0 [INFO] [stderr] Compiling envy v0.4.2 [INFO] [stderr] Compiling dary_heap v0.3.7 [INFO] [stderr] Compiling console v0.16.0 [INFO] [stderr] Compiling shellexpand v3.1.1 [INFO] [stderr] Compiling unicode-normalization-alignments v0.1.12 [INFO] [stderr] Compiling unit-prefix v0.5.1 [INFO] [stderr] Compiling unicode_categories v0.1.1 [INFO] [stderr] Compiling ttf-parser v0.20.0 [INFO] [stderr] Compiling indicatif v0.18.0 [INFO] [stderr] Compiling readable v0.16.0 [INFO] [stderr] Compiling tokenizers v0.21.2 [INFO] [stderr] Compiling colored v3.0.0 [INFO] [stderr] Compiling config-derive v0.15.0 [INFO] [stderr] Compiling plotters v0.3.7 [INFO] [stderr] Compiling twelf v0.15.0 [INFO] [stderr] Compiling candle-nn v0.9.1 [INFO] [stderr] Compiling mgpt v0.1.0 (/opt/rustwide/workdir) [INFO] [stderr] Finished `dev` profile [unoptimized + debuginfo] target(s) in 30.31s [INFO] running `Command { std: "docker" "inspect" "321451fbdd1b8ba58280c9e83ec8c498cdcddfc307139aadfa7dd9413d6f18d1", kill_on_drop: false }` [INFO] running `Command { std: "docker" "rm" "-f" "321451fbdd1b8ba58280c9e83ec8c498cdcddfc307139aadfa7dd9413d6f18d1", kill_on_drop: false }` [INFO] [stdout] 321451fbdd1b8ba58280c9e83ec8c498cdcddfc307139aadfa7dd9413d6f18d1 [INFO] running `Command { std: "docker" "create" "-v" "/var/lib/crater-agent-workspace/builds/worker-5-tc2/target:/opt/rustwide/target:rw,Z" "-v" "/var/lib/crater-agent-workspace/builds/worker-5-tc2/source:/opt/rustwide/workdir:ro,Z" "-v" "/var/lib/crater-agent-workspace/cargo-home:/opt/rustwide/cargo-home:ro,Z" "-v" "/var/lib/crater-agent-workspace/rustup-home:/opt/rustwide/rustup-home:ro,Z" "-e" "SOURCE_DIR=/opt/rustwide/workdir" "-e" "CARGO_TARGET_DIR=/opt/rustwide/target" "-e" "CARGO_INCREMENTAL=0" "-e" "RUST_BACKTRACE=full" "-e" "RUSTFLAGS=--cap-lints=forbid" "-e" "RUSTDOCFLAGS=--cap-lints=forbid" "-e" "CARGO_HOME=/opt/rustwide/cargo-home" "-e" "RUSTUP_HOME=/opt/rustwide/rustup-home" "-w" "/opt/rustwide/workdir" "-m" "1610612736" "--user" "0:0" "--network" "none" "ghcr.io/rust-lang/crates-build-env/linux@sha256:7ad1b28ee6f5f7f699f6cf7015098d6ccdd96d6f2d78dd06228f5b4c9faf309c" "/opt/rustwide/cargo-home/bin/cargo" "+16fbf98f22730e073c09e7b03f0eafb87a295545" "test" "--frozen" "--no-run" "--message-format=json", kill_on_drop: false }` [INFO] [stdout] 5bca9c604c0b122d92125a14b5375c106b719c75b9b9d9518acb21cc9fd1f6d2 [INFO] running `Command { std: "docker" "start" "-a" "5bca9c604c0b122d92125a14b5375c106b719c75b9b9d9518acb21cc9fd1f6d2", kill_on_drop: false }` [INFO] [stderr] Compiling mgpt v0.1.0 (/opt/rustwide/workdir) [INFO] [stderr] Finished `test` profile [unoptimized + debuginfo] target(s) in 2.48s [INFO] running `Command { std: "docker" "inspect" "5bca9c604c0b122d92125a14b5375c106b719c75b9b9d9518acb21cc9fd1f6d2", kill_on_drop: false }` [INFO] running `Command { std: "docker" "rm" "-f" "5bca9c604c0b122d92125a14b5375c106b719c75b9b9d9518acb21cc9fd1f6d2", kill_on_drop: false }` [INFO] [stdout] 5bca9c604c0b122d92125a14b5375c106b719c75b9b9d9518acb21cc9fd1f6d2 [INFO] running `Command { std: "docker" "create" "-v" "/var/lib/crater-agent-workspace/builds/worker-5-tc2/target:/opt/rustwide/target:rw,Z" "-v" "/var/lib/crater-agent-workspace/builds/worker-5-tc2/source:/opt/rustwide/workdir:ro,Z" "-v" "/var/lib/crater-agent-workspace/cargo-home:/opt/rustwide/cargo-home:ro,Z" "-v" "/var/lib/crater-agent-workspace/rustup-home:/opt/rustwide/rustup-home:ro,Z" "-e" "SOURCE_DIR=/opt/rustwide/workdir" "-e" "CARGO_TARGET_DIR=/opt/rustwide/target" "-e" "CARGO_INCREMENTAL=0" "-e" "RUST_BACKTRACE=full" "-e" "RUSTFLAGS=--cap-lints=forbid" "-e" "RUSTDOCFLAGS=--cap-lints=forbid" "-e" "CARGO_HOME=/opt/rustwide/cargo-home" "-e" "RUSTUP_HOME=/opt/rustwide/rustup-home" "-w" "/opt/rustwide/workdir" "-m" "1610612736" "--user" "0:0" "--network" "none" "ghcr.io/rust-lang/crates-build-env/linux@sha256:7ad1b28ee6f5f7f699f6cf7015098d6ccdd96d6f2d78dd06228f5b4c9faf309c" "/opt/rustwide/cargo-home/bin/cargo" "+16fbf98f22730e073c09e7b03f0eafb87a295545" "test" "--frozen", kill_on_drop: false }` [INFO] [stdout] d53a5d915d7f3036aa230a7c1c37b856621811a5d2947a1efc015b7178e557b6 [INFO] running `Command { std: "docker" "start" "-a" "d53a5d915d7f3036aa230a7c1c37b856621811a5d2947a1efc015b7178e557b6", kill_on_drop: false }` [INFO] [stderr] Finished `test` profile [unoptimized + debuginfo] target(s) in 0.15s [INFO] [stderr] Running unittests src/lib.rs (/opt/rustwide/target/debug/deps/mgpt-38dd3fe9065cd5dc) [INFO] [stdout] [INFO] [stdout] running 38 tests [INFO] [stdout] test datasets::pretrain_dataset_loader::tests::test_decode_line_valid_and_invalid_json ... ok [INFO] [stdout] test config::tests::test_load_config ... ok [INFO] [stdout] test llm::feedforward::tests::test_mlp_forward ... ok [INFO] [stdout] test datasets::pretrain_dataset_loader::tests::test_iter_overall_loader ... FAILED [INFO] [stdout] test datasets::pretrain_dataset_loader::tests::test_load_tokenizer ... FAILED [INFO] [stdout] test datasets::pretrain_dataset_loader::tests::test_make_result_empty_input ... FAILED [INFO] [stdout] test datasets::pretrain_dataset_loader::tests::test_make_result_valid_input ... FAILED [INFO] [stdout] test datasets::pretrain_dataset_loader::tests::test_prtrain_dataset_loader ... FAILED [INFO] [stdout] test llm::inference::tests::test_batch_inputs_with_padding ... FAILED [INFO] [stdout] test llm::inference::tests::test_model_params ... FAILED [INFO] [stdout] test llm::inference::tests::test_inference ... FAILED [INFO] [stdout] test datasets::pretrain_dataset_loader::tests::test_process_line_padding_and_truncation ... FAILED [INFO] [stdout] test llm::multi_head_attention::tests::test_tril ... ok [INFO] [stdout] test llm::multi_head_attention::tests::test_multihead_attn_forward ... ok [INFO] [stdout] test llm::mgpt::tests::test_llm_generate_for_batch_inputs ... FAILED [INFO] [stdout] test llm::mgpt::tests::test_llm_generate_with_tokenizer ... FAILED [INFO] [stdout] test llm::nn_utils::tests::test_masked_cross_entropy_basic ... ok [INFO] [stdout] test llm::nn_utils::tests::test_multinomial_sample_batch_2_vocab_4_n3 ... ok [INFO] [stdout] test llm::nn_utils::tests::test_multinomial_sample_invalid_input ... ok [INFO] [stdout] test llm::nn_utils::tests::test_multinomial_sample_zero_prob ... ok [INFO] [stdout] test llm::positional_encoding::tests::test_sinusoidial_position_encoding ... ok [INFO] [stdout] test llm::scheduler::tests::test_lr_scheduler_cosine ... ok [INFO] [stdout] test llm::nn_utils::tests::test_draw_loss_series ... FAILED [INFO] [stdout] test llm::pretrain::tests::test_pretrain_llm ... FAILED [INFO] [stdout] test processor::logits_processor_list::tests::test_logits_processor_list ... ok [INFO] [stdout] test processor::repetition_penalty_processor::tests::test_repetition_penalty_processor ... ok [INFO] [stdout] test processor::temperature_processor::tests::test_temperature_processor ... ok [INFO] [stdout] test processor::topk_processor::tests::test_top_k_processor ... ok [INFO] [stdout] test processor::topp_processor::tests::test_top_p_processor ... ok [INFO] [stdout] test processor::utils::tests::test_normalized_random_vec ... ok [INFO] [stdout] test llm::mgpt::tests::test_llm_forward ... ok [INFO] [stdout] test llm::transformer_block::tests::test_transformer_block ... ok [INFO] [stdout] test llm::mgpt::tests::test_llm_generate_with_1batch ... ok [INFO] [stdout] test llm::mgpt::tests::test_llm_generate ... ok [INFO] [stdout] test llm::mgpt::tests::test_llm_generate_with_generation_config ... ok [INFO] [stdout] test tokenizer::bert_tokenizer::tests::test_custom_tokenizer ... FAILED [INFO] [stdout] test llm::scheduler::tests::test_plot_schedule ... ok [INFO] [stdout] test llm::pretrain::tests::test_pretrain_llm_with_manual_data ... ok [INFO] [stdout] [INFO] [stdout] failures: [INFO] [stdout] [INFO] [stdout] ---- datasets::pretrain_dataset_loader::tests::test_iter_overall_loader stdout ---- [INFO] [stdout] start to load tokenizer from: ./out/tokenizer/tokenizer.json [INFO] [stdout] Error: No such file or directory (os error 2) [INFO] [stdout] [INFO] [stdout] Stack backtrace: [INFO] [stdout] 0: anyhow::error:: for anyhow::Error>::from [INFO] [stdout] at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/anyhow-1.0.98/src/backtrace.rs:27:14 [INFO] [stdout] 1: as core::ops::try_trait::FromResidual>>::from_residual [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/core/src/result.rs:2174:27 [INFO] [stdout] 2: mgpt::datasets::pretrain_dataset_loader::tests::test_iter_overall_loader [INFO] [stdout] at ./src/datasets/pretrain_dataset_loader.rs:367:25 [INFO] [stdout] 3: mgpt::datasets::pretrain_dataset_loader::tests::test_iter_overall_loader::{{closure}} [INFO] [stdout] at ./src/datasets/pretrain_dataset_loader.rs:366:38 [INFO] [stdout] 4: core::ops::function::FnOnce::call_once [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/core/src/ops/function.rs:253:5 [INFO] [stdout] 5: core::ops::function::FnOnce::call_once [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/core/src/ops/function.rs:253:5 [INFO] [stdout] 6: test::__rust_begin_short_backtrace [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/test/src/lib.rs:663:18 [INFO] [stdout] 7: test::run_test_in_process::{{closure}} [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/test/src/lib.rs:686:74 [INFO] [stdout] 8: as core::ops::function::FnOnce<()>>::call_once [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/core/src/panic/unwind_safe.rs:272:9 [INFO] [stdout] 9: std::panicking::catch_unwind::do_call [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/panicking.rs:590:40 [INFO] [stdout] 10: std::panicking::catch_unwind [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/panicking.rs:553:19 [INFO] [stdout] 11: std::panic::catch_unwind [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/panic.rs:359:14 [INFO] [stdout] 12: test::run_test_in_process [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/test/src/lib.rs:686:27 [INFO] [stdout] 13: test::run_test::{{closure}} [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/test/src/lib.rs:607:43 [INFO] [stdout] 14: test::run_test::{{closure}} [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/test/src/lib.rs:637:41 [INFO] [stdout] 15: std::sys::backtrace::__rust_begin_short_backtrace [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/sys/backtrace.rs:158:18 [INFO] [stdout] 16: std::thread::Builder::spawn_unchecked_::{{closure}}::{{closure}} [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/thread/mod.rs:559:17 [INFO] [stdout] 17: as core::ops::function::FnOnce<()>>::call_once [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/core/src/panic/unwind_safe.rs:272:9 [INFO] [stdout] 18: std::panicking::catch_unwind::do_call [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/panicking.rs:590:40 [INFO] [stdout] 19: std::panicking::catch_unwind [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/panicking.rs:553:19 [INFO] [stdout] 20: std::panic::catch_unwind [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/panic.rs:359:14 [INFO] [stdout] 21: std::thread::Builder::spawn_unchecked_::{{closure}} [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/thread/mod.rs:557:30 [INFO] [stdout] 22: core::ops::function::FnOnce::call_once{{vtable.shim}} [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/core/src/ops/function.rs:253:5 [INFO] [stdout] 23: as core::ops::function::FnOnce>::call_once [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/alloc/src/boxed.rs:1985:9 [INFO] [stdout] 24: std::sys::pal::unix::thread::Thread::new::thread_start [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/sys/pal/unix/thread.rs:118:17 [INFO] [stdout] 25: [INFO] [stdout] 26: clone [INFO] [stdout] [INFO] [stdout] ---- datasets::pretrain_dataset_loader::tests::test_load_tokenizer stdout ---- [INFO] [stdout] start to load tokenizer from: ./out/tokenizer/tokenizer.json [INFO] [stdout] Error: No such file or directory (os error 2) [INFO] [stdout] [INFO] [stdout] Stack backtrace: [INFO] [stdout] 0: anyhow::error:: for anyhow::Error>::from [INFO] [stdout] at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/anyhow-1.0.98/src/backtrace.rs:27:14 [INFO] [stdout] 1: as core::ops::try_trait::FromResidual>>::from_residual [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/core/src/result.rs:2174:27 [INFO] [stdout] 2: mgpt::datasets::pretrain_dataset_loader::tests::test_load_tokenizer [INFO] [stdout] at ./src/datasets/pretrain_dataset_loader.rs:239:25 [INFO] [stdout] 3: mgpt::datasets::pretrain_dataset_loader::tests::test_load_tokenizer::{{closure}} [INFO] [stdout] at ./src/datasets/pretrain_dataset_loader.rs:238:33 [INFO] [stdout] 4: core::ops::function::FnOnce::call_once [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/core/src/ops/function.rs:253:5 [INFO] [stdout] 5: core::ops::function::FnOnce::call_once [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/core/src/ops/function.rs:253:5 [INFO] [stdout] 6: test::__rust_begin_short_backtrace [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/test/src/lib.rs:663:18 [INFO] [stdout] 7: test::run_test_in_process::{{closure}} [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/test/src/lib.rs:686:74 [INFO] [stdout] 8: as core::ops::function::FnOnce<()>>::call_once [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/core/src/panic/unwind_safe.rs:272:9 [INFO] [stdout] 9: std::panicking::catch_unwind::do_call [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/panicking.rs:590:40 [INFO] [stdout] 10: std::panicking::catch_unwind [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/panicking.rs:553:19 [INFO] [stdout] 11: std::panic::catch_unwind [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/panic.rs:359:14 [INFO] [stdout] 12: test::run_test_in_process [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/test/src/lib.rs:686:27 [INFO] [stdout] 13: test::run_test::{{closure}} [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/test/src/lib.rs:607:43 [INFO] [stdout] 14: test::run_test::{{closure}} [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/test/src/lib.rs:637:41 [INFO] [stdout] 15: std::sys::backtrace::__rust_begin_short_backtrace [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/sys/backtrace.rs:158:18 [INFO] [stdout] 16: std::thread::Builder::spawn_unchecked_::{{closure}}::{{closure}} [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/thread/mod.rs:559:17 [INFO] [stdout] 17: as core::ops::function::FnOnce<()>>::call_once [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/core/src/panic/unwind_safe.rs:272:9 [INFO] [stdout] 18: std::panicking::catch_unwind::do_call [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/panicking.rs:590:40 [INFO] [stdout] 19: std::panicking::catch_unwind [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/panicking.rs:553:19 [INFO] [stdout] 20: std::panic::catch_unwind [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/panic.rs:359:14 [INFO] [stdout] 21: std::thread::Builder::spawn_unchecked_::{{closure}} [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/thread/mod.rs:557:30 [INFO] [stdout] 22: core::ops::function::FnOnce::call_once{{vtable.shim}} [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/core/src/ops/function.rs:253:5 [INFO] [stdout] 23: as core::ops::function::FnOnce>::call_once [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/alloc/src/boxed.rs:1985:9 [INFO] [stdout] 24: std::sys::pal::unix::thread::Thread::new::thread_start [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/sys/pal/unix/thread.rs:118:17 [INFO] [stdout] 25: [INFO] [stdout] 26: clone [INFO] [stdout] [INFO] [stdout] ---- datasets::pretrain_dataset_loader::tests::test_make_result_empty_input stdout ---- [INFO] [stdout] start to load tokenizer from: ./out/tokenizer/tokenizer.json [INFO] [stdout] [INFO] [stdout] thread 'datasets::pretrain_dataset_loader::tests::test_make_result_empty_input' (21) panicked at src/datasets/pretrain_dataset_loader.rs:323:74: [INFO] [stdout] called `Result::unwrap()` on an `Err` value: TokenizersError(Os { code: 2, kind: NotFound, message: "No such file or directory" }) [INFO] [stdout] stack backtrace: [INFO] [stdout] 0: 0x56451b81b522 - std::backtrace_rs::backtrace::libunwind::trace::h4b3f5960ed1c0917 [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/../../backtrace/src/backtrace/libunwind.rs:117:9 [INFO] [stdout] 1: 0x56451b81b522 - std::backtrace_rs::backtrace::trace_unsynchronized::h2f16383c47f8bd04 [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/../../backtrace/src/backtrace/mod.rs:66:14 [INFO] [stdout] 2: 0x56451b81b522 - std::sys::backtrace::_print_fmt::h698d661724c270ea [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/sys/backtrace.rs:66:9 [INFO] [stdout] 3: 0x56451b81b522 - ::fmt::h1c8375013274064c [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/sys/backtrace.rs:39:26 [INFO] [stdout] 4: 0x56451b82f34f - core::fmt::rt::Argument::fmt::h9232377a772d3f8c [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/core/src/fmt/rt.rs:173:76 [INFO] [stdout] 5: 0x56451b82f34f - core::fmt::write::h6dc9e168423d9633 [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/core/src/fmt/mod.rs:1468:25 [INFO] [stdout] 6: 0x56451b7e5ec3 - std::io::default_write_fmt::hec53a38f10739f86 [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/io/mod.rs:639:11 [INFO] [stdout] 7: 0x56451b7e5ec3 - std::io::Write::write_fmt::hf5089d8bbbe5aaa6 [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/io/mod.rs:1954:13 [INFO] [stdout] 8: 0x56451b7f2372 - std::sys::backtrace::BacktraceLock::print::hf83c3d8cbd01f904 [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/sys/backtrace.rs:42:9 [INFO] [stdout] 9: 0x56451b7f7c4f - std::panicking::default_hook::{{closure}}::h3d4c49adc0898362 [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/panicking.rs:301:27 [INFO] [stdout] 10: 0x56451b7f7ae1 - std::panicking::default_hook::hccea28f2487e7471 [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/panicking.rs:325:9 [INFO] [stdout] 11: 0x56451a0be0ce - as core::ops::function::Fn>::call::h06e1b96f84d5792f [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/alloc/src/boxed.rs:1999:9 [INFO] [stdout] 12: 0x56451a0be0ce - test::test_main_with_exit_callback::{{closure}}::hf6db6d7b1242fcd4 [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/test/src/lib.rs:145:21 [INFO] [stdout] 13: 0x56451b7f839e - as core::ops::function::Fn>::call::hfeb094aff1e6591e [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/alloc/src/boxed.rs:1999:9 [INFO] [stdout] 14: 0x56451b7f839e - std::panicking::panic_with_hook::h694b8196629eb5e1 [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/panicking.rs:842:13 [INFO] [stdout] 15: 0x56451b7f80ba - std::panicking::panic_handler::{{closure}}::h0d99dd765f7e6554 [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/panicking.rs:707:13 [INFO] [stdout] 16: 0x56451b7f24b9 - std::sys::backtrace::__rust_end_short_backtrace::h8b43cbde72f7d592 [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/sys/backtrace.rs:174:18 [INFO] [stdout] 17: 0x56451b7d900d - __rustc[d1dec8fc82f9a585]::rust_begin_unwind [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/panicking.rs:698:5 [INFO] [stdout] 18: 0x56451b839b30 - core::panicking::panic_fmt::h315eeb0e4d90fcdc [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/core/src/panicking.rs:75:14 [INFO] [stdout] 19: 0x56451b838b66 - core::result::unwrap_failed::h3188dfca3824df71 [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/core/src/result.rs:1852:5 [INFO] [stdout] 20: 0x564519f82fa2 - core::result::Result::unwrap::hcd53f1a1a9e42b13 [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/core/src/result.rs:1223:23 [INFO] [stdout] 21: 0x564519f82fa2 - mgpt::datasets::pretrain_dataset_loader::tests::test_make_result_empty_input::h9c0a93606fd10c95 [INFO] [stdout] at /opt/rustwide/workdir/src/datasets/pretrain_dataset_loader.rs:323:74 [INFO] [stdout] 22: 0x564519f83387 - mgpt::datasets::pretrain_dataset_loader::tests::test_make_result_empty_input::{{closure}}::h034ce37fe64ee369 [INFO] [stdout] at /opt/rustwide/workdir/src/datasets/pretrain_dataset_loader.rs:322:38 [INFO] [stdout] 23: 0x56451a025f06 - core::ops::function::FnOnce::call_once::he320c0fd6af8d488 [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/core/src/ops/function.rs:253:5 [INFO] [stdout] 24: 0x56451a0bdf3b - core::ops::function::FnOnce::call_once::h5852f0076c8c0d8d [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/core/src/ops/function.rs:253:5 [INFO] [stdout] 25: 0x56451a0bdf3b - test::__rust_begin_short_backtrace::hc8457233131a06d0 [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/test/src/lib.rs:663:18 [INFO] [stdout] 26: 0x56451a0d3995 - test::run_test_in_process::{{closure}}::ha3eda53784ae2639 [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/test/src/lib.rs:686:74 [INFO] [stdout] 27: 0x56451a0d3995 - as core::ops::function::FnOnce<()>>::call_once::hbd60f847cd185bc6 [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/core/src/panic/unwind_safe.rs:272:9 [INFO] [stdout] 28: 0x56451a0d3995 - std::panicking::catch_unwind::do_call::h11c1623aed953668 [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/panicking.rs:590:40 [INFO] [stdout] 29: 0x56451a0d3995 - std::panicking::catch_unwind::hf8551f73b52d8ed0 [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/panicking.rs:553:19 [INFO] [stdout] 30: 0x56451a0d3995 - std::panic::catch_unwind::h66ff870d847fc7cf [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/panic.rs:359:14 [INFO] [stdout] 31: 0x56451a0d3995 - test::run_test_in_process::h808a5fa5cb24efde [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/test/src/lib.rs:686:27 [INFO] [stdout] 32: 0x56451a0d3995 - test::run_test::{{closure}}::hcf5782fef76f68bf [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/test/src/lib.rs:607:43 [INFO] [stdout] 33: 0x56451a0aa364 - test::run_test::{{closure}}::h59bb1f8f7167fee7 [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/test/src/lib.rs:637:41 [INFO] [stdout] 34: 0x56451a0aa364 - std::sys::backtrace::__rust_begin_short_backtrace::h073f551083d60480 [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/sys/backtrace.rs:158:18 [INFO] [stdout] 35: 0x56451a0adc4a - std::thread::Builder::spawn_unchecked_::{{closure}}::{{closure}}::h9d6eff1e4da9d0f2 [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/thread/mod.rs:559:17 [INFO] [stdout] 36: 0x56451a0adc4a - as core::ops::function::FnOnce<()>>::call_once::he37f75191b40d4b0 [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/core/src/panic/unwind_safe.rs:272:9 [INFO] [stdout] 37: 0x56451a0adc4a - std::panicking::catch_unwind::do_call::h339b26991b458079 [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/panicking.rs:590:40 [INFO] [stdout] 38: 0x56451a0adc4a - std::panicking::catch_unwind::h2e5906b8a84cca89 [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/panicking.rs:553:19 [INFO] [stdout] 39: 0x56451a0adc4a - std::panic::catch_unwind::h3fea03a95c906d68 [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/panic.rs:359:14 [INFO] [stdout] 40: 0x56451a0adc4a - std::thread::Builder::spawn_unchecked_::{{closure}}::hab1ac593ae8241c8 [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/thread/mod.rs:557:30 [INFO] [stdout] 41: 0x56451a0adc4a - core::ops::function::FnOnce::call_once{{vtable.shim}}::hb46ae6da79faf2c5 [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/core/src/ops/function.rs:253:5 [INFO] [stdout] 42: 0x56451b7ec42f - as core::ops::function::FnOnce>::call_once::h317217a5c86437d1 [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/alloc/src/boxed.rs:1985:9 [INFO] [stdout] 43: 0x56451b7ec42f - std::sys::pal::unix::thread::Thread::new::thread_start::h9b0e1220f8cf5182 [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/sys/pal/unix/thread.rs:118:17 [INFO] [stdout] 44: 0x7299d8f4caa4 - [INFO] [stdout] 45: 0x7299d8fd9a34 - clone [INFO] [stdout] 46: 0x0 - [INFO] [stdout] [INFO] [stdout] ---- datasets::pretrain_dataset_loader::tests::test_make_result_valid_input stdout ---- [INFO] [stdout] start to load tokenizer from: ./out/tokenizer/tokenizer.json [INFO] [stdout] [INFO] [stdout] thread 'datasets::pretrain_dataset_loader::tests::test_make_result_valid_input' (22) panicked at src/datasets/pretrain_dataset_loader.rs:343:74: [INFO] [stdout] called `Result::unwrap()` on an `Err` value: TokenizersError(Os { code: 2, kind: NotFound, message: "No such file or directory" }) [INFO] [stdout] stack backtrace: [INFO] [stdout] 0: 0x56451b81b522 - std::backtrace_rs::backtrace::libunwind::trace::h4b3f5960ed1c0917 [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/../../backtrace/src/backtrace/libunwind.rs:117:9 [INFO] [stdout] 1: 0x56451b81b522 - std::backtrace_rs::backtrace::trace_unsynchronized::h2f16383c47f8bd04 [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/../../backtrace/src/backtrace/mod.rs:66:14 [INFO] [stdout] 2: 0x56451b81b522 - std::sys::backtrace::_print_fmt::h698d661724c270ea [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/sys/backtrace.rs:66:9 [INFO] [stdout] 3: 0x56451b81b522 - ::fmt::h1c8375013274064c [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/sys/backtrace.rs:39:26 [INFO] [stdout] 4: 0x56451b82f34f - core::fmt::rt::Argument::fmt::h9232377a772d3f8c [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/core/src/fmt/rt.rs:173:76 [INFO] [stdout] 5: 0x56451b82f34f - core::fmt::write::h6dc9e168423d9633 [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/core/src/fmt/mod.rs:1468:25 [INFO] [stdout] 6: 0x56451b7e5ec3 - std::io::default_write_fmt::hec53a38f10739f86 [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/io/mod.rs:639:11 [INFO] [stdout] 7: 0x56451b7e5ec3 - std::io::Write::write_fmt::hf5089d8bbbe5aaa6 [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/io/mod.rs:1954:13 [INFO] [stdout] 8: 0x56451b7f2372 - std::sys::backtrace::BacktraceLock::print::hf83c3d8cbd01f904 [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/sys/backtrace.rs:42:9 [INFO] [stdout] 9: 0x56451b7f7c4f - std::panicking::default_hook::{{closure}}::h3d4c49adc0898362 [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/panicking.rs:301:27 [INFO] [stdout] 10: 0x56451b7f7ae1 - std::panicking::default_hook::hccea28f2487e7471 [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/panicking.rs:325:9 [INFO] [stdout] 11: 0x56451a0be0ce - as core::ops::function::Fn>::call::h06e1b96f84d5792f [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/alloc/src/boxed.rs:1999:9 [INFO] [stdout] 12: 0x56451a0be0ce - test::test_main_with_exit_callback::{{closure}}::hf6db6d7b1242fcd4 [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/test/src/lib.rs:145:21 [INFO] [stdout] 13: 0x56451b7f839e - as core::ops::function::Fn>::call::hfeb094aff1e6591e [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/alloc/src/boxed.rs:1999:9 [INFO] [stdout] 14: 0x56451b7f839e - std::panicking::panic_with_hook::h694b8196629eb5e1 [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/panicking.rs:842:13 [INFO] [stdout] 15: 0x56451b7f80ba - std::panicking::panic_handler::{{closure}}::h0d99dd765f7e6554 [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/panicking.rs:707:13 [INFO] [stdout] 16: 0x56451b7f24b9 - std::sys::backtrace::__rust_end_short_backtrace::h8b43cbde72f7d592 [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/sys/backtrace.rs:174:18 [INFO] [stdout] 17: 0x56451b7d900d - __rustc[d1dec8fc82f9a585]::rust_begin_unwind [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/panicking.rs:698:5 [INFO] [stdout] 18: 0x56451b839b30 - core::panicking::panic_fmt::h315eeb0e4d90fcdc [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/core/src/panicking.rs:75:14 [INFO] [stdout] 19: 0x56451b838b66 - core::result::unwrap_failed::h3188dfca3824df71 [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/core/src/result.rs:1852:5 [INFO] [stdout] 20: 0x564519f8347a - core::result::Result::unwrap::hcd53f1a1a9e42b13 [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/core/src/result.rs:1223:23 [INFO] [stdout] 21: 0x564519f8347a - mgpt::datasets::pretrain_dataset_loader::tests::test_make_result_valid_input::h2f35e90863074299 [INFO] [stdout] at /opt/rustwide/workdir/src/datasets/pretrain_dataset_loader.rs:343:74 [INFO] [stdout] 22: 0x564519f83ff7 - mgpt::datasets::pretrain_dataset_loader::tests::test_make_result_valid_input::{{closure}}::h5071391811b63bca [INFO] [stdout] at /opt/rustwide/workdir/src/datasets/pretrain_dataset_loader.rs:342:38 [INFO] [stdout] 23: 0x56451a025c16 - core::ops::function::FnOnce::call_once::hc7d1856d140854e0 [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/core/src/ops/function.rs:253:5 [INFO] [stdout] 24: 0x56451a0bdf3b - core::ops::function::FnOnce::call_once::h5852f0076c8c0d8d [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/core/src/ops/function.rs:253:5 [INFO] [stdout] 25: 0x56451a0bdf3b - test::__rust_begin_short_backtrace::hc8457233131a06d0 [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/test/src/lib.rs:663:18 [INFO] [stdout] 26: 0x56451a0d3995 - test::run_test_in_process::{{closure}}::ha3eda53784ae2639 [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/test/src/lib.rs:686:74 [INFO] [stdout] 27: 0x56451a0d3995 - as core::ops::function::FnOnce<()>>::call_once::hbd60f847cd185bc6 [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/core/src/panic/unwind_safe.rs:272:9 [INFO] [stdout] 28: 0x56451a0d3995 - std::panicking::catch_unwind::do_call::h11c1623aed953668 [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/panicking.rs:590:40 [INFO] [stdout] 29: 0x56451a0d3995 - std::panicking::catch_unwind::hf8551f73b52d8ed0 [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/panicking.rs:553:19 [INFO] [stdout] 30: 0x56451a0d3995 - std::panic::catch_unwind::h66ff870d847fc7cf [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/panic.rs:359:14 [INFO] [stdout] 31: 0x56451a0d3995 - test::run_test_in_process::h808a5fa5cb24efde [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/test/src/lib.rs:686:27 [INFO] [stdout] 32: 0x56451a0d3995 - test::run_test::{{closure}}::hcf5782fef76f68bf [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/test/src/lib.rs:607:43 [INFO] [stdout] 33: 0x56451a0aa364 - test::run_test::{{closure}}::h59bb1f8f7167fee7 [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/test/src/lib.rs:637:41 [INFO] [stdout] 34: 0x56451a0aa364 - std::sys::backtrace::__rust_begin_short_backtrace::h073f551083d60480 [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/sys/backtrace.rs:158:18 [INFO] [stdout] 35: 0x56451a0adc4a - std::thread::Builder::spawn_unchecked_::{{closure}}::{{closure}}::h9d6eff1e4da9d0f2 [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/thread/mod.rs:559:17 [INFO] [stdout] 36: 0x56451a0adc4a - as core::ops::function::FnOnce<()>>::call_once::he37f75191b40d4b0 [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/core/src/panic/unwind_safe.rs:272:9 [INFO] [stdout] 37: 0x56451a0adc4a - std::panicking::catch_unwind::do_call::h339b26991b458079 [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/panicking.rs:590:40 [INFO] [stdout] 38: 0x56451a0adc4a - std::panicking::catch_unwind::h2e5906b8a84cca89 [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/panicking.rs:553:19 [INFO] [stdout] 39: 0x56451a0adc4a - std::panic::catch_unwind::h3fea03a95c906d68 [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/panic.rs:359:14 [INFO] [stdout] 40: 0x56451a0adc4a - std::thread::Builder::spawn_unchecked_::{{closure}}::hab1ac593ae8241c8 [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/thread/mod.rs:557:30 [INFO] [stdout] 41: 0x56451a0adc4a - core::ops::function::FnOnce::call_once{{vtable.shim}}::hb46ae6da79faf2c5 [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/core/src/ops/function.rs:253:5 [INFO] [stdout] 42: 0x56451b7ec42f - as core::ops::function::FnOnce>::call_once::h317217a5c86437d1 [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/alloc/src/boxed.rs:1985:9 [INFO] [stdout] 43: 0x56451b7ec42f - std::sys::pal::unix::thread::Thread::new::thread_start::h9b0e1220f8cf5182 [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/sys/pal/unix/thread.rs:118:17 [INFO] [stdout] 44: 0x7299d8f4caa4 - [INFO] [stdout] 45: 0x7299d8fd9a34 - clone [INFO] [stdout] 46: 0x0 - [INFO] [stdout] [INFO] [stdout] ---- datasets::pretrain_dataset_loader::tests::test_prtrain_dataset_loader stdout ---- [INFO] [stdout] start to load tokenizer from: ./out/tokenizer/tokenizer.json [INFO] [stdout] Error: No such file or directory (os error 2) [INFO] [stdout] [INFO] [stdout] Stack backtrace: [INFO] [stdout] 0: anyhow::error:: for anyhow::Error>::from [INFO] [stdout] at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/anyhow-1.0.98/src/backtrace.rs:27:14 [INFO] [stdout] 1: as core::ops::try_trait::FromResidual>>::from_residual [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/core/src/result.rs:2174:27 [INFO] [stdout] 2: mgpt::datasets::pretrain_dataset_loader::tests::test_prtrain_dataset_loader [INFO] [stdout] at ./src/datasets/pretrain_dataset_loader.rs:246:25 [INFO] [stdout] 3: mgpt::datasets::pretrain_dataset_loader::tests::test_prtrain_dataset_loader::{{closure}} [INFO] [stdout] at ./src/datasets/pretrain_dataset_loader.rs:245:41 [INFO] [stdout] 4: core::ops::function::FnOnce::call_once [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/core/src/ops/function.rs:253:5 [INFO] [stdout] 5: core::ops::function::FnOnce::call_once [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/core/src/ops/function.rs:253:5 [INFO] [stdout] 6: test::__rust_begin_short_backtrace [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/test/src/lib.rs:663:18 [INFO] [stdout] 7: test::run_test_in_process::{{closure}} [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/test/src/lib.rs:686:74 [INFO] [stdout] 8: as core::ops::function::FnOnce<()>>::call_once [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/core/src/panic/unwind_safe.rs:272:9 [INFO] [stdout] 9: std::panicking::catch_unwind::do_call [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/panicking.rs:590:40 [INFO] [stdout] 10: std::panicking::catch_unwind [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/panicking.rs:553:19 [INFO] [stdout] 11: std::panic::catch_unwind [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/panic.rs:359:14 [INFO] [stdout] 12: test::run_test_in_process [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/test/src/lib.rs:686:27 [INFO] [stdout] 13: test::run_test::{{closure}} [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/test/src/lib.rs:607:43 [INFO] [stdout] 14: test::run_test::{{closure}} [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/test/src/lib.rs:637:41 [INFO] [stdout] 15: std::sys::backtrace::__rust_begin_short_backtrace [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/sys/backtrace.rs:158:18 [INFO] [stdout] 16: std::thread::Builder::spawn_unchecked_::{{closure}}::{{closure}} [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/thread/mod.rs:559:17 [INFO] [stdout] 17: as core::ops::function::FnOnce<()>>::call_once [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/core/src/panic/unwind_safe.rs:272:9 [INFO] [stdout] 18: std::panicking::catch_unwind::do_call [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/panicking.rs:590:40 [INFO] [stdout] 19: std::panicking::catch_unwind [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/panicking.rs:553:19 [INFO] [stdout] 20: std::panic::catch_unwind [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/panic.rs:359:14 [INFO] [stdout] 21: std::thread::Builder::spawn_unchecked_::{{closure}} [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/thread/mod.rs:557:30 [INFO] [stdout] 22: core::ops::function::FnOnce::call_once{{vtable.shim}} [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/core/src/ops/function.rs:253:5 [INFO] [stdout] 23: as core::ops::function::FnOnce>::call_once [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/alloc/src/boxed.rs:1985:9 [INFO] [stdout] 24: std::sys::pal::unix::thread::Thread::new::thread_start [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/sys/pal/unix/thread.rs:118:17 [INFO] [stdout] 25: [INFO] [stdout] 26: clone [INFO] [stdout] [INFO] [stdout] ---- llm::inference::tests::test_batch_inputs_with_padding stdout ---- [INFO] [stdout] start to load tokenizer from: ./out/tokenizer/tokenizer.json [INFO] [stdout] Error: No such file or directory (os error 2) [INFO] [stdout] [INFO] [stdout] Stack backtrace: [INFO] [stdout] 0: anyhow::error:: for anyhow::Error>::from [INFO] [stdout] at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/anyhow-1.0.98/src/backtrace.rs:27:14 [INFO] [stdout] 1: as core::ops::try_trait::FromResidual>>::from_residual [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/core/src/result.rs:2174:27 [INFO] [stdout] 2: mgpt::llm::inference::tests::test_batch_inputs_with_padding [INFO] [stdout] at ./src/llm/inference.rs:142:25 [INFO] [stdout] 3: mgpt::llm::inference::tests::test_batch_inputs_with_padding::{{closure}} [INFO] [stdout] at ./src/llm/inference.rs:141:44 [INFO] [stdout] 4: core::ops::function::FnOnce::call_once [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/core/src/ops/function.rs:253:5 [INFO] [stdout] 5: core::ops::function::FnOnce::call_once [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/core/src/ops/function.rs:253:5 [INFO] [stdout] 6: test::__rust_begin_short_backtrace [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/test/src/lib.rs:663:18 [INFO] [stdout] 7: test::run_test_in_process::{{closure}} [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/test/src/lib.rs:686:74 [INFO] [stdout] 8: as core::ops::function::FnOnce<()>>::call_once [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/core/src/panic/unwind_safe.rs:272:9 [INFO] [stdout] 9: std::panicking::catch_unwind::do_call [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/panicking.rs:590:40 [INFO] [stdout] 10: std::panicking::catch_unwind [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/panicking.rs:553:19 [INFO] [stdout] 11: std::panic::catch_unwind [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/panic.rs:359:14 [INFO] [stdout] 12: test::run_test_in_process [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/test/src/lib.rs:686:27 [INFO] [stdout] 13: test::run_test::{{closure}} [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/test/src/lib.rs:607:43 [INFO] [stdout] 14: test::run_test::{{closure}} [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/test/src/lib.rs:637:41 [INFO] [stdout] 15: std::sys::backtrace::__rust_begin_short_backtrace [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/sys/backtrace.rs:158:18 [INFO] [stdout] 16: std::thread::Builder::spawn_unchecked_::{{closure}}::{{closure}} [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/thread/mod.rs:559:17 [INFO] [stdout] 17: as core::ops::function::FnOnce<()>>::call_once [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/core/src/panic/unwind_safe.rs:272:9 [INFO] [stdout] 18: std::panicking::catch_unwind::do_call [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/panicking.rs:590:40 [INFO] [stdout] 19: std::panicking::catch_unwind [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/panicking.rs:553:19 [INFO] [stdout] 20: std::panic::catch_unwind [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/panic.rs:359:14 [INFO] [stdout] 21: std::thread::Builder::spawn_unchecked_::{{closure}} [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/thread/mod.rs:557:30 [INFO] [stdout] 22: core::ops::function::FnOnce::call_once{{vtable.shim}} [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/core/src/ops/function.rs:253:5 [INFO] [stdout] 23: as core::ops::function::FnOnce>::call_once [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/alloc/src/boxed.rs:1985:9 [INFO] [stdout] 24: std::sys::pal::unix::thread::Thread::new::thread_start [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/sys/pal/unix/thread.rs:118:17 [INFO] [stdout] 25: [INFO] [stdout] 26: clone [INFO] [stdout] [INFO] [stdout] ---- llm::inference::tests::test_model_params stdout ---- [INFO] [stdout] Error: No such file or directory (os error 2) [INFO] [stdout] [INFO] [stdout] Stack backtrace: [INFO] [stdout] 0: anyhow::error:: for anyhow::Error>::from [INFO] [stdout] at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/anyhow-1.0.98/src/backtrace.rs:27:14 [INFO] [stdout] 1: as core::ops::try_trait::FromResidual>>::from_residual [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/core/src/result.rs:2174:27 [INFO] [stdout] 2: mgpt::llm::inference::tests::test_model_params [INFO] [stdout] at ./src/llm/inference.rs:170:13 [INFO] [stdout] 3: mgpt::llm::inference::tests::test_model_params::{{closure}} [INFO] [stdout] at ./src/llm/inference.rs:168:31 [INFO] [stdout] 4: core::ops::function::FnOnce::call_once [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/core/src/ops/function.rs:253:5 [INFO] [stdout] 5: core::ops::function::FnOnce::call_once [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/core/src/ops/function.rs:253:5 [INFO] [stdout] 6: test::__rust_begin_short_backtrace [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/test/src/lib.rs:663:18 [INFO] [stdout] 7: test::run_test_in_process::{{closure}} [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/test/src/lib.rs:686:74 [INFO] [stdout] 8: as core::ops::function::FnOnce<()>>::call_once [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/core/src/panic/unwind_safe.rs:272:9 [INFO] [stdout] 9: std::panicking::catch_unwind::do_call [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/panicking.rs:590:40 [INFO] [stdout] 10: std::panicking::catch_unwind [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/panicking.rs:553:19 [INFO] [stdout] 11: std::panic::catch_unwind [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/panic.rs:359:14 [INFO] [stdout] 12: test::run_test_in_process [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/test/src/lib.rs:686:27 [INFO] [stdout] 13: test::run_test::{{closure}} [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/test/src/lib.rs:607:43 [INFO] [stdout] 14: test::run_test::{{closure}} [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/test/src/lib.rs:637:41 [INFO] [stdout] 15: std::sys::backtrace::__rust_begin_short_backtrace [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/sys/backtrace.rs:158:18 [INFO] [stdout] 16: std::thread::Builder::spawn_unchecked_::{{closure}}::{{closure}} [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/thread/mod.rs:559:17 [INFO] [stdout] 17: as core::ops::function::FnOnce<()>>::call_once [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/core/src/panic/unwind_safe.rs:272:9 [INFO] [stdout] 18: std::panicking::catch_unwind::do_call [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/panicking.rs:590:40 [INFO] [stdout] 19: std::panicking::catch_unwind [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/panicking.rs:553:19 [INFO] [stdout] 20: std::panic::catch_unwind [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/panic.rs:359:14 [INFO] [stdout] 21: std::thread::Builder::spawn_unchecked_::{{closure}} [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/thread/mod.rs:557:30 [INFO] [stdout] 22: core::ops::function::FnOnce::call_once{{vtable.shim}} [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/core/src/ops/function.rs:253:5 [INFO] [stdout] 23: as core::ops::function::FnOnce>::call_once [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/alloc/src/boxed.rs:1985:9 [INFO] [stdout] 24: std::sys::pal::unix::thread::Thread::new::thread_start [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/sys/pal/unix/thread.rs:118:17 [INFO] [stdout] 25: [INFO] [stdout] 26: clone [INFO] [stdout] [INFO] [stdout] ---- llm::inference::tests::test_inference stdout ---- [INFO] [stdout] start to load tokenizer from: ./out/tokenizer/tokenizer.json [INFO] [stdout] Error: No such file or directory (os error 2) [INFO] [stdout] [INFO] [stdout] Stack backtrace: [INFO] [stdout] 0: anyhow::error:: for anyhow::Error>::from [INFO] [stdout] at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/anyhow-1.0.98/src/backtrace.rs:27:14 [INFO] [stdout] 1: as core::ops::try_trait::FromResidual>>::from_residual [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/core/src/result.rs:2174:27 [INFO] [stdout] 2: mgpt::llm::inference::tests::test_inference [INFO] [stdout] at ./src/llm/inference.rs:105:25 [INFO] [stdout] 3: mgpt::llm::inference::tests::test_inference::{{closure}} [INFO] [stdout] at ./src/llm/inference.rs:103:28 [INFO] [stdout] 4: core::ops::function::FnOnce::call_once [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/core/src/ops/function.rs:253:5 [INFO] [stdout] 5: core::ops::function::FnOnce::call_once [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/core/src/ops/function.rs:253:5 [INFO] [stdout] 6: test::__rust_begin_short_backtrace [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/test/src/lib.rs:663:18 [INFO] [stdout] 7: test::run_test_in_process::{{closure}} [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/test/src/lib.rs:686:74 [INFO] [stdout] 8: as core::ops::function::FnOnce<()>>::call_once [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/core/src/panic/unwind_safe.rs:272:9 [INFO] [stdout] 9: std::panicking::catch_unwind::do_call [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/panicking.rs:590:40 [INFO] [stdout] 10: std::panicking::catch_unwind [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/panicking.rs:553:19 [INFO] [stdout] 11: std::panic::catch_unwind [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/panic.rs:359:14 [INFO] [stdout] 12: test::run_test_in_process [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/test/src/lib.rs:686:27 [INFO] [stdout] 13: test::run_test::{{closure}} [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/test/src/lib.rs:607:43 [INFO] [stdout] 14: test::run_test::{{closure}} [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/test/src/lib.rs:637:41 [INFO] [stdout] 15: std::sys::backtrace::__rust_begin_short_backtrace [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/sys/backtrace.rs:158:18 [INFO] [stdout] 16: std::thread::Builder::spawn_unchecked_::{{closure}}::{{closure}} [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/thread/mod.rs:559:17 [INFO] [stdout] 17: as core::ops::function::FnOnce<()>>::call_once [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/core/src/panic/unwind_safe.rs:272:9 [INFO] [stdout] 18: std::panicking::catch_unwind::do_call [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/panicking.rs:590:40 [INFO] [stdout] 19: std::panicking::catch_unwind [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/panicking.rs:553:19 [INFO] [stdout] 20: std::panic::catch_unwind [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/panic.rs:359:14 [INFO] [stdout] 21: std::thread::Builder::spawn_unchecked_::{{closure}} [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/thread/mod.rs:557:30 [INFO] [stdout] 22: core::ops::function::FnOnce::call_once{{vtable.shim}} [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/core/src/ops/function.rs:253:5 [INFO] [stdout] 23: as core::ops::function::FnOnce>::call_once [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/alloc/src/boxed.rs:1985:9 [INFO] [stdout] 24: std::sys::pal::unix::thread::Thread::new::thread_start [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/sys/pal/unix/thread.rs:118:17 [INFO] [stdout] 25: [INFO] [stdout] 26: clone [INFO] [stdout] [INFO] [stdout] ---- datasets::pretrain_dataset_loader::tests::test_process_line_padding_and_truncation stdout ---- [INFO] [stdout] start to load tokenizer from: ./out/tokenizer/tokenizer.json [INFO] [stdout] Error: No such file or directory (os error 2) [INFO] [stdout] [INFO] [stdout] Stack backtrace: [INFO] [stdout] 0: anyhow::error:: for anyhow::Error>::from [INFO] [stdout] at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/anyhow-1.0.98/src/backtrace.rs:27:14 [INFO] [stdout] 1: as core::ops::try_trait::FromResidual>>::from_residual [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/core/src/result.rs:2174:27 [INFO] [stdout] 2: mgpt::datasets::pretrain_dataset_loader::tests::test_process_line_padding_and_truncation [INFO] [stdout] at ./src/datasets/pretrain_dataset_loader.rs:270:25 [INFO] [stdout] 3: mgpt::datasets::pretrain_dataset_loader::tests::test_process_line_padding_and_truncation::{{closure}} [INFO] [stdout] at ./src/datasets/pretrain_dataset_loader.rs:269:54 [INFO] [stdout] 4: core::ops::function::FnOnce::call_once [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/core/src/ops/function.rs:253:5 [INFO] [stdout] 5: core::ops::function::FnOnce::call_once [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/core/src/ops/function.rs:253:5 [INFO] [stdout] 6: test::__rust_begin_short_backtrace [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/test/src/lib.rs:663:18 [INFO] [stdout] 7: test::run_test_in_process::{{closure}} [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/test/src/lib.rs:686:74 [INFO] [stdout] 8: as core::ops::function::FnOnce<()>>::call_once [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/core/src/panic/unwind_safe.rs:272:9 [INFO] [stdout] 9: std::panicking::catch_unwind::do_call [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/panicking.rs:590:40 [INFO] [stdout] 10: std::panicking::catch_unwind [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/panicking.rs:553:19 [INFO] [stdout] 11: std::panic::catch_unwind [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/panic.rs:359:14 [INFO] [stdout] 12: test::run_test_in_process [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/test/src/lib.rs:686:27 [INFO] [stdout] 13: test::run_test::{{closure}} [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/test/src/lib.rs:607:43 [INFO] [stdout] 14: test::run_test::{{closure}} [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/test/src/lib.rs:637:41 [INFO] [stdout] 15: std::sys::backtrace::__rust_begin_short_backtrace [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/sys/backtrace.rs:158:18 [INFO] [stdout] 16: std::thread::Builder::spawn_unchecked_::{{closure}}::{{closure}} [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/thread/mod.rs:559:17 [INFO] [stdout] 17: as core::ops::function::FnOnce<()>>::call_once [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/core/src/panic/unwind_safe.rs:272:9 [INFO] [stdout] 18: std::panicking::catch_unwind::do_call [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/panicking.rs:590:40 [INFO] [stdout] 19: std::panicking::catch_unwind [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/panicking.rs:553:19 [INFO] [stdout] 20: std::panic::catch_unwind [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/panic.rs:359:14 [INFO] [stdout] 21: std::thread::Builder::spawn_unchecked_::{{closure}} [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/thread/mod.rs:557:30 [INFO] [stdout] 22: core::ops::function::FnOnce::call_once{{vtable.shim}} [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/core/src/ops/function.rs:253:5 [INFO] [stdout] 23: as core::ops::function::FnOnce>::call_once [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/alloc/src/boxed.rs:1985:9 [INFO] [stdout] 24: std::sys::pal::unix::thread::Thread::new::thread_start [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/sys/pal/unix/thread.rs:118:17 [INFO] [stdout] 25: [INFO] [stdout] 26: clone [INFO] [stdout] [INFO] [stdout] ---- llm::mgpt::tests::test_llm_generate_for_batch_inputs stdout ---- [INFO] [stdout] start to load tokenizer from: ./out/tokenizer/tokenizer.json [INFO] [stdout] Error: No such file or directory (os error 2) [INFO] [stdout] [INFO] [stdout] Stack backtrace: [INFO] [stdout] 0: anyhow::error:: for anyhow::Error>::from [INFO] [stdout] at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/anyhow-1.0.98/src/backtrace.rs:27:14 [INFO] [stdout] 1: as core::ops::try_trait::FromResidual>>::from_residual [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/core/src/result.rs:2174:27 [INFO] [stdout] 2: mgpt::llm::mgpt::tests::test_llm_generate_for_batch_inputs [INFO] [stdout] at ./src/llm/mgpt.rs:464:25 [INFO] [stdout] 3: mgpt::llm::mgpt::tests::test_llm_generate_for_batch_inputs::{{closure}} [INFO] [stdout] at ./src/llm/mgpt.rs:457:48 [INFO] [stdout] 4: core::ops::function::FnOnce::call_once [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/core/src/ops/function.rs:253:5 [INFO] [stdout] 5: core::ops::function::FnOnce::call_once [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/core/src/ops/function.rs:253:5 [INFO] [stdout] 6: test::__rust_begin_short_backtrace [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/test/src/lib.rs:663:18 [INFO] [stdout] 7: test::run_test_in_process::{{closure}} [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/test/src/lib.rs:686:74 [INFO] [stdout] 8: as core::ops::function::FnOnce<()>>::call_once [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/core/src/panic/unwind_safe.rs:272:9 [INFO] [stdout] 9: std::panicking::catch_unwind::do_call [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/panicking.rs:590:40 [INFO] [stdout] 10: std::panicking::catch_unwind [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/panicking.rs:553:19 [INFO] [stdout] 11: std::panic::catch_unwind [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/panic.rs:359:14 [INFO] [stdout] 12: test::run_test_in_process [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/test/src/lib.rs:686:27 [INFO] [stdout] 13: test::run_test::{{closure}} [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/test/src/lib.rs:607:43 [INFO] [stdout] 14: test::run_test::{{closure}} [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/test/src/lib.rs:637:41 [INFO] [stdout] 15: std::sys::backtrace::__rust_begin_short_backtrace [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/sys/backtrace.rs:158:18 [INFO] [stdout] 16: std::thread::Builder::spawn_unchecked_::{{closure}}::{{closure}} [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/thread/mod.rs:559:17 [INFO] [stdout] 17: as core::ops::function::FnOnce<()>>::call_once [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/core/src/panic/unwind_safe.rs:272:9 [INFO] [stdout] 18: std::panicking::catch_unwind::do_call [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/panicking.rs:590:40 [INFO] [stdout] 19: std::panicking::catch_unwind [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/panicking.rs:553:19 [INFO] [stdout] 20: std::panic::catch_unwind [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/panic.rs:359:14 [INFO] [stdout] 21: std::thread::Builder::spawn_unchecked_::{{closure}} [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/thread/mod.rs:557:30 [INFO] [stdout] 22: core::ops::function::FnOnce::call_once{{vtable.shim}} [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/core/src/ops/function.rs:253:5 [INFO] [stdout] 23: as core::ops::function::FnOnce>::call_once [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/alloc/src/boxed.rs:1985:9 [INFO] [stdout] 24: std::sys::pal::unix::thread::Thread::new::thread_start [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/sys/pal/unix/thread.rs:118:17 [INFO] [stdout] 25: [INFO] [stdout] 26: clone [INFO] [stdout] [INFO] [stdout] ---- llm::mgpt::tests::test_llm_generate_with_tokenizer stdout ---- [INFO] [stdout] start to load tokenizer from: ./out/tokenizer/tokenizer.json [INFO] [stdout] Error: No such file or directory (os error 2) [INFO] [stdout] [INFO] [stdout] Stack backtrace: [INFO] [stdout] 0: anyhow::error:: for anyhow::Error>::from [INFO] [stdout] at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/anyhow-1.0.98/src/backtrace.rs:27:14 [INFO] [stdout] 1: as core::ops::try_trait::FromResidual>>::from_residual [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/core/src/result.rs:2174:27 [INFO] [stdout] 2: mgpt::llm::mgpt::tests::test_llm_generate_with_tokenizer [INFO] [stdout] at ./src/llm/mgpt.rs:423:25 [INFO] [stdout] 3: mgpt::llm::mgpt::tests::test_llm_generate_with_tokenizer::{{closure}} [INFO] [stdout] at ./src/llm/mgpt.rs:416:46 [INFO] [stdout] 4: core::ops::function::FnOnce::call_once [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/core/src/ops/function.rs:253:5 [INFO] [stdout] 5: core::ops::function::FnOnce::call_once [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/core/src/ops/function.rs:253:5 [INFO] [stdout] 6: test::__rust_begin_short_backtrace [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/test/src/lib.rs:663:18 [INFO] [stdout] 7: test::run_test_in_process::{{closure}} [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/test/src/lib.rs:686:74 [INFO] [stdout] 8: as core::ops::function::FnOnce<()>>::call_once [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/core/src/panic/unwind_safe.rs:272:9 [INFO] [stdout] 9: std::panicking::catch_unwind::do_call [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/panicking.rs:590:40 [INFO] [stdout] 10: std::panicking::catch_unwind [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/panicking.rs:553:19 [INFO] [stdout] 11: std::panic::catch_unwind [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/panic.rs:359:14 [INFO] [stdout] 12: test::run_test_in_process [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/test/src/lib.rs:686:27 [INFO] [stdout] 13: test::run_test::{{closure}} [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/test/src/lib.rs:607:43 [INFO] [stdout] 14: test::run_test::{{closure}} [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/test/src/lib.rs:637:41 [INFO] [stdout] 15: std::sys::backtrace::__rust_begin_short_backtrace [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/sys/backtrace.rs:158:18 [INFO] [stdout] 16: std::thread::Builder::spawn_unchecked_::{{closure}}::{{closure}} [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/thread/mod.rs:559:17 [INFO] [stdout] 17: as core::ops::function::FnOnce<()>>::call_once [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/core/src/panic/unwind_safe.rs:272:9 [INFO] [stdout] 18: std::panicking::catch_unwind::do_call [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/panicking.rs:590:40 [INFO] [stdout] 19: std::panicking::catch_unwind [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/panicking.rs:553:19 [INFO] [stdout] 20: std::panic::catch_unwind [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/panic.rs:359:14 [INFO] [stdout] 21: std::thread::Builder::spawn_unchecked_::{{closure}} [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/thread/mod.rs:557:30 [INFO] [stdout] 22: core::ops::function::FnOnce::call_once{{vtable.shim}} [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/core/src/ops/function.rs:253:5 [INFO] [stdout] 23: as core::ops::function::FnOnce>::call_once [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/alloc/src/boxed.rs:1985:9 [INFO] [stdout] 24: std::sys::pal::unix::thread::Thread::new::thread_start [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/sys/pal/unix/thread.rs:118:17 [INFO] [stdout] 25: [INFO] [stdout] 26: clone [INFO] [stdout] [INFO] [stdout] ---- llm::nn_utils::tests::test_draw_loss_series stdout ---- [INFO] [stdout] Error: Read-only file system (os error 30) [INFO] [stdout] [INFO] [stdout] Stack backtrace: [INFO] [stdout] 0: anyhow::error:: for anyhow::Error>::from [INFO] [stdout] at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/anyhow-1.0.98/src/backtrace.rs:27:14 [INFO] [stdout] 1: as core::ops::try_trait::FromResidual>>::from_residual [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/core/src/result.rs:2174:27 [INFO] [stdout] 2: mgpt::llm::nn_utils::tests::test_draw_loss_series [INFO] [stdout] at ./src/llm/nn_utils.rs:297:9 [INFO] [stdout] 3: mgpt::llm::nn_utils::tests::test_draw_loss_series::{{closure}} [INFO] [stdout] at ./src/llm/nn_utils.rs:286:35 [INFO] [stdout] 4: core::ops::function::FnOnce::call_once [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/core/src/ops/function.rs:253:5 [INFO] [stdout] 5: core::ops::function::FnOnce::call_once [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/core/src/ops/function.rs:253:5 [INFO] [stdout] 6: test::__rust_begin_short_backtrace [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/test/src/lib.rs:663:18 [INFO] [stdout] 7: test::run_test_in_process::{{closure}} [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/test/src/lib.rs:686:74 [INFO] [stdout] 8: as core::ops::function::FnOnce<()>>::call_once [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/core/src/panic/unwind_safe.rs:272:9 [INFO] [stdout] 9: std::panicking::catch_unwind::do_call [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/panicking.rs:590:40 [INFO] [stdout] 10: std::panicking::catch_unwind [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/panicking.rs:553:19 [INFO] [stdout] 11: std::panic::catch_unwind [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/panic.rs:359:14 [INFO] [stdout] 12: test::run_test_in_process [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/test/src/lib.rs:686:27 [INFO] [stdout] 13: test::run_test::{{closure}} [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/test/src/lib.rs:607:43 [INFO] [stdout] 14: test::run_test::{{closure}} [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/test/src/lib.rs:637:41 [INFO] [stdout] 15: std::sys::backtrace::__rust_begin_short_backtrace [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/sys/backtrace.rs:158:18 [INFO] [stdout] 16: std::thread::Builder::spawn_unchecked_::{{closure}}::{{closure}} [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/thread/mod.rs:559:17 [INFO] [stdout] 17: as core::ops::function::FnOnce<()>>::call_once [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/core/src/panic/unwind_safe.rs:272:9 [INFO] [stdout] 18: std::panicking::catch_unwind::do_call [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/panicking.rs:590:40 [INFO] [stdout] 19: std::panicking::catch_unwind [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/panicking.rs:553:19 [INFO] [stdout] 20: std::panic::catch_unwind [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/panic.rs:359:14 [INFO] [stdout] 21: std::thread::Builder::spawn_unchecked_::{{closure}} [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/thread/mod.rs:557:30 [INFO] [stdout] 22: core::ops::function::FnOnce::call_once{{vtable.shim}} [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/core/src/ops/function.rs:253:5 [INFO] [stdout] 23: as core::ops::function::FnOnce>::call_once [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/alloc/src/boxed.rs:1985:9 [INFO] [stdout] 24: std::sys::pal::unix::thread::Thread::new::thread_start [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/sys/pal/unix/thread.rs:118:17 [INFO] [stdout] 25: [INFO] [stdout] 26: clone [INFO] [stdout] [INFO] [stdout] ---- llm::pretrain::tests::test_pretrain_llm stdout ---- [INFO] [stdout] [INFO] [stdout] thread 'llm::pretrain::tests::test_pretrain_llm' (43) panicked at src/llm/pretrain.rs:303:80: [INFO] [stdout] called `Result::unwrap()` on an `Err` value: Os { code: 2, kind: NotFound, message: "No such file or directory" } [INFO] [stdout] stack backtrace: [INFO] [stdout] 0: 0x56451b81b522 - std::backtrace_rs::backtrace::libunwind::trace::h4b3f5960ed1c0917 [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/../../backtrace/src/backtrace/libunwind.rs:117:9 [INFO] [stdout] 1: 0x56451b81b522 - std::backtrace_rs::backtrace::trace_unsynchronized::h2f16383c47f8bd04 [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/../../backtrace/src/backtrace/mod.rs:66:14 [INFO] [stdout] 2: 0x56451b81b522 - std::sys::backtrace::_print_fmt::h698d661724c270ea [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/sys/backtrace.rs:66:9 [INFO] [stdout] 3: 0x56451b81b522 - ::fmt::h1c8375013274064c [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/sys/backtrace.rs:39:26 [INFO] [stdout] 4: 0x56451b82f34f - core::fmt::rt::Argument::fmt::h9232377a772d3f8c [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/core/src/fmt/rt.rs:173:76 [INFO] [stdout] 5: 0x56451b82f34f - core::fmt::write::h6dc9e168423d9633 [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/core/src/fmt/mod.rs:1468:25 [INFO] [stdout] 6: 0x56451b7e5ec3 - std::io::default_write_fmt::hec53a38f10739f86 [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/io/mod.rs:639:11 [INFO] [stdout] 7: 0x56451b7e5ec3 - std::io::Write::write_fmt::hf5089d8bbbe5aaa6 [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/io/mod.rs:1954:13 [INFO] [stdout] 8: 0x56451b7f2372 - std::sys::backtrace::BacktraceLock::print::hf83c3d8cbd01f904 [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/sys/backtrace.rs:42:9 [INFO] [stdout] 9: 0x56451b7f7c4f - std::panicking::default_hook::{{closure}}::h3d4c49adc0898362 [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/panicking.rs:301:27 [INFO] [stdout] 10: 0x56451b7f7ae1 - std::panicking::default_hook::hccea28f2487e7471 [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/panicking.rs:325:9 [INFO] [stdout] 11: 0x56451a0be0ce - as core::ops::function::Fn>::call::h06e1b96f84d5792f [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/alloc/src/boxed.rs:1999:9 [INFO] [stdout] 12: 0x56451a0be0ce - test::test_main_with_exit_callback::{{closure}}::hf6db6d7b1242fcd4 [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/test/src/lib.rs:145:21 [INFO] [stdout] 13: 0x56451b7f839e - as core::ops::function::Fn>::call::hfeb094aff1e6591e [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/alloc/src/boxed.rs:1999:9 [INFO] [stdout] 14: 0x56451b7f839e - std::panicking::panic_with_hook::h694b8196629eb5e1 [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/panicking.rs:842:13 [INFO] [stdout] 15: 0x56451b7f80ba - std::panicking::panic_handler::{{closure}}::h0d99dd765f7e6554 [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/panicking.rs:707:13 [INFO] [stdout] 16: 0x56451b7f24b9 - std::sys::backtrace::__rust_end_short_backtrace::h8b43cbde72f7d592 [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/sys/backtrace.rs:174:18 [INFO] [stdout] 17: 0x56451b7d900d - __rustc[d1dec8fc82f9a585]::rust_begin_unwind [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/panicking.rs:698:5 [INFO] [stdout] 18: 0x56451b839b30 - core::panicking::panic_fmt::h315eeb0e4d90fcdc [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/core/src/panicking.rs:75:14 [INFO] [stdout] 19: 0x56451b838b66 - core::result::unwrap_failed::h3188dfca3824df71 [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/core/src/result.rs:1852:5 [INFO] [stdout] 20: 0x564519f5fa04 - core::result::Result::unwrap::he59c8498453a03d6 [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/core/src/result.rs:1223:23 [INFO] [stdout] 21: 0x564519f5fa04 - mgpt::llm::pretrain::tests::test_pretrain_llm::hffb196839c8ac67d [INFO] [stdout] at /opt/rustwide/workdir/src/llm/pretrain.rs:303:80 [INFO] [stdout] 22: 0x564519f61417 - mgpt::llm::pretrain::tests::test_pretrain_llm::{{closure}}::hc5f7dc6833e9eab3 [INFO] [stdout] at /opt/rustwide/workdir/src/llm/pretrain.rs:301:31 [INFO] [stdout] 23: 0x56451a0250c6 - core::ops::function::FnOnce::call_once::h2ee163688c868b50 [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/core/src/ops/function.rs:253:5 [INFO] [stdout] 24: 0x56451a0bdf3b - core::ops::function::FnOnce::call_once::h5852f0076c8c0d8d [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/core/src/ops/function.rs:253:5 [INFO] [stdout] 25: 0x56451a0bdf3b - test::__rust_begin_short_backtrace::hc8457233131a06d0 [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/test/src/lib.rs:663:18 [INFO] [stdout] 26: 0x56451a0d3995 - test::run_test_in_process::{{closure}}::ha3eda53784ae2639 [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/test/src/lib.rs:686:74 [INFO] [stdout] 27: 0x56451a0d3995 - as core::ops::function::FnOnce<()>>::call_once::hbd60f847cd185bc6 [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/core/src/panic/unwind_safe.rs:272:9 [INFO] [stdout] 28: 0x56451a0d3995 - std::panicking::catch_unwind::do_call::h11c1623aed953668 [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/panicking.rs:590:40 [INFO] [stdout] 29: 0x56451a0d3995 - std::panicking::catch_unwind::hf8551f73b52d8ed0 [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/panicking.rs:553:19 [INFO] [stdout] 30: 0x56451a0d3995 - std::panic::catch_unwind::h66ff870d847fc7cf [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/panic.rs:359:14 [INFO] [stdout] 31: 0x56451a0d3995 - test::run_test_in_process::h808a5fa5cb24efde [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/test/src/lib.rs:686:27 [INFO] [stdout] 32: 0x56451a0d3995 - test::run_test::{{closure}}::hcf5782fef76f68bf [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/test/src/lib.rs:607:43 [INFO] [stdout] 33: 0x56451a0aa364 - test::run_test::{{closure}}::h59bb1f8f7167fee7 [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/test/src/lib.rs:637:41 [INFO] [stdout] 34: 0x56451a0aa364 - std::sys::backtrace::__rust_begin_short_backtrace::h073f551083d60480 [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/sys/backtrace.rs:158:18 [INFO] [stdout] 35: 0x56451a0adc4a - std::thread::Builder::spawn_unchecked_::{{closure}}::{{closure}}::h9d6eff1e4da9d0f2 [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/thread/mod.rs:559:17 [INFO] [stdout] 36: 0x56451a0adc4a - as core::ops::function::FnOnce<()>>::call_once::he37f75191b40d4b0 [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/core/src/panic/unwind_safe.rs:272:9 [INFO] [stdout] 37: 0x56451a0adc4a - std::panicking::catch_unwind::do_call::h339b26991b458079 [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/panicking.rs:590:40 [INFO] [stdout] 38: 0x56451a0adc4a - std::panicking::catch_unwind::h2e5906b8a84cca89 [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/panicking.rs:553:19 [INFO] [stdout] 39: 0x56451a0adc4a - std::panic::catch_unwind::h3fea03a95c906d68 [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/panic.rs:359:14 [INFO] [stdout] 40: 0x56451a0adc4a - std::thread::Builder::spawn_unchecked_::{{closure}}::hab1ac593ae8241c8 [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/thread/mod.rs:557:30 [INFO] [stdout] 41: 0x56451a0adc4a - core::ops::function::FnOnce::call_once{{vtable.shim}}::hb46ae6da79faf2c5 [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/core/src/ops/function.rs:253:5 [INFO] [stdout] 42: 0x56451b7ec42f - as core::ops::function::FnOnce>::call_once::h317217a5c86437d1 [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/alloc/src/boxed.rs:1985:9 [INFO] [stdout] 43: 0x56451b7ec42f - std::sys::pal::unix::thread::Thread::new::thread_start::h9b0e1220f8cf5182 [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/sys/pal/unix/thread.rs:118:17 [INFO] [stdout] 44: 0x7299d8f4caa4 - [INFO] [stdout] 45: 0x7299d8fd9a34 - clone [INFO] [stdout] 46: 0x0 - [INFO] [stdout] [INFO] [stdout] ---- tokenizer::bert_tokenizer::tests::test_custom_tokenizer stdout ---- [INFO] [stdout] Error: Read-only file system (os error 30) [INFO] [stdout] [INFO] [stdout] Stack backtrace: [INFO] [stdout] 0: anyhow::error:: for anyhow::Error>::from [INFO] [stdout] at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/anyhow-1.0.98/src/backtrace.rs:27:14 [INFO] [stdout] 1: as core::ops::try_trait::FromResidual>>::from_residual [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/core/src/result.rs:2174:27 [INFO] [stdout] 2: mgpt::tokenizer::bert_tokenizer::tests::test_custom_tokenizer [INFO] [stdout] at ./src/tokenizer/bert_tokenizer.rs:57:9 [INFO] [stdout] 3: mgpt::tokenizer::bert_tokenizer::tests::test_custom_tokenizer::{{closure}} [INFO] [stdout] at ./src/tokenizer/bert_tokenizer.rs:53:35 [INFO] [stdout] 4: core::ops::function::FnOnce::call_once [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/core/src/ops/function.rs:253:5 [INFO] [stdout] 5: core::ops::function::FnOnce::call_once [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/core/src/ops/function.rs:253:5 [INFO] [stdout] 6: test::__rust_begin_short_backtrace [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/test/src/lib.rs:663:18 [INFO] [stdout] 7: test::run_test_in_process::{{closure}} [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/test/src/lib.rs:686:74 [INFO] [stdout] 8: as core::ops::function::FnOnce<()>>::call_once [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/core/src/panic/unwind_safe.rs:272:9 [INFO] [stdout] 9: std::panicking::catch_unwind::do_call [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/panicking.rs:590:40 [INFO] [stdout] 10: std::panicking::catch_unwind [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/panicking.rs:553:19 [INFO] [stdout] 11: std::panic::catch_unwind [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/panic.rs:359:14 [INFO] [stdout] 12: test::run_test_in_process [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/test/src/lib.rs:686:27 [INFO] [stdout] 13: test::run_test::{{closure}} [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/test/src/lib.rs:607:43 [INFO] [stdout] 14: test::run_test::{{closure}} [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/test/src/lib.rs:637:41 [INFO] [stdout] 15: std::sys::backtrace::__rust_begin_short_backtrace [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/sys/backtrace.rs:158:18 [INFO] [stdout] 16: std::thread::Builder::spawn_unchecked_::{{closure}}::{{closure}} [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/thread/mod.rs:559:17 [INFO] [stdout] 17: as core::ops::function::FnOnce<()>>::call_once [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/core/src/panic/unwind_safe.rs:272:9 [INFO] [stdout] 18: std::panicking::catch_unwind::do_call [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/panicking.rs:590:40 [INFO] [stdout] 19: std::panicking::catch_unwind [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/panicking.rs:553:19 [INFO] [stdout] 20: std::panic::catch_unwind [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/panic.rs:359:14 [INFO] [stdout] 21: std::thread::Builder::spawn_unchecked_::{{closure}} [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/thread/mod.rs:557:30 [INFO] [stdout] 22: core::ops::function::FnOnce::call_once{{vtable.shim}} [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/core/src/ops/function.rs:253:5 [INFO] [stdout] 23: as core::ops::function::FnOnce>::call_once [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/alloc/src/boxed.rs:1985:9 [INFO] [stdout] 24: std::sys::pal::unix::thread::Thread::new::thread_start [INFO] [stdout] at /rustc/16fbf98f22730e073c09e7b03f0eafb87a295545/library/std/src/sys/pal/unix/thread.rs:118:17 [INFO] [stdout] 25: [INFO] [stdout] 26: clone [INFO] [stdout] [INFO] [stdout] [INFO] [stdout] failures: [INFO] [stdout] datasets::pretrain_dataset_loader::tests::test_iter_overall_loader [INFO] [stdout] datasets::pretrain_dataset_loader::tests::test_load_tokenizer [INFO] [stdout] datasets::pretrain_dataset_loader::tests::test_make_result_empty_input [INFO] [stdout] datasets::pretrain_dataset_loader::tests::test_make_result_valid_input [INFO] [stdout] datasets::pretrain_dataset_loader::tests::test_process_line_padding_and_truncation [INFO] [stdout] datasets::pretrain_dataset_loader::tests::test_prtrain_dataset_loader [INFO] [stdout] llm::inference::tests::test_batch_inputs_with_padding [INFO] [stdout] llm::inference::tests::test_inference [INFO] [stdout] llm::inference::tests::test_model_params [INFO] [stdout] llm::mgpt::tests::test_llm_generate_for_batch_inputs [INFO] [stdout] llm::mgpt::tests::test_llm_generate_with_tokenizer [INFO] [stdout] llm::nn_utils::tests::test_draw_loss_series [INFO] [stdout] llm::pretrain::tests::test_pretrain_llm [INFO] [stdout] tokenizer::bert_tokenizer::tests::test_custom_tokenizer [INFO] [stdout] [INFO] [stdout] test result: FAILED. 24 passed; 14 failed; 0 ignored; 0 measured; 0 filtered out; finished in 1.74s [INFO] [stdout] [INFO] [stderr] error: test failed, to rerun pass `--lib` [INFO] running `Command { std: "docker" "inspect" "d53a5d915d7f3036aa230a7c1c37b856621811a5d2947a1efc015b7178e557b6", kill_on_drop: false }` [INFO] running `Command { std: "docker" "rm" "-f" "d53a5d915d7f3036aa230a7c1c37b856621811a5d2947a1efc015b7178e557b6", kill_on_drop: false }` [INFO] [stdout] d53a5d915d7f3036aa230a7c1c37b856621811a5d2947a1efc015b7178e557b6