[INFO] fetching crate ai-workbench-lib 0.4.0...
[INFO] testing ai-workbench-lib-0.4.0 against try#c2e32f1c9652b13ed99608599c1e855462f421f3 for pr-146098-7
[INFO] extracting crate ai-workbench-lib 0.4.0 into /workspace/builds/worker-1-tc2/source
[INFO] started tweaking crates.io crate ai-workbench-lib 0.4.0
[INFO] finished tweaking crates.io crate ai-workbench-lib 0.4.0
[INFO] tweaked toml for crates.io crate ai-workbench-lib 0.4.0 written to /workspace/builds/worker-1-tc2/source/Cargo.toml
[INFO] validating manifest of crates.io crate ai-workbench-lib 0.4.0 on toolchain c2e32f1c9652b13ed99608599c1e855462f421f3
[INFO] running `Command { std: CARGO_HOME="/workspace/cargo-home" RUSTUP_HOME="/workspace/rustup-home" "/workspace/cargo-home/bin/cargo" "+c2e32f1c9652b13ed99608599c1e855462f421f3" "metadata" "--manifest-path" "Cargo.toml" "--no-deps", kill_on_drop: false }`
[INFO] crate crates.io crate ai-workbench-lib 0.4.0 already has a lockfile, it will not be regenerated
[INFO] running `Command { std: CARGO_HOME="/workspace/cargo-home" RUSTUP_HOME="/workspace/rustup-home" "/workspace/cargo-home/bin/cargo" "+c2e32f1c9652b13ed99608599c1e855462f421f3" "fetch" "--manifest-path" "Cargo.toml", kill_on_drop: false }`
[INFO] running `Command { std: "docker" "create" "-v" "/var/lib/crater-agent-workspace/builds/worker-1-tc2/target:/opt/rustwide/target:rw,Z" "-v" "/var/lib/crater-agent-workspace/builds/worker-1-tc2/source:/opt/rustwide/workdir:ro,Z" "-v" "/var/lib/crater-agent-workspace/cargo-home:/opt/rustwide/cargo-home:ro,Z" "-v" "/var/lib/crater-agent-workspace/rustup-home:/opt/rustwide/rustup-home:ro,Z" "-e" "SOURCE_DIR=/opt/rustwide/workdir" "-e" "CARGO_TARGET_DIR=/opt/rustwide/target" "-e" "CARGO_HOME=/opt/rustwide/cargo-home" "-e" "RUSTUP_HOME=/opt/rustwide/rustup-home" "-w" "/opt/rustwide/workdir" "-m" "1610612736" "--user" "0:0" "--network" "none" "ghcr.io/rust-lang/crates-build-env/linux@sha256:4848fb76d95f26979359cc7e45710b1dbc8f3acb7aeedee7c460d7702230f228" "/opt/rustwide/cargo-home/bin/cargo" "+c2e32f1c9652b13ed99608599c1e855462f421f3" "metadata" "--no-deps" "--format-version=1", kill_on_drop: false }`
[INFO] [stdout] 0694f88b3f0f1f4edd119c016964a3bff7349672c5a2ca99a0d936fdf246f5f5
[INFO] running `Command { std: "docker" "start" "-a" "0694f88b3f0f1f4edd119c016964a3bff7349672c5a2ca99a0d936fdf246f5f5", kill_on_drop: false }`
[INFO] running `Command { std: "docker" "inspect" "0694f88b3f0f1f4edd119c016964a3bff7349672c5a2ca99a0d936fdf246f5f5", kill_on_drop: false }`
[INFO] running `Command { std: "docker" "rm" "-f" "0694f88b3f0f1f4edd119c016964a3bff7349672c5a2ca99a0d936fdf246f5f5", kill_on_drop: false }`
[INFO] [stdout] 0694f88b3f0f1f4edd119c016964a3bff7349672c5a2ca99a0d936fdf246f5f5
[INFO] running `Command { std: "docker" "create" "-v" "/var/lib/crater-agent-workspace/builds/worker-1-tc2/target:/opt/rustwide/target:rw,Z" "-v" "/var/lib/crater-agent-workspace/builds/worker-1-tc2/source:/opt/rustwide/workdir:ro,Z" "-v" "/var/lib/crater-agent-workspace/cargo-home:/opt/rustwide/cargo-home:ro,Z" "-v" "/var/lib/crater-agent-workspace/rustup-home:/opt/rustwide/rustup-home:ro,Z" "-e" "SOURCE_DIR=/opt/rustwide/workdir" "-e" "CARGO_TARGET_DIR=/opt/rustwide/target" "-e" "CARGO_INCREMENTAL=0" "-e" "RUST_BACKTRACE=full" "-e" "RUSTFLAGS=--cap-lints=forbid" "-e" "RUSTDOCFLAGS=--cap-lints=forbid" "-e" "CARGO_HOME=/opt/rustwide/cargo-home" "-e" "RUSTUP_HOME=/opt/rustwide/rustup-home" "-w" "/opt/rustwide/workdir" "-m" "1610612736" "--user" "0:0" "--network" "none" "ghcr.io/rust-lang/crates-build-env/linux@sha256:4848fb76d95f26979359cc7e45710b1dbc8f3acb7aeedee7c460d7702230f228" "/opt/rustwide/cargo-home/bin/cargo" "+c2e32f1c9652b13ed99608599c1e855462f421f3" "build" "--frozen" "--message-format=json", kill_on_drop: false }`
[INFO] [stdout] 093af6e2abf98aa26a948995893a37b3840dd687baef24544d482dadbab37a3f
[INFO] running `Command { std: "docker" "start" "-a" "093af6e2abf98aa26a948995893a37b3840dd687baef24544d482dadbab37a3f", kill_on_drop: false }`
[INFO] [stderr]    Compiling libc v0.2.175
[INFO] [stderr]    Compiling proc-macro2 v1.0.101
[INFO] [stderr]    Compiling cfg-if v1.0.3
[INFO] [stderr]    Compiling smallvec v1.15.1
[INFO] [stderr]    Compiling parking_lot_core v0.9.11
[INFO] [stderr]    Compiling memchr v2.7.5
[INFO] [stderr]    Compiling lock_api v0.4.13
[INFO] [stderr]    Compiling find-msvc-tools v0.1.0
[INFO] [stderr]    Compiling slab v0.4.11
[INFO] [stderr]    Compiling tracing-core v0.1.34
[INFO] [stderr]    Compiling http v0.2.12
[INFO] [stderr]    Compiling num-traits v0.2.19
[INFO] [stderr]    Compiling deranged v0.5.3
[INFO] [stderr]    Compiling vsimd v0.8.0
[INFO] [stderr]    Compiling time-core v0.1.5
[INFO] [stderr]    Compiling outref v0.5.2
[INFO] [stderr]    Compiling either v1.15.0
[INFO] [stderr]    Compiling block-buffer v0.10.4
[INFO] [stderr]    Compiling dunce v1.0.5
[INFO] [stderr]    Compiling bytes-utils v0.1.4
[INFO] [stderr]    Compiling foldhash v0.1.5
[INFO] [stderr]    Compiling fs_extra v1.3.0
[INFO] [stderr]    Compiling aws-lc-rs v1.13.3
[INFO] [stderr]    Compiling percent-encoding v2.3.2
[INFO] [stderr]    Compiling hashbrown v0.15.5
[INFO] [stderr]    Compiling getrandom v0.3.3
[INFO] [stderr]    Compiling digest v0.10.7
[INFO] [stderr]    Compiling der v0.6.1
[INFO] [stderr]    Compiling rustls-pki-types v1.12.0
[INFO] [stderr]    Compiling base64-simd v0.8.0
[INFO] [stderr]    Compiling base64ct v1.8.0
[INFO] [stderr]    Compiling crc32fast v1.5.0
[INFO] [stderr]    Compiling semver v1.0.26
[INFO] [stderr]    Compiling rustls v0.21.12
[INFO] [stderr]    Compiling num-integer v0.1.46
[INFO] [stderr]    Compiling quote v1.0.40
[INFO] [stderr]    Compiling mio v1.0.4
[INFO] [stderr]    Compiling socket2 v0.6.0
[INFO] [stderr]    Compiling signal-hook-registry v1.4.6
[INFO] [stderr]    Compiling jobserver v0.1.34
[INFO] [stderr]    Compiling getrandom v0.2.16
[INFO] [stderr]    Compiling parking_lot v0.12.4
[INFO] [stderr]    Compiling syn v2.0.106
[INFO] [stderr]    Compiling rand_core v0.6.4
[INFO] [stderr]    Compiling cc v1.2.35
[INFO] [stderr]    Compiling rustls v0.23.31
[INFO] [stderr]    Compiling http-body v0.4.6
[INFO] [stderr]    Compiling indexmap v2.11.0
[INFO] [stderr]    Compiling base16ct v0.1.1
[INFO] [stderr]    Compiling ff v0.12.1
[INFO] [stderr]    Compiling crypto-bigint v0.4.9
[INFO] [stderr]    Compiling spki v0.6.0
[INFO] [stderr]    Compiling socket2 v0.5.10
[INFO] [stderr]    Compiling group v0.12.1
[INFO] [stderr]    Compiling hmac v0.12.1
[INFO] [stderr]    Compiling rustls-native-certs v0.6.3
[INFO] [stderr]    Compiling sha2 v0.10.9
[INFO] [stderr]    Compiling signature v1.6.4
[INFO] [stderr]    Compiling rustls-native-certs v0.8.1
[INFO] [stderr]    Compiling tower v0.5.2
[INFO] [stderr]    Compiling pkcs8 v0.9.0
[INFO] [stderr]    Compiling rustc_version v0.4.1
[INFO] [stderr]    Compiling sec1 v0.3.0
[INFO] [stderr]    Compiling cmake v0.1.54
[INFO] [stderr]    Compiling form_urlencoded v1.2.2
[INFO] [stderr]    Compiling writeable v0.6.1
[INFO] [stderr]    Compiling litemap v0.8.0
[INFO] [stderr]    Compiling crypto-bigint v0.5.5
[INFO] [stderr]    Compiling icu_properties_data v2.0.1
[INFO] [stderr]    Compiling icu_normalizer_data v2.0.0
[INFO] [stderr]    Compiling zerocopy v0.8.26
[INFO] [stderr]    Compiling uuid v1.18.1
[INFO] [stderr]    Compiling regex-lite v0.1.7
[INFO] [stderr]    Compiling rand_core v0.9.3
[INFO] [stderr]    Compiling aho-corasick v1.1.3
[INFO] [stderr]    Compiling regex-syntax v0.8.6
[INFO] [stderr]    Compiling aws-types v1.3.8
[INFO] [stderr]    Compiling xmlparser v0.13.6
[INFO] [stderr]    Compiling elliptic-curve v0.12.3
[INFO] [stderr]    Compiling rfc6979 v0.3.1
[INFO] [stderr]    Compiling crc v3.3.0
[INFO] [stderr]    Compiling urlencoding v2.1.3
[INFO] [stderr]    Compiling sha1 v0.10.6
[INFO] [stderr]    Compiling md-5 v0.10.6
[INFO] [stderr]    Compiling anyhow v1.0.99
[INFO] [stderr]    Compiling serde_json v1.0.143
[INFO] [stderr]    Compiling aws-smithy-xml v0.60.10
[INFO] [stderr]    Compiling ring v0.17.14
[INFO] [stderr]    Compiling aws-lc-sys v0.30.0
[INFO] [stderr]    Compiling lru v0.12.5
[INFO] [stderr]    Compiling tracing-log v0.2.0
[INFO] [stderr]    Compiling csv-core v0.1.12
[INFO] [stderr]    Compiling time v0.3.42
[INFO] [stderr]    Compiling thread_local v1.1.9
[INFO] [stderr]    Compiling nu-ansi-term v0.50.1
[INFO] [stderr]    Compiling ecdsa v0.14.8
[INFO] [stderr]    Compiling unicode-id v0.3.5
[INFO] [stderr]    Compiling markdown v1.0.0
[INFO] [stderr]    Compiling tracing-subscriber v0.3.20
[INFO] [stderr]    Compiling p256 v0.11.1
[INFO] [stderr]    Compiling regex-automata v0.4.10
[INFO] [stderr]    Compiling ppv-lite86 v0.2.21
[INFO] [stderr]    Compiling rand_chacha v0.9.0
[INFO] [stderr]    Compiling rand v0.9.2
[INFO] [stderr]    Compiling synstructure v0.13.2
[INFO] [stderr]    Compiling rustls-webpki v0.101.7
[INFO] [stderr]    Compiling sct v0.7.1
[INFO] [stderr]    Compiling tokio-macros v2.5.0
[INFO] [stderr]    Compiling futures-macro v0.3.31
[INFO] [stderr]    Compiling tracing-attributes v0.1.30
[INFO] [stderr]    Compiling zerofrom-derive v0.1.6
[INFO] [stderr]    Compiling yoke-derive v0.8.0
[INFO] [stderr]    Compiling zerovec-derive v0.11.1
[INFO] [stderr]    Compiling displaydoc v0.2.5
[INFO] [stderr]    Compiling tokio v1.47.1
[INFO] [stderr]    Compiling serde_derive v1.0.219
[INFO] [stderr]    Compiling regex v1.11.2
[INFO] [stderr]    Compiling futures-util v0.3.31
[INFO] [stderr]    Compiling zerofrom v0.1.6
[INFO] [stderr]    Compiling yoke v0.8.0
[INFO] [stderr]    Compiling tracing v0.1.41
[INFO] [stderr]    Compiling zerotrie v0.2.2
[INFO] [stderr]    Compiling zerovec v0.11.4
[INFO] [stderr]    Compiling crc-fast v1.3.0
[INFO] [stderr]    Compiling tinystr v0.8.1
[INFO] [stderr]    Compiling potential_utf v0.1.3
[INFO] [stderr]    Compiling serde v1.0.219
[INFO] [stderr]    Compiling icu_collections v2.0.0
[INFO] [stderr]    Compiling icu_locale_core v2.0.0
[INFO] [stderr]    Compiling icu_provider v2.0.0
[INFO] [stderr]    Compiling futures-executor v0.3.31
[INFO] [stderr]    Compiling futures v0.3.31
[INFO] [stderr]    Compiling icu_properties v2.0.1
[INFO] [stderr]    Compiling icu_normalizer v2.0.0
[INFO] [stderr]    Compiling idna_adapter v1.2.1
[INFO] [stderr]    Compiling idna v1.1.0
[INFO] [stderr]    Compiling rustls-webpki v0.103.4
[INFO] [stderr]    Compiling url v2.5.7
[INFO] [stderr]    Compiling csv v1.3.1
[INFO] [stderr]    Compiling tokio-util v0.7.16
[INFO] [stderr]    Compiling aws-smithy-async v1.2.5
[INFO] [stderr]    Compiling tokio-rustls v0.24.1
[INFO] [stderr]    Compiling aws-smithy-types v1.3.2
[INFO] [stderr]    Compiling h2 v0.4.12
[INFO] [stderr]    Compiling h2 v0.3.27
[INFO] [stderr]    Compiling aws-smithy-runtime-api v1.9.0
[INFO] [stderr]    Compiling aws-smithy-eventstream v0.60.11
[INFO] [stderr]    Compiling aws-smithy-json v0.61.5
[INFO] [stderr]    Compiling aws-smithy-query v0.60.7
[INFO] [stderr]    Compiling aws-smithy-http v0.62.3
[INFO] [stderr]    Compiling aws-credential-types v1.2.6
[INFO] [stderr]    Compiling aws-smithy-observability v0.1.3
[INFO] [stderr]    Compiling aws-sigv4 v1.3.4
[INFO] [stderr]    Compiling aws-smithy-checksums v0.63.8
[INFO] [stderr]    Compiling hyper v1.7.0
[INFO] [stderr]    Compiling hyper v0.14.32
[INFO] [stderr]    Compiling hyper-util v0.1.16
[INFO] [stderr]    Compiling tokio-rustls v0.26.2
[INFO] [stderr]    Compiling hyper-rustls v0.27.7
[INFO] [stderr]    Compiling hyper-rustls v0.24.2
[INFO] [stderr]    Compiling aws-smithy-http-client v1.1.1
[INFO] [stderr]    Compiling aws-smithy-runtime v1.9.1
[INFO] [stderr]    Compiling aws-runtime v1.5.10
[INFO] [stderr]    Compiling aws-sdk-sso v1.83.0
[INFO] [stderr]    Compiling aws-sdk-ssooidc v1.84.0
[INFO] [stderr]    Compiling aws-sdk-sts v1.85.0
[INFO] [stderr]    Compiling aws-sdk-bedrockruntime v1.105.0
[INFO] [stderr]    Compiling aws-sdk-s3 v1.104.0
[INFO] [stderr]    Compiling aws-config v1.8.6
[INFO] [stderr]    Compiling ai-workbench-lib v0.4.0 (/opt/rustwide/workdir)
[INFO] [stderr]     Finished `dev` profile [unoptimized + debuginfo] target(s) in 2m 28s
[INFO] running `Command { std: "docker" "inspect" "093af6e2abf98aa26a948995893a37b3840dd687baef24544d482dadbab37a3f", kill_on_drop: false }`
[INFO] running `Command { std: "docker" "rm" "-f" "093af6e2abf98aa26a948995893a37b3840dd687baef24544d482dadbab37a3f", kill_on_drop: false }`
[INFO] [stdout] 093af6e2abf98aa26a948995893a37b3840dd687baef24544d482dadbab37a3f
[INFO] running `Command { std: "docker" "create" "-v" "/var/lib/crater-agent-workspace/builds/worker-1-tc2/target:/opt/rustwide/target:rw,Z" "-v" "/var/lib/crater-agent-workspace/builds/worker-1-tc2/source:/opt/rustwide/workdir:ro,Z" "-v" "/var/lib/crater-agent-workspace/cargo-home:/opt/rustwide/cargo-home:ro,Z" "-v" "/var/lib/crater-agent-workspace/rustup-home:/opt/rustwide/rustup-home:ro,Z" "-e" "SOURCE_DIR=/opt/rustwide/workdir" "-e" "CARGO_TARGET_DIR=/opt/rustwide/target" "-e" "CARGO_INCREMENTAL=0" "-e" "RUST_BACKTRACE=full" "-e" "RUSTFLAGS=--cap-lints=forbid" "-e" "RUSTDOCFLAGS=--cap-lints=forbid" "-e" "CARGO_HOME=/opt/rustwide/cargo-home" "-e" "RUSTUP_HOME=/opt/rustwide/rustup-home" "-w" "/opt/rustwide/workdir" "-m" "1610612736" "--user" "0:0" "--network" "none" "ghcr.io/rust-lang/crates-build-env/linux@sha256:4848fb76d95f26979359cc7e45710b1dbc8f3acb7aeedee7c460d7702230f228" "/opt/rustwide/cargo-home/bin/cargo" "+c2e32f1c9652b13ed99608599c1e855462f421f3" "test" "--frozen" "--no-run" "--message-format=json", kill_on_drop: false }`
[INFO] [stdout] 91ab1b0a424479a04d72f460f2860ad17a5bb8103048a2597a77479ed7a77b70
[INFO] running `Command { std: "docker" "start" "-a" "91ab1b0a424479a04d72f460f2860ad17a5bb8103048a2597a77479ed7a77b70", kill_on_drop: false }`
[INFO] [stderr]    Compiling ai-workbench-lib v0.4.0 (/opt/rustwide/workdir)
[INFO] [stderr]     Finished `test` profile [unoptimized + debuginfo] target(s) in 8.01s
[INFO] running `Command { std: "docker" "inspect" "91ab1b0a424479a04d72f460f2860ad17a5bb8103048a2597a77479ed7a77b70", kill_on_drop: false }`
[INFO] running `Command { std: "docker" "rm" "-f" "91ab1b0a424479a04d72f460f2860ad17a5bb8103048a2597a77479ed7a77b70", kill_on_drop: false }`
[INFO] [stdout] 91ab1b0a424479a04d72f460f2860ad17a5bb8103048a2597a77479ed7a77b70
[INFO] running `Command { std: "docker" "create" "-v" "/var/lib/crater-agent-workspace/builds/worker-1-tc2/target:/opt/rustwide/target:rw,Z" "-v" "/var/lib/crater-agent-workspace/builds/worker-1-tc2/source:/opt/rustwide/workdir:ro,Z" "-v" "/var/lib/crater-agent-workspace/cargo-home:/opt/rustwide/cargo-home:ro,Z" "-v" "/var/lib/crater-agent-workspace/rustup-home:/opt/rustwide/rustup-home:ro,Z" "-e" "SOURCE_DIR=/opt/rustwide/workdir" "-e" "CARGO_TARGET_DIR=/opt/rustwide/target" "-e" "CARGO_INCREMENTAL=0" "-e" "RUST_BACKTRACE=full" "-e" "RUSTFLAGS=--cap-lints=forbid" "-e" "RUSTDOCFLAGS=--cap-lints=forbid" "-e" "CARGO_HOME=/opt/rustwide/cargo-home" "-e" "RUSTUP_HOME=/opt/rustwide/rustup-home" "-w" "/opt/rustwide/workdir" "-m" "1610612736" "--user" "0:0" "--network" "none" "ghcr.io/rust-lang/crates-build-env/linux@sha256:4848fb76d95f26979359cc7e45710b1dbc8f3acb7aeedee7c460d7702230f228" "/opt/rustwide/cargo-home/bin/cargo" "+c2e32f1c9652b13ed99608599c1e855462f421f3" "test" "--frozen", kill_on_drop: false }`
[INFO] [stdout] 295690670bb3ced118ab22ee038d27aff5e5d03ba003da87bfa33a012aa34748
[INFO] running `Command { std: "docker" "start" "-a" "295690670bb3ced118ab22ee038d27aff5e5d03ba003da87bfa33a012aa34748", kill_on_drop: false }`
[INFO] [stderr]     Finished `test` profile [unoptimized + debuginfo] target(s) in 0.28s
[INFO] [stderr]      Running unittests src/lib.rs (/opt/rustwide/target/debug/deps/ai_workbench_lib-ffeccc978854461e)
[INFO] [stdout] 
[INFO] [stdout] running 37 tests
[INFO] [stdout] test modules::file_discovery::file_discovery_tests::test_file_info_structure ... ok
[INFO] [stdout] test modules::file_splitter::splitter::tests::test_empty_file ... ok
[INFO] [stdout] test modules::file_splitter::splitter::tests::test_file_type_detection_by_extension ... ok
[INFO] [stdout] test modules::file_splitter::splitter::tests::test_invalid_csv_fallback ... ok
[INFO] [stdout] test modules::file_splitter::splitter::tests::test_json_file_splitting ... ok
[INFO] [stdout] test modules::file_splitter::splitter_types::code_splitter::tests::test_simple_code_splitting ... ok
[INFO] [stdout] test modules::file_splitter::splitter::integration_tests::test_large_csv_file_splitting ... ok
[INFO] [stdout] test modules::file_splitter::splitter_types::csv_splitter::tests::test_record_boundary_preservation ... ok
[INFO] [stdout] test modules::file_splitter::splitter_types::csv_splitter::tests::test_csv_format_validation ... ok
[INFO] [stdout] test modules::file_splitter::splitter_types::csv_splitter::tests::test_csv_single_chunk ... ok
[INFO] [stdout] test modules::file_splitter::splitter_types::csv_splitter::tests::test_csv_multiple_chunks_with_headers ... ok
[INFO] [stdout] test modules::file_splitter::splitter_types::csv_splitter::tests::test_tsv_format ... ok
[INFO] [stdout] test modules::file_splitter::splitter_types::json_splitter::tests::test_json_lines_splitting ... ok
[INFO] [stdout] test modules::file_splitter::splitter_types::code_splitter::tests::test_large_file_splitting ... ok
[INFO] [stdout] test modules::file_splitter::splitter_types::text_splitter::tests::test_large_file_multiple_chunks ... ok
[INFO] [stdout] test modules::file_splitter::splitter_types::text_splitter::tests::test_line_boundary_preservation ... ok
[INFO] [stdout] test modules::file_splitter::splitter_types::json_splitter::tests::test_json_lines_detection ... ok
[INFO] [stdout] test modules::file_splitter::splitter_types::text_splitter::tests::test_small_file_single_chunk ... ok
[INFO] [stdout] test modules::file_splitter::splitter::integration_tests::test_large_text_file_splitting ... ok
[INFO] [stdout] test modules::file_splitter::splitter::integration_tests::test_mixed_file_types ... FAILED
[INFO] [stdout] test modules::file_splitter::splitter::tests::test_binary_file_rejection ... FAILED
[INFO] [stdout] test modules::file_splitter::splitter_types::json_splitter::tests::test_json_array_detection ... FAILED
[INFO] [stdout] test modules::file_splitter::splitter_types::json_splitter::tests::test_standard_json_detection ... FAILED
[INFO] [stdout] test modules::file_splitter::splitter_types::json_splitter::tests::test_invalid_json_rejection ... FAILED
[INFO] [stdout] test modules::file_splitter::splitter::integration_tests::test_edge_cases ... FAILED
[INFO] [stdout] test modules::file_splitter::splitter::integration_tests::test_performance_characteristics ... FAILED
[INFO] [stdout] test modules::file_discovery::file_discovery_tests::test_file_filtering ... ok
[INFO] [stdout] test modules::file_discovery::file_discovery_tests::test_processing_summary ... ok
[INFO] [stdout] test modules::job_processor::core::job_processor_tests::test_job_processor_with_large_file ... ok
[INFO] [stdout] [2m2025-11-11T07:22:52.360208Z[0m [31mERROR[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m discover_files failed: Failed to list objects in S3
[INFO] [stdout] [2m2025-11-11T07:22:52.373153Z[0m [31mERROR[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m Failed to get metadata for file 'test-data/sample.txt': Failed to get metadata for test-data/sample.txt
[INFO] [stdout] [2m2025-11-11T07:22:52.373215Z[0m [31mERROR[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m This usually means:
[INFO] [stdout] [2m2025-11-11T07:22:52.373226Z[0m [31mERROR[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m   1. File does not exist at s3://ai-workbench-6c9c43db-7fe6-42f1-8b11-8f82323f83f0-eu-west-2/test-data/sample.txt
[INFO] [stdout] [2m2025-11-11T07:22:52.373239Z[0m [31mERROR[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m   2. No permissions to access the file
[INFO] [stdout] [2m2025-11-11T07:22:52.373249Z[0m [31mERROR[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m   3. S3 service is unavailable
[INFO] [stdout] [2m2025-11-11T07:22:52.373256Z[0m [31mERROR[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m   4. Incorrect bucket name or file path
[INFO] [stdout] [2m2025-11-11T07:22:52.373269Z[0m [31mERROR[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m discover_files failed: Failed to get metadata for single file: test-data/sample.txt
[INFO] [stdout] [2m2025-11-11T07:22:52.360260Z[0m [31mERROR[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m Error details: Failed to list objects in S3
[INFO] [stdout] 
[INFO] [stdout] Caused by:
[INFO] [stdout]     S3 ListObjectsV2 failed for folder 'text_files/': dispatch failure
[INFO] [stdout] 
[INFO] [stdout] Stack backtrace:
[INFO] [stdout]    0: anyhow::error::<impl anyhow::Error>::msg
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/anyhow-1.0.99/src/backtrace.rs:27:14
[INFO] [stdout]    1: ai_workbench_lib::modules::file_discovery::FileDiscovery::process_folder::{{closure}}
[INFO] [stdout]              at ./src/../modules/file_discovery.rs:157:32
[INFO] [stdout]    2: ai_workbench_lib::modules::file_discovery::FileDiscovery::discover_files::{{closure}}
[INFO] [stdout]              at ./src/../modules/file_discovery.rs:31:45
[INFO] [stdout]    3: ai_workbench_lib::modules::job_processor::core::JobProcessor::discover_files::{{closure}}
[INFO] [stdout]              at ./src/../modules/job_processor/core.rs:102:83
[INFO] [stdout]    4: ai_workbench_lib::modules::job_processor::core::JobProcessor::run::{{closure}}
[INFO] [stdout]              at ./src/../modules/job_processor/core.rs:57:43
[INFO] [stdout]    5: ai_workbench_lib::modules::job_processor::core::job_processor_tests::test_job_processor_folder_end_to_end::{{closure}}
[INFO] [stdout]              at ./src/../modules/job_processor/core.rs:787:31
[INFO] [stdout]    6: <core::pin::Pin<P> as core::future::future::Future>::poll
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/future/future.rs:133:9
[INFO] [stdout]    7: <core::pin::Pin<P> as core::future::future::Future>::poll
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/future/future.rs:133:9
[INFO] [stdout]    8: tokio::runtime::scheduler::current_thread::CoreGuard::block_on::{{closure}}::{{closure}}::{{closure}}
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:742:70
[INFO] [stdout]    9: tokio::task::coop::with_budget
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/task/coop/mod.rs:167:5
[INFO] [stdout]   10: tokio::task::coop::budget
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/task/coop/mod.rs:133:5
[INFO] [stdout]   11: tokio::runtime::scheduler::current_thread::CoreGuard::block_on::{{closure}}::{{closure}}
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:742:25
[INFO] [stdout]   12: tokio::runtime::scheduler::current_thread::Context::enter
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:432:19
[INFO] [stdout]   13: tokio::runtime::scheduler::current_thread::CoreGuard::block_on::{{closure}}
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:741:44
[INFO] [stdout]   14: tokio::runtime::scheduler::current_thread::CoreGuard::enter::{{closure}}
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:829:68
[INFO] [stdout]   15: tokio::runtime::context::scoped::Scoped<T>::set
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/context/scoped.rs:40:9
[INFO] [stdout]   16: tokio::runtime::context::set_scheduler::{{closure}}
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/context.rs:176:38
[INFO] [stdout]   17: std::thread::local::LocalKey<T>::try_with
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/thread/local.rs:508:12
[INFO] [stdout]   18: std::thread::local::LocalKey<T>::with
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/thread/local.rs:472:20
[INFO] [stdout]   19: tokio::runtime::context::set_scheduler
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/context.rs:176:17
[INFO] [stdout]   20: tokio::runtime::scheduler::current_thread::CoreGuard::enter
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:829:27
[INFO] [stdout]   21: tokio::runtime::scheduler::current_thread::CoreGuard::block_on
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:729:24
[INFO] [stdout]   22: tokio::runtime::scheduler::current_thread::CurrentThread::block_on::{{closure}}
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:200:33
[INFO] [stdout]   23: tokio::runtime::context::runtime::enter_runtime
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/context/runtime.rs:65:16
[INFO] [stdout]   24: tokio::runtime::scheduler::current_thread::CurrentThread::block_on
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:188:9
[INFO] [stdout]   25: tokio::runtime::runtime::Runtime::block_on_inner
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/runtime.rs:356:52
[INFO] [stdout]   26: tokio::runtime::runtime::Runtime::block_on
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/runtime.rs:330:18
[INFO] [stdout]   27: ai_workbench_lib::modules::job_processor::core::job_processor_tests::test_job_processor_folder_end_to_end
[INFO] [stdout]              at ./src/../modules/job_processor/core.rs:787:37
[INFO] [stdout]   28: ai_workbench_lib::modules::job_processor::core::job_processor_tests::test_job_processor_folder_end_to_end::{{closure}}
[INFO] [stdout]              at ./src/../modules/job_processor/core.rs:759:52
[INFO] [stdout]   29: core::ops::function::FnOnce::call_once
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/ops/function.rs:250:5
[INFO] [stdout]   30: core::ops::function::FnOnce::call_once
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/ops/function.rs:250:5
[INFO] [stdout]   31: test::__rust_begin_short_backtrace
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/test/src/lib.rs:663:18
[INFO] [stdout]   32: test::run_test_in_process::{{closure}}
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/test/src/lib.rs:686:74
[INFO] [stdout]   33: <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/panic/unwind_safe.rs:274:9
[INFO] [stdout]   34: std::panicking::catch_unwind::do_call
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panicking.rs:590:40
[INFO] [stdout]   35: std::panicking::catch_unwind
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panicking.rs:553:19
[INFO] [stdout]   36: std::panic::catch_unwind
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panic.rs:359:14
[INFO] [stdout]   37: test::run_test_in_process
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/test/src/lib.rs:686:27
[INFO] [stdout]   38: test::run_test::{{closure}}
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/test/src/lib.rs:607:43
[INFO] [stdout]   39: test::run_test::{{closure}}
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/test/src/lib.rs:637:41
[INFO] [stdout]   40: std::sys::backtrace::__rust_begin_short_backtrace
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/sys/backtrace.rs:158:18
[INFO] [stdout]   41: std::thread::Builder::spawn_unchecked_::{{closure}}::{{closure}}
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/thread/mod.rs:562:17
[INFO] [stdout]   42: <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/panic/unwind_safe.rs:274:9
[INFO] [stdout]   43: std::panicking::catch_unwind::do_call
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panicking.rs:590:40
[INFO] [stdout]   44: std::panicking::catch_unwind
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panicking.rs:553:19
[INFO] [stdout]   45: std::panic::catch_unwind
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panic.rs:359:14
[INFO] [stdout]   46: std::thread::Builder::spawn_unchecked_::{{closure}}
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/thread/mod.rs:560:30
[INFO] [stdout]   47: core::ops::function::FnOnce::call_once{{vtable.shim}}
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/ops/function.rs:250:5
[INFO] [stdout]   48: <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/alloc/src/boxed.rs:2085:9
[INFO] [stdout]   49: std::sys::thread::unix::Thread::new::thread_start
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/sys/thread/unix.rs:124:17
[INFO] [stdout]   50: <unknown>
[INFO] [stdout]   51: clone
[INFO] [stdout] [2m2025-11-11T07:22:52.373579Z[0m [32m INFO[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m === FileDiscovery::discover_files DEBUG END ===
[INFO] [stdout] [2m2025-11-11T07:22:52.373612Z[0m [31mERROR[0m [2mai_workbench_lib::modules::job_processor::core[0m[2m:[0m File discovery failed with error: Failed to list objects in S3
[INFO] [stdout] [2m2025-11-11T07:22:52.373635Z[0m [31mERROR[0m [2mai_workbench_lib::modules::job_processor::core[0m[2m:[0m Error context: Failed to list objects in S3
[INFO] [stdout] 
[INFO] [stdout] Caused by:
[INFO] [stdout]     S3 ListObjectsV2 failed for folder 'text_files/': dispatch failure
[INFO] [stdout] 
[INFO] [stdout] Stack backtrace:
[INFO] [stdout]    0: anyhow::error::<impl anyhow::Error>::msg
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/anyhow-1.0.99/src/backtrace.rs:27:14
[INFO] [stdout]    1: ai_workbench_lib::modules::file_discovery::FileDiscovery::process_folder::{{closure}}
[INFO] [stdout]              at ./src/../modules/file_discovery.rs:157:32
[INFO] [stdout]    2: ai_workbench_lib::modules::file_discovery::FileDiscovery::discover_files::{{closure}}
[INFO] [stdout]              at ./src/../modules/file_discovery.rs:31:45
[INFO] [stdout]    3: ai_workbench_lib::modules::job_processor::core::JobProcessor::discover_files::{{closure}}
[INFO] [stdout]              at ./src/../modules/job_processor/core.rs:102:83
[INFO] [stdout]    4: ai_workbench_lib::modules::job_processor::core::JobProcessor::run::{{closure}}
[INFO] [stdout]              at ./src/../modules/job_processor/core.rs:57:43
[INFO] [stdout]    5: ai_workbench_lib::modules::job_processor::core::job_processor_tests::test_job_processor_folder_end_to_end::{{closure}}
[INFO] [stdout]              at ./src/../modules/job_processor/core.rs:787:31
[INFO] [stdout]    6: <core::pin::Pin<P> as core::future::future::Future>::poll
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/future/future.rs:133:9
[INFO] [stdout]    7: <core::pin::Pin<P> as core::future::future::Future>::poll
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/future/future.rs:133:9
[INFO] [stdout]    8: tokio::runtime::scheduler::current_thread::CoreGuard::block_on::{{closure}}::{{closure}}::{{closure}}
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:742:70
[INFO] [stdout]    9: tokio::task::coop::with_budget
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/task/coop/mod.rs:167:5
[INFO] [stdout]   10: tokio::task::coop::budget
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/task/coop/mod.rs:133:5
[INFO] [stdout]   11: tokio::runtime::scheduler::current_thread::CoreGuard::block_on::{{closure}}::{{closure}}
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:742:25
[INFO] [stdout]   12: tokio::runtime::scheduler::current_thread::Context::enter
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:432:19
[INFO] [stdout]   13: tokio::runtime::scheduler::current_thread::CoreGuard::block_on::{{closure}}
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:741:44
[INFO] [stdout]   14: tokio::runtime::scheduler::current_thread::CoreGuard::enter::{{closure}}
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:829:68
[INFO] [stdout]   15: tokio::runtime::context::scoped::Scoped<T>::set
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/context/scoped.rs:40:9
[INFO] [stdout]   16: tokio::runtime::context::set_scheduler::{{closure}}
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/context.rs:176:38
[INFO] [stdout]   17: std::thread::local::LocalKey<T>::try_with
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/thread/local.rs:508:12
[INFO] [stdout]   18: std::thread::local::LocalKey<T>::with
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/thread/local.rs:472:20
[INFO] [stdout]   19: tokio::runtime::context::set_scheduler
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/context.rs:176:17
[INFO] [stdout]   20: tokio::runtime::scheduler::current_thread::CoreGuard::enter
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:829:27
[INFO] [stdout]   21: tokio::runtime::scheduler::current_thread::CoreGuard::block_on
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:729:24
[INFO] [stdout]   22: tokio::runtime::scheduler::current_thread::CurrentThread::block_on::{{closure}}
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:200:33
[INFO] [stdout]   23: tokio::runtime::context::runtime::enter_runtime
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/context/runtime.rs:65:16
[INFO] [stdout]   24: tokio::runtime::scheduler::current_thread::CurrentThread::block_on
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:188:9
[INFO] [stdout]   25: tokio::runtime::runtime::Runtime::block_on_inner
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/runtime.rs:356:52
[INFO] [stdout]   26: tokio::runtime::runtime::Runtime::block_on
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/runtime.rs:330:18
[INFO] [stdout]   27: ai_workbench_lib::modules::job_processor::core::job_processor_tests::test_job_processor_folder_end_to_end
[INFO] [stdout]              at ./src/../modules/job_processor/core.rs:787:37
[INFO] [stdout]   28: ai_workbench_lib::modules::job_processor::core::job_processor_tests::test_job_processor_folder_end_to_end::{{closure}}
[INFO] [stdout]              at ./src/../modules/job_processor/core.rs:759:52
[INFO] [stdout]   29: core::ops::function::FnOnce::call_once
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/ops/function.rs:250:5
[INFO] [stdout]   30: core::ops::function::FnOnce::call_once
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/ops/function.rs:250:5
[INFO] [stdout]   31: test::__rust_begin_short_backtrace
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/test/src/lib.rs:663:18
[INFO] [stdout]   32: test::run_test_in_process::{{closure}}
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/test/src/lib.rs:686:74
[INFO] [stdout]   33: <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/panic/unwind_safe.rs:274:9
[INFO] [stdout]   34: std::panicking::catch_unwind::do_call
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panicking.rs:590:40
[INFO] [stdout]   35: std::panicking::catch_unwind
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panicking.rs:553:19
[INFO] [stdout]   36: std::panic::catch_unwind
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panic.rs:359:14
[INFO] [stdout]   37: test::run_test_in_process
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/test/src/lib.rs:686:27
[INFO] [stdout]   38: test::run_test::{{closure}}
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/test/src/lib.rs:607:43
[INFO] [stdout]   39: test::run_test::{{closure}}
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/test/src/lib.rs:637:41
[INFO] [stdout]   40: std::sys::backtrace::__rust_begin_short_backtrace
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/sys/backtrace.rs:158:18
[INFO] [stdout]   41: std::thread::Builder::spawn_unchecked_::{{closure}}::{{closure}}
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/thread/mod.rs:562:17
[INFO] [stdout]   42: <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/panic/unwind_safe.rs:274:9
[INFO] [stdout]   43: std::panicking::catch_unwind::do_call
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panicking.rs:590:40
[INFO] [stdout]   44: std::panicking::catch_unwind
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panicking.rs:553:19
[INFO] [stdout]   45: std::panic::catch_unwind
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panic.rs:359:14
[INFO] [stdout]   46: std::thread::Builder::spawn_unchecked_::{{closure}}
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/thread/mod.rs:560:30
[INFO] [stdout]   47: core::ops::function::FnOnce::call_once{{vtable.shim}}
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/ops/function.rs:250:5
[INFO] [stdout]   48: <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/alloc/src/boxed.rs:2085:9
[INFO] [stdout]   49: std::sys::thread::unix::Thread::new::thread_start
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/sys/thread/unix.rs:124:17
[INFO] [stdout]   50: <unknown>
[INFO] [stdout]   51: clone
[INFO] [stdout] [2m2025-11-11T07:22:52.374180Z[0m [32m INFO[0m [2mai_workbench_lib::modules::job_processor::core[0m[2m:[0m === FILE DISCOVERY DEBUG END (FAILED) ===
[INFO] [stdout] [2m2025-11-11T07:22:52.373296Z[0m [31mERROR[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m Error details: Failed to get metadata for single file: test-data/sample.txt
[INFO] [stdout] 
[INFO] [stdout] Caused by:
[INFO] [stdout]     0: Failed to get metadata for test-data/sample.txt
[INFO] [stdout]     1: S3 HEAD request failed for 'test-data/sample.txt': dispatch failure
[INFO] [stdout] 
[INFO] [stdout] Stack backtrace:
[INFO] [stdout]    0: anyhow::error::<impl anyhow::Error>::msg
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/anyhow-1.0.99/src/backtrace.rs:27:14
[INFO] [stdout]    1: ai_workbench_lib::modules::file_discovery::FileDiscovery::get_file_metadata::{{closure}}
[INFO] [stdout]              at ./src/../modules/file_discovery.rs:267:28
[INFO] [stdout]    2: ai_workbench_lib::modules::file_discovery::FileDiscovery::process_single_file::{{closure}}
[INFO] [stdout]              at ./src/../modules/file_discovery.rs:59:64
[INFO] [stdout]    3: ai_workbench_lib::modules::file_discovery::FileDiscovery::discover_files::{{closure}}
[INFO] [stdout]              at ./src/../modules/file_discovery.rs:35:50
[INFO] [stdout]    4: ai_workbench_lib::modules::file_discovery::file_discovery_tests::test_single_file_discovery::{{closure}}
[INFO] [stdout]              at ./src/../modules/file_discovery.rs:441:62
[INFO] [stdout]    5: <core::pin::Pin<P> as core::future::future::Future>::poll
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/future/future.rs:133:9
[INFO] [stdout]    6: <core::pin::Pin<P> as core::future::future::Future>::poll
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/future/future.rs:133:9
[INFO] [stdout]    7: tokio::runtime::scheduler::current_thread::CoreGuard::block_on::{{closure}}::{{closure}}::{{closure}}
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:742:70
[INFO] [stdout]    8: tokio::task::coop::with_budget
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/task/coop/mod.rs:167:5
[INFO] [stdout]    9: tokio::task::coop::budget
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/task/coop/mod.rs:133:5
[INFO] [stdout]   10: tokio::runtime::scheduler::current_thread::CoreGuard::block_on::{{closure}}::{{closure}}
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:742:25
[INFO] [stdout]   11: tokio::runtime::scheduler::current_thread::Context::enter
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:432:19
[INFO] [stdout]   12: tokio::runtime::scheduler::current_thread::CoreGuard::block_on::{{closure}}
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:741:44
[INFO] [stdout]   13: tokio::runtime::scheduler::current_thread::CoreGuard::enter::{{closure}}
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:829:68
[INFO] [stdout]   14: tokio::runtime::context::scoped::Scoped<T>::set
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/context/scoped.rs:40:9
[INFO] [stdout]   15: tokio::runtime::context::set_scheduler::{{closure}}
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/context.rs:176:38
[INFO] [stdout]   16: std::thread::local::LocalKey<T>::try_with
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/thread/local.rs:508:12
[INFO] [stdout]   17: std::thread::local::LocalKey<T>::with
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/thread/local.rs:472:20
[INFO] [stdout]   18: tokio::runtime::context::set_scheduler
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/context.rs:176:17
[INFO] [stdout]   19: tokio::runtime::scheduler::current_thread::CoreGuard::enter
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:829:27
[INFO] [stdout]   20: tokio::runtime::scheduler::current_thread::CoreGuard::block_on
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:729:24
[INFO] [stdout]   21: tokio::runtime::scheduler::current_thread::CurrentThread::block_on::{{closure}}
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:200:33
[INFO] [stdout]   22: tokio::runtime::context::runtime::enter_runtime
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/context/runtime.rs:65:16
[INFO] [stdout]   23: tokio::runtime::scheduler::current_thread::CurrentThread::block_on
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:188:9
[INFO] [stdout]   24: tokio::runtime::runtime::Runtime::block_on_inner
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/runtime.rs:356:52
[INFO] [stdout]   25: tokio::runtime::runtime::Runtime::block_on
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/runtime.rs:330:18
[INFO] [stdout]   26: ai_workbench_lib::modules::file_discovery::file_discovery_tests::test_single_file_discovery
[INFO] [stdout]              at ./src/../modules/file_discovery.rs:443:22
[INFO] [stdout]   27: ai_workbench_lib::modules::file_discovery::file_discovery_tests::test_single_file_discovery::{{closure}}
[INFO] [stdout]              at ./src/../modules/file_discovery.rs:433:42
[INFO] [stdout]   28: core::ops::function::FnOnce::call_once
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/ops/function.rs:250:5
[INFO] [stdout]   29: core::ops::function::FnOnce::call_once
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/ops/function.rs:250:5
[INFO] [stdout]   30: test::__rust_begin_short_backtrace
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/test/src/lib.rs:663:18
[INFO] [stdout]   31: test::run_test_in_process::{{closure}}
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/test/src/lib.rs:686:74
[INFO] [stdout]   32: <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/panic/unwind_safe.rs:274:9
[INFO] [stdout]   33: std::panicking::catch_unwind::do_call
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panicking.rs:590:40
[INFO] [stdout]   34: std::panicking::catch_unwind
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panicking.rs:553:19
[INFO] [stdout]   35: std::panic::catch_unwind
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panic.rs:359:14
[INFO] [stdout]   36: test::run_test_in_process
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/test/src/lib.rs:686:27
[INFO] [stdout]   37: test::run_test::{{closure}}
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/test/src/lib.rs:607:43
[INFO] [stdout]   38: test::run_test::{{closure}}
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/test/src/lib.rs:637:41
[INFO] [stdout]   39: std::sys::backtrace::__rust_begin_short_backtrace
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/sys/backtrace.rs:158:18
[INFO] [stdout]   40: std::thread::Builder::spawn_unchecked_::{{closure}}::{{closure}}
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/thread/mod.rs:562:17
[INFO] [stdout]   41: <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/panic/unwind_safe.rs:274:9
[INFO] [stdout]   42: std::panicking::catch_unwind::do_call
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panicking.rs:590:40
[INFO] [stdout]   43: std::panicking::catch_unwind
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panicking.rs:553:19
[INFO] [stdout]   44: std::panic::catch_unwind
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panic.rs:359:14
[INFO] [stdout]   45: std::thread::Builder::spawn_unchecked_::{{closure}}
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/thread/mod.rs:560:30
[INFO] [stdout]   46: core::ops::function::FnOnce::call_once{{vtable.shim}}
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/ops/function.rs:250:5
[INFO] [stdout]   47: <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/alloc/src/boxed.rs:2085:9
[INFO] [stdout]   48: std::sys::thread::unix::Thread::new::thread_start
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/sys/thread/unix.rs:124:17
[INFO] [stdout]   49: <unknown>
[INFO] [stdout]   50: clone
[INFO] [stdout] [2m2025-11-11T07:22:52.374318Z[0m [32m INFO[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m === FileDiscovery::discover_files DEBUG END ===
[INFO] [stdout] test modules::file_discovery::file_discovery_tests::test_single_file_discovery ... ok
[INFO] [stdout] [2m2025-11-11T07:22:52.375104Z[0m [31mERROR[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m Failed to get metadata for file 'test-data/sample.txt': Failed to get metadata for test-data/sample.txt
[INFO] [stdout] [2m2025-11-11T07:22:52.375143Z[0m [31mERROR[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m This usually means:
[INFO] [stdout] [2m2025-11-11T07:22:52.375152Z[0m [31mERROR[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m   1. File does not exist at s3://ai-workbench-6c9c43db-7fe6-42f1-8b11-8f82323f83f0-eu-west-2/test-data/sample.txt
[INFO] [stdout] [2m2025-11-11T07:22:52.375164Z[0m [31mERROR[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m   2. No permissions to access the file
[INFO] [stdout] [2m2025-11-11T07:22:52.375180Z[0m [31mERROR[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m   3. S3 service is unavailable
[INFO] [stdout] [2m2025-11-11T07:22:52.375188Z[0m [31mERROR[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m   4. Incorrect bucket name or file path
[INFO] [stdout] [2m2025-11-11T07:22:52.375178Z[0m [31mERROR[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m discover_files failed: Failed to list objects in S3
[INFO] [stdout] [2m2025-11-11T07:22:52.375198Z[0m [31mERROR[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m discover_files failed: Failed to get metadata for single file: test-data/sample.txt
[INFO] [stdout] [2m2025-11-11T07:22:52.375486Z[0m [32m INFO[0m [2mai_workbench_lib::modules::job_processor::core[0m[2m:[0m === FILE DISCOVERY DEBUG START ===
[INFO] [stdout] [2m2025-11-11T07:22:52.375642Z[0m [32m INFO[0m [2mai_workbench_lib::modules::job_processor::core[0m[2m:[0m Input specification: 'text_files/'
[INFO] [stdout] [2m2025-11-11T07:22:52.375671Z[0m [32m INFO[0m [2mai_workbench_lib::modules::job_processor::core[0m[2m:[0m Workspace bucket: 'ai-workbench-6c9c43db-7fe6-42f1-8b11-8f82323f83f0-eu-west-2'
[INFO] [stdout] [2m2025-11-11T07:22:52.375682Z[0m [32m INFO[0m [2mai_workbench_lib::modules::job_processor::core[0m[2m:[0m Full S3 path would be: s3://ai-workbench-6c9c43db-7fe6-42f1-8b11-8f82323f83f0-eu-west-2/text_files/
[INFO] [stdout] [2m2025-11-11T07:22:52.375695Z[0m [32m INFO[0m [2mai_workbench_lib::modules::job_processor::core[0m[2m:[0m FileDiscovery instance created successfully
[INFO] [stdout] [2m2025-11-11T07:22:52.375704Z[0m [32m INFO[0m [2mai_workbench_lib::modules::job_processor::core[0m[2m:[0m Attempting to discover files...
[INFO] [stdout] [2m2025-11-11T07:22:52.375716Z[0m [32m INFO[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m === FileDiscovery::discover_files DEBUG START ===
[INFO] [stdout] [2m2025-11-11T07:22:52.375725Z[0m [32m INFO[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m Input spec: 'text_files/'
[INFO] [stdout] [2m2025-11-11T07:22:52.375734Z[0m [32m INFO[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m Workspace bucket: 'ai-workbench-6c9c43db-7fe6-42f1-8b11-8f82323f83f0-eu-west-2'
[INFO] [stdout] [2m2025-11-11T07:22:52.375746Z[0m [32m INFO[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m Detected folder input (ends with '/') - processing as folder
[INFO] [stdout] [2m2025-11-11T07:22:52.375759Z[0m [32m INFO[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m === process_folder DEBUG START ===
[INFO] [stdout] [2m2025-11-11T07:22:52.375769Z[0m [32m INFO[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m Processing folder: 'text_files/'
[INFO] [stdout] [2m2025-11-11T07:22:52.375777Z[0m [32m INFO[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m Full S3 path: s3://ai-workbench-6c9c43db-7fe6-42f1-8b11-8f82323f83f0-eu-west-2/text_files/
[INFO] [stdout] [2m2025-11-11T07:22:52.375788Z[0m [32m INFO[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m Making S3 ListObjects request #1
[INFO] [stdout] [2m2025-11-11T07:22:52.375803Z[0m [32m INFO[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m Sending ListObjectsV2 request to S3...
[INFO] [stdout] [2m2025-11-11T07:22:52.375202Z[0m [31mERROR[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m Error details: Failed to list objects in S3
[INFO] [stdout] 
[INFO] [stdout] Caused by:
[INFO] [stdout]     S3 ListObjectsV2 failed for folder 'text_files/': dispatch failure
[INFO] [stdout] 
[INFO] [stdout] Stack backtrace:
[INFO] [stdout]    0: anyhow::error::<impl anyhow::Error>::msg
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/anyhow-1.0.99/src/backtrace.rs:27:14
[INFO] [stdout]    1: ai_workbench_lib::modules::file_discovery::FileDiscovery::process_folder::{{closure}}
[INFO] [stdout]              at ./src/../modules/file_discovery.rs:157:32
[INFO] [stdout]    2: ai_workbench_lib::modules::file_discovery::FileDiscovery::discover_files::{{closure}}
[INFO] [stdout]              at ./src/../modules/file_discovery.rs:31:45
[INFO] [stdout]    3: ai_workbench_lib::modules::file_discovery::file_discovery_tests::test_folder_discovery::{{closure}}
[INFO] [stdout]              at ./src/../modules/file_discovery.rs:470:60
[INFO] [stdout]    4: <core::pin::Pin<P> as core::future::future::Future>::poll
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/future/future.rs:133:9
[INFO] [stdout]    5: <core::pin::Pin<P> as core::future::future::Future>::poll
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/future/future.rs:133:9
[INFO] [stdout]    6: tokio::runtime::scheduler::current_thread::CoreGuard::block_on::{{closure}}::{{closure}}::{{closure}}
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:742:70
[INFO] [stdout]    7: tokio::task::coop::with_budget
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/task/coop/mod.rs:167:5
[INFO] [stdout]    8: tokio::task::coop::budget
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/task/coop/mod.rs:133:5
[INFO] [stdout]    9: tokio::runtime::scheduler::current_thread::CoreGuard::block_on::{{closure}}::{{closure}}
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:742:25
[INFO] [stdout]   10: tokio::runtime::scheduler::current_thread::Context::enter
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:432:19
[INFO] [stdout]   11: tokio::runtime::scheduler::current_thread::CoreGuard::block_on::{{closure}}
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:741:44
[INFO] [stdout]   12: tokio::runtime::scheduler::current_thread::CoreGuard::enter::{{closure}}
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:829:68
[INFO] [stdout]   13: tokio::runtime::context::scoped::Scoped<T>::set
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/context/scoped.rs:40:9
[INFO] [stdout]   14: tokio::runtime::context::set_scheduler::{{closure}}
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/context.rs:176:38
[INFO] [stdout]   15: std::thread::local::LocalKey<T>::try_with
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/thread/local.rs:508:12
[INFO] [stdout]   16: std::thread::local::LocalKey<T>::with
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/thread/local.rs:472:20
[INFO] [stdout]   17: tokio::runtime::context::set_scheduler
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/context.rs:176:17
[INFO] [stdout]   18: tokio::runtime::scheduler::current_thread::CoreGuard::enter
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:829:27
[INFO] [stdout]   19: tokio::runtime::scheduler::current_thread::CoreGuard::block_on
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:729:24
[INFO] [stdout]   20: tokio::runtime::scheduler::current_thread::CurrentThread::block_on::{{closure}}
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:200:33
[INFO] [stdout]   21: tokio::runtime::context::runtime::enter_runtime
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/context/runtime.rs:65:16
[INFO] [stdout]   22: tokio::runtime::scheduler::current_thread::CurrentThread::block_on
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:188:9
[INFO] [stdout]   23: tokio::runtime::runtime::Runtime::block_on_inner
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/runtime.rs:356:52
[INFO] [stdout]   24: tokio::runtime::runtime::Runtime::block_on
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/runtime.rs:330:18
[INFO] [stdout]   25: ai_workbench_lib::modules::file_discovery::file_discovery_tests::test_folder_discovery
[INFO] [stdout]              at ./src/../modules/file_discovery.rs:472:22
[INFO] [stdout]   26: ai_workbench_lib::modules::file_discovery::file_discovery_tests::test_folder_discovery::{{closure}}
[INFO] [stdout]              at ./src/../modules/file_discovery.rs:462:37
[INFO] [stdout]   27: core::ops::function::FnOnce::call_once
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/ops/function.rs:250:5
[INFO] [stdout]   28: core::ops::function::FnOnce::call_once
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/ops/function.rs:250:5
[INFO] [stdout]   29: test::__rust_begin_short_backtrace
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/test/src/lib.rs:663:18
[INFO] [stdout]   30: test::run_test_in_process::{{closure}}
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/test/src/lib.rs:686:74
[INFO] [stdout]   31: <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/panic/unwind_safe.rs:274:9
[INFO] [stdout]   32: std::panicking::catch_unwind::do_call
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panicking.rs:590:40
[INFO] [stdout]   33: std::panicking::catch_unwind
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panicking.rs:553:19
[INFO] [stdout]   34: std::panic::catch_unwind
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panic.rs:359:14
[INFO] [stdout]   35: test::run_test_in_process
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/test/src/lib.rs:686:27
[INFO] [stdout]   36: test::run_test::{{closure}}
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/test/src/lib.rs:607:43
[INFO] [stdout]   37: test::run_test::{{closure}}
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/test/src/lib.rs:637:41
[INFO] [stdout]   38: std::sys::backtrace::__rust_begin_short_backtrace
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/sys/backtrace.rs:158:18
[INFO] [stdout]   39: std::thread::Builder::spawn_unchecked_::{{closure}}::{{closure}}
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/thread/mod.rs:562:17
[INFO] [stdout]   40: <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/panic/unwind_safe.rs:274:9
[INFO] [stdout]   41: std::panicking::catch_unwind::do_call
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panicking.rs:590:40
[INFO] [stdout]   42: std::panicking::catch_unwind
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panicking.rs:553:19
[INFO] [stdout]   43: std::panic::catch_unwind
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panic.rs:359:14
[INFO] [stdout]   44: std::thread::Builder::spawn_unchecked_::{{closure}}
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/thread/mod.rs:560:30
[INFO] [stdout]   45: core::ops::function::FnOnce::call_once{{vtable.shim}}
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/ops/function.rs:250:5
[INFO] [stdout]   46: <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/alloc/src/boxed.rs:2085:9
[INFO] [stdout]   47: std::sys::thread::unix::Thread::new::thread_start
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/sys/thread/unix.rs:124:17
[INFO] [stdout]   48: <unknown>
[INFO] [stdout]   49: clone
[INFO] [stdout] [2m2025-11-11T07:22:52.375851Z[0m [32m INFO[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m === FileDiscovery::discover_files DEBUG END ===
[INFO] [stdout] test modules::job_processor::core::job_processor_tests::test_job_processor_folder_end_to_end ... ok
[INFO] [stdout] [2m2025-11-11T07:22:52.377685Z[0m [32m INFO[0m [2mai_workbench_lib::modules::job_processor::core[0m[2m:[0m === FILE DISCOVERY DEBUG START ===
[INFO] [stdout] [2m2025-11-11T07:22:52.377714Z[0m [32m INFO[0m [2mai_workbench_lib::modules::job_processor::core[0m[2m:[0m Input specification: 'text_files/0c472776-7c22-464b-93bf-3714fd229b01.txt'
[INFO] [stdout] [2m2025-11-11T07:22:52.377725Z[0m [32m INFO[0m [2mai_workbench_lib::modules::job_processor::core[0m[2m:[0m Workspace bucket: 'ai-workbench-6c9c43db-7fe6-42f1-8b11-8f82323f83f0-eu-west-2'
[INFO] [stdout] [2m2025-11-11T07:22:52.377736Z[0m [32m INFO[0m [2mai_workbench_lib::modules::job_processor::core[0m[2m:[0m Full S3 path would be: s3://ai-workbench-6c9c43db-7fe6-42f1-8b11-8f82323f83f0-eu-west-2/text_files/0c472776-7c22-464b-93bf-3714fd229b01.txt
[INFO] [stdout] [2m2025-11-11T07:22:52.377750Z[0m [32m INFO[0m [2mai_workbench_lib::modules::job_processor::core[0m[2m:[0m FileDiscovery instance created successfully
[INFO] [stdout] [2m2025-11-11T07:22:52.377758Z[0m [32m INFO[0m [2mai_workbench_lib::modules::job_processor::core[0m[2m:[0m Attempting to discover files...
[INFO] [stdout] [2m2025-11-11T07:22:52.377769Z[0m [32m INFO[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m === FileDiscovery::discover_files DEBUG START ===
[INFO] [stdout] [2m2025-11-11T07:22:52.377779Z[0m [32m INFO[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m Input spec: 'text_files/0c472776-7c22-464b-93bf-3714fd229b01.txt'
[INFO] [stdout] [2m2025-11-11T07:22:52.377788Z[0m [32m INFO[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m Workspace bucket: 'ai-workbench-6c9c43db-7fe6-42f1-8b11-8f82323f83f0-eu-west-2'
[INFO] [stdout] [2m2025-11-11T07:22:52.377802Z[0m [32m INFO[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m Detected single file input (no trailing '/') - processing as single file
[INFO] [stdout] [2m2025-11-11T07:22:52.377815Z[0m [32m INFO[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m === process_single_file DEBUG START ===
[INFO] [stdout] [2m2025-11-11T07:22:52.377826Z[0m [32m INFO[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m Processing single file: 'text_files/0c472776-7c22-464b-93bf-3714fd229b01.txt'
[INFO] [stdout] [2m2025-11-11T07:22:52.377837Z[0m [32m INFO[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m Full S3 path: s3://ai-workbench-6c9c43db-7fe6-42f1-8b11-8f82323f83f0-eu-west-2/text_files/0c472776-7c22-464b-93bf-3714fd229b01.txt
[INFO] [stdout] [2m2025-11-11T07:22:52.377850Z[0m [32m INFO[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m Attempting to get file metadata...
[INFO] [stdout] [2m2025-11-11T07:22:52.377894Z[0m [32m INFO[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m === get_file_metadata DEBUG START ===
[INFO] [stdout] [2m2025-11-11T07:22:52.377904Z[0m [32m INFO[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m Getting metadata for file: 'text_files/0c472776-7c22-464b-93bf-3714fd229b01.txt'
[INFO] [stdout] [2m2025-11-11T07:22:52.377915Z[0m [32m INFO[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m Bucket: 'ai-workbench-6c9c43db-7fe6-42f1-8b11-8f82323f83f0-eu-west-2'
[INFO] [stdout] [2m2025-11-11T07:22:52.377925Z[0m [32m INFO[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m Full S3 path: s3://ai-workbench-6c9c43db-7fe6-42f1-8b11-8f82323f83f0-eu-west-2/text_files/0c472776-7c22-464b-93bf-3714fd229b01.txt
[INFO] [stdout] [2m2025-11-11T07:22:52.377939Z[0m [32m INFO[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m Sending HEAD request to S3...
[INFO] [stdout] [2m2025-11-11T07:22:52.378836Z[0m [31mERROR[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m S3 ListObjectsV2 request failed: dispatch failure
[INFO] [stdout] [2m2025-11-11T07:22:52.378859Z[0m [31mERROR[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m Error details: DispatchFailure(DispatchFailure { source: ConnectorError { kind: Other(None), source: CredentialsNotLoaded(CredentialsNotLoaded { source: Some("no providers in chain provided credentials") }), connection: Unknown } })
[INFO] [stdout] [2m2025-11-11T07:22:52.378890Z[0m [31mERROR[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m Possible causes:
[INFO] [stdout] [2m2025-11-11T07:22:52.378900Z[0m [31mERROR[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m   1. Folder/prefix 'text_files/' does not exist in bucket 'ai-workbench-6c9c43db-7fe6-42f1-8b11-8f82323f83f0-eu-west-2'
[INFO] [stdout] [2m2025-11-11T07:22:52.378912Z[0m [31mERROR[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m   2. No list permissions for this bucket/prefix
[INFO] [stdout] [2m2025-11-11T07:22:52.378921Z[0m [31mERROR[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m   3. AWS credentials are invalid or expired
[INFO] [stdout] [2m2025-11-11T07:22:52.378930Z[0m [31mERROR[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m   4. Bucket 'ai-workbench-6c9c43db-7fe6-42f1-8b11-8f82323f83f0-eu-west-2' does not exist
[INFO] [stdout] [2m2025-11-11T07:22:52.378941Z[0m [31mERROR[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m   5. Network connectivity issues
[INFO] [stdout] [2m2025-11-11T07:22:52.379000Z[0m [31mERROR[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m discover_files failed: Failed to list objects in S3
[INFO] [stdout] [2m2025-11-11T07:22:52.375610Z[0m [31mERROR[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m discover_files failed: Failed to list objects in S3
[INFO] [stdout] [2m2025-11-11T07:22:52.379015Z[0m [31mERROR[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m Error details: Failed to list objects in S3
[INFO] [stdout] 
[INFO] [stdout] Caused by:
[INFO] [stdout]     S3 ListObjectsV2 failed for folder 'text_files/': dispatch failure
[INFO] [stdout] 
[INFO] [stdout] Stack backtrace:
[INFO] [stdout]    0: anyhow::error::<impl anyhow::Error>::msg
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/anyhow-1.0.99/src/backtrace.rs:27:14
[INFO] [stdout]    1: ai_workbench_lib::modules::file_discovery::FileDiscovery::process_folder::{{closure}}
[INFO] [stdout]              at ./src/../modules/file_discovery.rs:157:32
[INFO] [stdout]    2: ai_workbench_lib::modules::file_discovery::FileDiscovery::discover_files::{{closure}}
[INFO] [stdout]              at ./src/../modules/file_discovery.rs:31:45
[INFO] [stdout]    3: ai_workbench_lib::modules::job_processor::core::JobProcessor::discover_files::{{closure}}
[INFO] [stdout]              at ./src/../modules/job_processor/core.rs:102:83
[INFO] [stdout]    4: ai_workbench_lib::modules::job_processor::core::job_processor_tests::test_job_processor_config_validation::{{closure}}
[INFO] [stdout]              at ./src/../modules/job_processor/core.rs:1024:46
[INFO] [stdout]    5: <core::pin::Pin<P> as core::future::future::Future>::poll
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/future/future.rs:133:9
[INFO] [stdout]    6: <core::pin::Pin<P> as core::future::future::Future>::poll
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/future/future.rs:133:9
[INFO] [stdout]    7: tokio::runtime::scheduler::current_thread::CoreGuard::block_on::{{closure}}::{{closure}}::{{closure}}
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:742:70
[INFO] [stdout]    8: tokio::task::coop::with_budget
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/task/coop/mod.rs:167:5
[INFO] [stdout]    9: tokio::task::coop::budget
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/task/coop/mod.rs:133:5
[INFO] [stdout]   10: tokio::runtime::scheduler::current_thread::CoreGuard::block_on::{{closure}}::{{closure}}
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:742:25
[INFO] [stdout]   11: tokio::runtime::scheduler::current_thread::Context::enter
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:432:19
[INFO] [stdout]   12: tokio::runtime::scheduler::current_thread::CoreGuard::block_on::{{closure}}
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:741:44
[INFO] [stdout]   13: tokio::runtime::scheduler::current_thread::CoreGuard::enter::{{closure}}
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:829:68
[INFO] [stdout]   14: tokio::runtime::context::scoped::Scoped<T>::set
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/context/scoped.rs:40:9
[INFO] [stdout]   15: tokio::runtime::context::set_scheduler::{{closure}}
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/context.rs:176:38
[INFO] [stdout]   16: std::thread::local::LocalKey<T>::try_with
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/thread/local.rs:508:12
[INFO] [stdout]   17: std::thread::local::LocalKey<T>::with
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/thread/local.rs:472:20
[INFO] [stdout]   18: tokio::runtime::context::set_scheduler
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/context.rs:176:17
[INFO] [stdout]   19: tokio::runtime::scheduler::current_thread::CoreGuard::enter
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:829:27
[INFO] [stdout]   20: tokio::runtime::scheduler::current_thread::CoreGuard::block_on
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:729:24
[INFO] [stdout]   21: tokio::runtime::scheduler::current_thread::CurrentThread::block_on::{{closure}}
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:200:33
[INFO] [stdout]   22: tokio::runtime::context::runtime::enter_runtime
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/context/runtime.rs:65:16
[INFO] [stdout]   23: tokio::runtime::scheduler::current_thread::CurrentThread::block_on
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:188:9
[INFO] [stdout]   24: tokio::runtime::runtime::Runtime::block_on_inner
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/runtime.rs:356:52
[INFO] [stdout]   25: tokio::runtime::runtime::Runtime::block_on
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/runtime.rs:330:18
[INFO] [stdout]   26: ai_workbench_lib::modules::job_processor::core::job_processor_tests::test_job_processor_config_validation
[INFO] [stdout]              at ./src/../modules/job_processor/core.rs:1017:31
[INFO] [stdout]   27: ai_workbench_lib::modules::job_processor::core::job_processor_tests::test_job_processor_config_validation::{{closure}}
[INFO] [stdout]              at ./src/../modules/job_processor/core.rs:980:52
[INFO] [stdout]   28: core::ops::function::FnOnce::call_once
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/ops/function.rs:250:5
[INFO] [stdout]   29: core::ops::function::FnOnce::call_once
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/ops/function.rs:250:5
[INFO] [stdout]   30: test::__rust_begin_short_backtrace
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/test/src/lib.rs:663:18
[INFO] [stdout]   31: test::run_test_in_process::{{closure}}
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/test/src/lib.rs:686:74
[INFO] [stdout]   32: <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/panic/unwind_safe.rs:274:9
[INFO] [stdout]   33: std::panicking::catch_unwind::do_call
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panicking.rs:590:40
[INFO] [stdout]   34: std::panicking::catch_unwind
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panicking.rs:553:19
[INFO] [stdout]   35: std::panic::catch_unwind
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panic.rs:359:14
[INFO] [stdout]   36: test::run_test_in_process
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/test/src/lib.rs:686:27
[INFO] [stdout]   37: test::run_test::{{closure}}
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/test/src/lib.rs:607:43
[INFO] [stdout]   38: test::run_test::{{closure}}
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/test/src/lib.rs:637:41
[INFO] [stdout]   39: std::sys::backtrace::__rust_begin_short_backtrace
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/sys/backtrace.rs:158:18
[INFO] [stdout]   40: std::thread::Builder::spawn_unchecked_::{{closure}}::{{closure}}
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/thread/mod.rs:562:17
[INFO] [stdout]   41: <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/panic/unwind_safe.rs:274:9
[INFO] [stdout]   42: std::panicking::catch_unwind::do_call
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panicking.rs:590:40
[INFO] [stdout]   43: std::panicking::catch_unwind
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panicking.rs:553:19
[INFO] [stdout]   44: std::panic::catch_unwind
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panic.rs:359:14
[INFO] [stdout]   45: std::thread::Builder::spawn_unchecked_::{{closure}}
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/thread/mod.rs:560:30
[INFO] [stdout]   46: core::ops::function::FnOnce::call_once{{vtable.shim}}
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/ops/function.rs:250:5
[INFO] [stdout]   47: <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/alloc/src/boxed.rs:2085:9
[INFO] [stdout]   48: std::sys::thread::unix::Thread::new::thread_start
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/sys/thread/unix.rs:124:17
[INFO] [stdout]   49: <unknown>
[INFO] [stdout]   50: clone
[INFO] [stdout] [2m2025-11-11T07:22:52.379840Z[0m [32m INFO[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m === FileDiscovery::discover_files DEBUG END ===
[INFO] [stdout] [2m2025-11-11T07:22:52.379856Z[0m [31mERROR[0m [2mai_workbench_lib::modules::job_processor::core[0m[2m:[0m File discovery failed with error: Failed to list objects in S3
[INFO] [stdout] [2m2025-11-11T07:22:52.379867Z[0m [31mERROR[0m [2mai_workbench_lib::modules::job_processor::core[0m[2m:[0m Error context: Failed to list objects in S3
[INFO] [stdout] 
[INFO] [stdout] Caused by:
[INFO] [stdout]     S3 ListObjectsV2 failed for folder 'text_files/': dispatch failure
[INFO] [stdout] 
[INFO] [stdout] Stack backtrace:
[INFO] [stdout]    0: anyhow::error::<impl anyhow::Error>::msg
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/anyhow-1.0.99/src/backtrace.rs:27:14
[INFO] [stdout]    1: ai_workbench_lib::modules::file_discovery::FileDiscovery::process_folder::{{closure}}
[INFO] [stdout]              at ./src/../modules/file_discovery.rs:157:32
[INFO] [stdout]    2: ai_workbench_lib::modules::file_discovery::FileDiscovery::discover_files::{{closure}}
[INFO] [stdout]              at ./src/../modules/file_discovery.rs:31:45
[INFO] [stdout]    3: ai_workbench_lib::modules::job_processor::core::JobProcessor::discover_files::{{closure}}
[INFO] [stdout]              at ./src/../modules/job_processor/core.rs:102:83
[INFO] [stdout]    4: ai_workbench_lib::modules::job_processor::core::job_processor_tests::test_job_processor_config_validation::{{closure}}
[INFO] [stdout]              at ./src/../modules/job_processor/core.rs:1024:46
[INFO] [stdout]    5: <core::pin::Pin<P> as core::future::future::Future>::poll
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/future/future.rs:133:9
[INFO] [stdout]    6: <core::pin::Pin<P> as core::future::future::Future>::poll
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/future/future.rs:133:9
[INFO] [stdout]    7: tokio::runtime::scheduler::current_thread::CoreGuard::block_on::{{closure}}::{{closure}}::{{closure}}
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:742:70
[INFO] [stdout]    8: tokio::task::coop::with_budget
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/task/coop/mod.rs:167:5
[INFO] [stdout]    9: tokio::task::coop::budget
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/task/coop/mod.rs:133:5
[INFO] [stdout]   10: tokio::runtime::scheduler::current_thread::CoreGuard::block_on::{{closure}}::{{closure}}
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:742:25
[INFO] [stdout]   11: tokio::runtime::scheduler::current_thread::Context::enter
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:432:19
[INFO] [stdout]   12: tokio::runtime::scheduler::current_thread::CoreGuard::block_on::{{closure}}
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:741:44
[INFO] [stdout]   13: tokio::runtime::scheduler::current_thread::CoreGuard::enter::{{closure}}
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:829:68
[INFO] [stdout]   14: tokio::runtime::context::scoped::Scoped<T>::set
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/context/scoped.rs:40:9
[INFO] [stdout]   15: tokio::runtime::context::set_scheduler::{{closure}}
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/context.rs:176:38
[INFO] [stdout]   16: std::thread::local::LocalKey<T>::try_with
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/thread/local.rs:508:12
[INFO] [stdout]   17: std::thread::local::LocalKey<T>::with
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/thread/local.rs:472:20
[INFO] [stdout]   18: tokio::runtime::context::set_scheduler
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/context.rs:176:17
[INFO] [stdout]   19: tokio::runtime::scheduler::current_thread::CoreGuard::enter
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:829:27
[INFO] [stdout]   20: tokio::runtime::scheduler::current_thread::CoreGuard::block_on
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:729:24
[INFO] [stdout]   21: tokio::runtime::scheduler::current_thread::CurrentThread::block_on::{{closure}}
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:200:33
[INFO] [stdout]   22: tokio::runtime::context::runtime::enter_runtime
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/context/runtime.rs:65:16
[INFO] [stdout]   23: tokio::runtime::scheduler::current_thread::CurrentThread::block_on
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:188:9
[INFO] [stdout]   24: tokio::runtime::runtime::Runtime::block_on_inner
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/runtime.rs:356:52
[INFO] [stdout]   25: tokio::runtime::runtime::Runtime::block_on
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/runtime.rs:330:18
[INFO] [stdout]   26: ai_workbench_lib::modules::job_processor::core::job_processor_tests::test_job_processor_config_validation
[INFO] [stdout]              at ./src/../modules/job_processor/core.rs:1017:31
[INFO] [stdout]   27: ai_workbench_lib::modules::job_processor::core::job_processor_tests::test_job_processor_config_validation::{{closure}}
[INFO] [stdout]              at ./src/../modules/job_processor/core.rs:980:52
[INFO] [stdout]   28: core::ops::function::FnOnce::call_once
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/ops/function.rs:250:5
[INFO] [stdout]   29: core::ops::function::FnOnce::call_once
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/ops/function.rs:250:5
[INFO] [stdout]   30: test::__rust_begin_short_backtrace
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/test/src/lib.rs:663:18
[INFO] [stdout]   31: test::run_test_in_process::{{closure}}
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/test/src/lib.rs:686:74
[INFO] [stdout]   32: <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/panic/unwind_safe.rs:274:9
[INFO] [stdout]   33: std::panicking::catch_unwind::do_call
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panicking.rs:590:40
[INFO] [stdout]   34: std::panicking::catch_unwind
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panicking.rs:553:19
[INFO] [stdout]   35: std::panic::catch_unwind
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panic.rs:359:14
[INFO] [stdout]   36: test::run_test_in_process
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/test/src/lib.rs:686:27
[INFO] [stdout]   37: test::run_test::{{closure}}
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/test/src/lib.rs:607:43
[INFO] [stdout]   38: test::run_test::{{closure}}
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/test/src/lib.rs:637:41
[INFO] [stdout]   39: std::sys::backtrace::__rust_begin_short_backtrace
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/sys/backtrace.rs:158:18
[INFO] [stdout]   40: std::thread::Builder::spawn_unchecked_::{{closure}}::{{closure}}
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/thread/mod.rs:562:17
[INFO] [stdout]   41: <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/panic/unwind_safe.rs:274:9
[INFO] [stdout]   42: std::panicking::catch_unwind::do_call
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panicking.rs:590:40
[INFO] [stdout]   43: std::panicking::catch_unwind
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panicking.rs:553:19
[INFO] [stdout]   44: std::panic::catch_unwind
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panic.rs:359:14
[INFO] [stdout]   45: std::thread::Builder::spawn_unchecked_::{{closure}}
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/thread/mod.rs:560:30
[INFO] [stdout]   46: core::ops::function::FnOnce::call_once{{vtable.shim}}
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/ops/function.rs:250:5
[INFO] [stdout]   47: <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/alloc/src/boxed.rs:2085:9
[INFO] [stdout]   48: std::sys::thread::unix::Thread::new::thread_start
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/sys/thread/unix.rs:124:17
[INFO] [stdout]   49: <unknown>
[INFO] [stdout]   50: clone
[INFO] [stdout] [2m2025-11-11T07:22:52.380344Z[0m [32m INFO[0m [2mai_workbench_lib::modules::job_processor::core[0m[2m:[0m === FILE DISCOVERY DEBUG END (FAILED) ===
[INFO] [stdout] [2m2025-11-11T07:22:52.380596Z[0m [31mERROR[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m discover_files failed: Failed to list objects in S3
[INFO] [stdout] [2m2025-11-11T07:22:52.375210Z[0m [31mERROR[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m Error details: Failed to get metadata for single file: test-data/sample.txt
[INFO] [stdout] 
[INFO] [stdout] Caused by:
[INFO] [stdout]     0: Failed to get metadata for test-data/sample.txt
[INFO] [stdout]     1: S3 HEAD request failed for 'test-data/sample.txt': dispatch failure
[INFO] [stdout] 
[INFO] [stdout] Stack backtrace:
[INFO] [stdout]    0: anyhow::error::<impl anyhow::Error>::msg
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/anyhow-1.0.99/src/backtrace.rs:27:14
[INFO] [stdout]    1: ai_workbench_lib::modules::file_discovery::FileDiscovery::get_file_metadata::{{closure}}
[INFO] [stdout]              at ./src/../modules/file_discovery.rs:267:28
[INFO] [stdout]    2: ai_workbench_lib::modules::file_discovery::FileDiscovery::process_single_file::{{closure}}
[INFO] [stdout]              at ./src/../modules/file_discovery.rs:59:64
[INFO] [stdout]    3: ai_workbench_lib::modules::file_discovery::FileDiscovery::discover_files::{{closure}}
[INFO] [stdout]              at ./src/../modules/file_discovery.rs:35:50
[INFO] [stdout]    4: ai_workbench_lib::modules::file_discovery::file_discovery_tests::test_file_type_detection::{{closure}}
[INFO] [stdout]              at ./src/../modules/file_discovery.rs:516:61
[INFO] [stdout]    5: <core::pin::Pin<P> as core::future::future::Future>::poll
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/future/future.rs:133:9
[INFO] [stdout]    6: <core::pin::Pin<P> as core::future::future::Future>::poll
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/future/future.rs:133:9
[INFO] [stdout]    7: tokio::runtime::scheduler::current_thread::CoreGuard::block_on::{{closure}}::{{closure}}::{{closure}}
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:742:70
[INFO] [stdout]    8: tokio::task::coop::with_budget
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/task/coop/mod.rs:167:5
[INFO] [stdout]    9: tokio::task::coop::budget
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/task/coop/mod.rs:133:5
[INFO] [stdout]   10: tokio::runtime::scheduler::current_thread::CoreGuard::block_on::{{closure}}::{{closure}}
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:742:25
[INFO] [stdout]   11: tokio::runtime::scheduler::current_thread::Context::enter
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:432:19
[INFO] [stdout]   12: tokio::runtime::scheduler::current_thread::CoreGuard::block_on::{{closure}}
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:741:44
[INFO] [stdout]   13: tokio::runtime::scheduler::current_thread::CoreGuard::enter::{{closure}}
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:829:68
[INFO] [stdout]   14: tokio::runtime::context::scoped::Scoped<T>::set
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/context/scoped.rs:40:9
[INFO] [stdout]   15: tokio::runtime::context::set_scheduler::{{closure}}
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/context.rs:176:38
[INFO] [stdout]   16: std::thread::local::LocalKey<T>::try_with
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/thread/local.rs:508:12
[INFO] [stdout]   17: std::thread::local::LocalKey<T>::with
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/thread/local.rs:472:20
[INFO] [stdout]   18: tokio::runtime::context::set_scheduler
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/context.rs:176:17
[INFO] [stdout]   19: tokio::runtime::scheduler::current_thread::CoreGuard::enter
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:829:27
[INFO] [stdout]   20: tokio::runtime::scheduler::current_thread::CoreGuard::block_on
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:729:24
[INFO] [stdout]   21: tokio::runtime::scheduler::current_thread::CurrentThread::block_on::{{closure}}
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:200:33
[INFO] [stdout]   22: tokio::runtime::context::runtime::enter_runtime
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/context/runtime.rs:65:16
[INFO] [stdout]   23: tokio::runtime::scheduler::current_thread::CurrentThread::block_on
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:188:9
[INFO] [stdout]   24: tokio::runtime::runtime::Runtime::block_on_inner
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/runtime.rs:356:52
[INFO] [stdout]   25: tokio::runtime::runtime::Runtime::block_on
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/runtime.rs:330:18
[INFO] [stdout]   26: ai_workbench_lib::modules::file_discovery::file_discovery_tests::test_file_type_detection
[INFO] [stdout]              at ./src/../modules/file_discovery.rs:515:36
[INFO] [stdout]   27: ai_workbench_lib::modules::file_discovery::file_discovery_tests::test_file_type_detection::{{closure}}
[INFO] [stdout]              at ./src/../modules/file_discovery.rs:498:40
[INFO] [stdout]   28: core::ops::function::FnOnce::call_once
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/ops/function.rs:250:5
[INFO] [stdout]   29: core::ops::function::FnOnce::call_once
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/ops/function.rs:250:5
[INFO] [stdout]   30: test::__rust_begin_short_backtrace
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/test/src/lib.rs:663:18
[INFO] [stdout]   31: test::run_test_in_process::{{closure}}
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/test/src/lib.rs:686:74
[INFO] [stdout]   32: <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/panic/unwind_safe.rs:274:9
[INFO] [stdout]   33: std::panicking::catch_unwind::do_call
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panicking.rs:590:40
[INFO] [stdout]   34: std::panicking::catch_unwind
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panicking.rs:553:19
[INFO] [stdout]   35: std::panic::catch_unwind
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panic.rs:359:14
[INFO] [stdout]   36: test::run_test_in_process
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/test/src/lib.rs:686:27
[INFO] [stdout]   37: test::run_test::{{closure}}
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/test/src/lib.rs:607:43
[INFO] [stdout]   38: test::run_test::{{closure}}
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/test/src/lib.rs:637:41
[INFO] [stdout]   39: std::sys::backtrace::__rust_begin_short_backtrace
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/sys/backtrace.rs:158:18
[INFO] [stdout]   40: std::thread::Builder::spawn_unchecked_::{{closure}}::{{closure}}
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/thread/mod.rs:562:17
[INFO] [stdout]   41: <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/panic/unwind_safe.rs:274:9
[INFO] [stdout]   42: std::panicking::catch_unwind::do_call
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panicking.rs:590:40
[INFO] [stdout]   43: std::panicking::catch_unwind
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panicking.rs:553:19
[INFO] [stdout]   44: std::panic::catch_unwind
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panic.rs:359:14
[INFO] [stdout]   45: std::thread::Builder::spawn_unchecked_::{{closure}}
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/thread/mod.rs:560:30
[INFO] [stdout]   46: core::ops::function::FnOnce::call_once{{vtable.shim}}
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/ops/function.rs:250:5
[INFO] [stdout]   47: <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/alloc/src/boxed.rs:2085:9
[INFO] [stdout]   48: std::sys::thread::unix::Thread::new::thread_start
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/sys/thread/unix.rs:124:17
[INFO] [stdout]   49: <unknown>
[INFO] [stdout]   50: clone
[INFO] [stdout] [2m2025-11-11T07:22:52.381337Z[0m [32m INFO[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m === FileDiscovery::discover_files DEBUG END ===
[INFO] [stdout] [2m2025-11-11T07:22:52.381374Z[0m [32m INFO[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m === FileDiscovery::discover_files DEBUG START ===
[INFO] [stdout] [2m2025-11-11T07:22:52.381386Z[0m [32m INFO[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m Input spec: 'test-data/data.csv'
[INFO] [stdout] [2m2025-11-11T07:22:52.381395Z[0m [32m INFO[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m Workspace bucket: 'ai-workbench-6c9c43db-7fe6-42f1-8b11-8f82323f83f0-eu-west-2'
[INFO] [stdout] [2m2025-11-11T07:22:52.381406Z[0m [32m INFO[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m Detected single file input (no trailing '/') - processing as single file
[INFO] [stdout] [2m2025-11-11T07:22:52.381420Z[0m [32m INFO[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m === process_single_file DEBUG START ===
[INFO] [stdout] [2m2025-11-11T07:22:52.381430Z[0m [32m INFO[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m Processing single file: 'test-data/data.csv'
[INFO] [stdout] [2m2025-11-11T07:22:52.381439Z[0m [32m INFO[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m Full S3 path: s3://ai-workbench-6c9c43db-7fe6-42f1-8b11-8f82323f83f0-eu-west-2/test-data/data.csv
[INFO] [stdout] [2m2025-11-11T07:22:52.381450Z[0m [32m INFO[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m Attempting to get file metadata...
[INFO] [stdout] [2m2025-11-11T07:22:52.381461Z[0m [32m INFO[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m === get_file_metadata DEBUG START ===
[INFO] [stdout] [2m2025-11-11T07:22:52.381471Z[0m [32m INFO[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m Getting metadata for file: 'test-data/data.csv'
[INFO] [stdout] [2m2025-11-11T07:22:52.381480Z[0m [32m INFO[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m Bucket: 'ai-workbench-6c9c43db-7fe6-42f1-8b11-8f82323f83f0-eu-west-2'
[INFO] [stdout] [2m2025-11-11T07:22:52.381489Z[0m [32m INFO[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m Full S3 path: s3://ai-workbench-6c9c43db-7fe6-42f1-8b11-8f82323f83f0-eu-west-2/test-data/data.csv
[INFO] [stdout] [2m2025-11-11T07:22:52.381500Z[0m [32m INFO[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m Sending HEAD request to S3...
[INFO] [stdout] [2m2025-11-11T07:22:52.383354Z[0m [31mERROR[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m S3 HEAD request failed for 'text_files/0c472776-7c22-464b-93bf-3714fd229b01.txt': dispatch failure
[INFO] [stdout] [2m2025-11-11T07:22:52.383392Z[0m [31mERROR[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m Error details: DispatchFailure(DispatchFailure { source: ConnectorError { kind: Other(None), source: CredentialsNotLoaded(CredentialsNotLoaded { source: Some("no providers in chain provided credentials") }), connection: Unknown } })
[INFO] [stdout] [2m2025-11-11T07:22:52.383432Z[0m [31mERROR[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m Possible causes:
[INFO] [stdout] [2m2025-11-11T07:22:52.383441Z[0m [31mERROR[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m   1. File 'text_files/0c472776-7c22-464b-93bf-3714fd229b01.txt' does not exist in bucket 'ai-workbench-6c9c43db-7fe6-42f1-8b11-8f82323f83f0-eu-west-2'
[INFO] [stdout] [2m2025-11-11T07:22:52.383455Z[0m [31mERROR[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m   2. No read permissions for this file/bucket
[INFO] [stdout] [2m2025-11-11T07:22:52.383465Z[0m [31mERROR[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m   3. AWS credentials are invalid or expired
[INFO] [stdout] [2m2025-11-11T07:22:52.383475Z[0m [31mERROR[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m   4. Network connectivity issues
[INFO] [stdout] [2m2025-11-11T07:22:52.383484Z[0m [31mERROR[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m   5. S3 service is temporarily unavailable
[INFO] [stdout] [2m2025-11-11T07:22:52.383546Z[0m [31mERROR[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m Failed to get metadata for file 'text_files/0c472776-7c22-464b-93bf-3714fd229b01.txt': Failed to get metadata for text_files/0c472776-7c22-464b-93bf-3714fd229b01.txt
[INFO] [stdout] [2m2025-11-11T07:22:52.383565Z[0m [31mERROR[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m This usually means:
[INFO] [stdout] [2m2025-11-11T07:22:52.383573Z[0m [31mERROR[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m   1. File does not exist at s3://ai-workbench-6c9c43db-7fe6-42f1-8b11-8f82323f83f0-eu-west-2/text_files/0c472776-7c22-464b-93bf-3714fd229b01.txt
[INFO] [stdout] [2m2025-11-11T07:22:52.383587Z[0m [31mERROR[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m   2. No permissions to access the file
[INFO] [stdout] [2m2025-11-11T07:22:52.383595Z[0m [31mERROR[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m   3. S3 service is unavailable
[INFO] [stdout] [2m2025-11-11T07:22:52.383604Z[0m [31mERROR[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m   4. Incorrect bucket name or file path
[INFO] [stdout] [2m2025-11-11T07:22:52.383615Z[0m [31mERROR[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m discover_files failed: Failed to get metadata for single file: text_files/0c472776-7c22-464b-93bf-3714fd229b01.txt
[INFO] [stdout] [2m2025-11-11T07:22:52.384015Z[0m [31mERROR[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m S3 HEAD request failed for 'test-data/data.csv': dispatch failure
[INFO] [stdout] [2m2025-11-11T07:22:52.384035Z[0m [31mERROR[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m Error details: DispatchFailure(DispatchFailure { source: ConnectorError { kind: Other(None), source: CredentialsNotLoaded(CredentialsNotLoaded { source: Some("no providers in chain provided credentials") }), connection: Unknown } })
[INFO] [stdout] [2m2025-11-11T07:22:52.384057Z[0m [31mERROR[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m Possible causes:
[INFO] [stdout] [2m2025-11-11T07:22:52.384065Z[0m [31mERROR[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m   1. File 'test-data/data.csv' does not exist in bucket 'ai-workbench-6c9c43db-7fe6-42f1-8b11-8f82323f83f0-eu-west-2'
[INFO] [stdout] [2m2025-11-11T07:22:52.384125Z[0m [31mERROR[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m   2. No read permissions for this file/bucket
[INFO] [stdout] [2m2025-11-11T07:22:52.384134Z[0m [31mERROR[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m   3. AWS credentials are invalid or expired
[INFO] [stdout] [2m2025-11-11T07:22:52.384143Z[0m [31mERROR[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m   4. Network connectivity issues
[INFO] [stdout] [2m2025-11-11T07:22:52.384155Z[0m [31mERROR[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m   5. S3 service is temporarily unavailable
[INFO] [stdout] [2m2025-11-11T07:22:52.384208Z[0m [31mERROR[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m Failed to get metadata for file 'test-data/data.csv': Failed to get metadata for test-data/data.csv
[INFO] [stdout] [2m2025-11-11T07:22:52.384223Z[0m [31mERROR[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m This usually means:
[INFO] [stdout] [2m2025-11-11T07:22:52.384229Z[0m [31mERROR[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m   1. File does not exist at s3://ai-workbench-6c9c43db-7fe6-42f1-8b11-8f82323f83f0-eu-west-2/test-data/data.csv
[INFO] [stdout] [2m2025-11-11T07:22:52.384241Z[0m [31mERROR[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m   2. No permissions to access the file
[INFO] [stdout] [2m2025-11-11T07:22:52.384249Z[0m [31mERROR[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m   3. S3 service is unavailable
[INFO] [stdout] [2m2025-11-11T07:22:52.384257Z[0m [31mERROR[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m   4. Incorrect bucket name or file path
[INFO] [stdout] [2m2025-11-11T07:22:52.384266Z[0m [31mERROR[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m discover_files failed: Failed to get metadata for single file: test-data/data.csv
[INFO] [stdout] [2m2025-11-11T07:22:52.383627Z[0m [31mERROR[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m Error details: Failed to get metadata for single file: text_files/0c472776-7c22-464b-93bf-3714fd229b01.txt
[INFO] [stdout] 
[INFO] [stdout] Caused by:
[INFO] [stdout]     0: Failed to get metadata for text_files/0c472776-7c22-464b-93bf-3714fd229b01.txt
[INFO] [stdout]     1: S3 HEAD request failed for 'text_files/0c472776-7c22-464b-93bf-3714fd229b01.txt': dispatch failure
[INFO] [stdout] 
[INFO] [stdout] Stack backtrace:
[INFO] [stdout]    0: anyhow::error::<impl anyhow::Error>::msg
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/anyhow-1.0.99/src/backtrace.rs:27:14
[INFO] [stdout]    1: ai_workbench_lib::modules::file_discovery::FileDiscovery::get_file_metadata::{{closure}}
[INFO] [stdout]              at ./src/../modules/file_discovery.rs:267:28
[INFO] [stdout]    2: ai_workbench_lib::modules::file_discovery::FileDiscovery::process_single_file::{{closure}}
[INFO] [stdout]              at ./src/../modules/file_discovery.rs:59:64
[INFO] [stdout]    3: ai_workbench_lib::modules::file_discovery::FileDiscovery::discover_files::{{closure}}
[INFO] [stdout]              at ./src/../modules/file_discovery.rs:35:50
[INFO] [stdout]    4: ai_workbench_lib::modules::job_processor::core::JobProcessor::discover_files::{{closure}}
[INFO] [stdout]              at ./src/../modules/job_processor/core.rs:102:83
[INFO] [stdout]    5: ai_workbench_lib::modules::job_processor::core::job_processor_tests::test_job_processor_single_file_end_to_end::{{closure}}
[INFO] [stdout]              at ./src/../modules/job_processor/core.rs:673:55
[INFO] [stdout]    6: <core::pin::Pin<P> as core::future::future::Future>::poll
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/future/future.rs:133:9
[INFO] [stdout]    7: <core::pin::Pin<P> as core::future::future::Future>::poll
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/future/future.rs:133:9
[INFO] [stdout]    8: tokio::runtime::scheduler::current_thread::CoreGuard::block_on::{{closure}}::{{closure}}::{{closure}}
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:742:70
[INFO] [stdout]    9: tokio::task::coop::with_budget
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/task/coop/mod.rs:167:5
[INFO] [stdout]   10: tokio::task::coop::budget
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/task/coop/mod.rs:133:5
[INFO] [stdout]   11: tokio::runtime::scheduler::current_thread::CoreGuard::block_on::{{closure}}::{{closure}}
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:742:25
[INFO] [stdout]   12: tokio::runtime::scheduler::current_thread::Context::enter
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:432:19
[INFO] [stdout]   13: tokio::runtime::scheduler::current_thread::CoreGuard::block_on::{{closure}}
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:741:44
[INFO] [stdout]   14: tokio::runtime::scheduler::current_thread::CoreGuard::enter::{{closure}}
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:829:68
[INFO] [stdout]   15: tokio::runtime::context::scoped::Scoped<T>::set
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/context/scoped.rs:40:9
[INFO] [stdout]   16: tokio::runtime::context::set_scheduler::{{closure}}
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/context.rs:176:38
[INFO] [stdout]   17: std::thread::local::LocalKey<T>::try_with
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/thread/local.rs:508:12
[INFO] [stdout]   18: std::thread::local::LocalKey<T>::with
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/thread/local.rs:472:20
[INFO] [stdout]   19: tokio::runtime::context::set_scheduler
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/context.rs:176:17
[INFO] [stdout]   20: tokio::runtime::scheduler::current_thread::CoreGuard::enter
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:829:27
[INFO] [stdout]   21: tokio::runtime::scheduler::current_thread::CoreGuard::block_on
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:729:24
[INFO] [stdout]   22: tokio::runtime::scheduler::current_thread::CurrentThread::block_on::{{closure}}
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:200:33
[INFO] [stdout]   23: tokio::runtime::context::runtime::enter_runtime
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/context/runtime.rs:65:16
[INFO] [stdout]   24: tokio::runtime::scheduler::current_thread::CurrentThread::block_on
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:188:9
[INFO] [stdout]   25: tokio::runtime::runtime::Runtime::block_on_inner
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/runtime.rs:356:52
[INFO] [stdout]   26: tokio::runtime::runtime::Runtime::block_on
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/runtime.rs:330:18
[INFO] [stdout]   27: ai_workbench_lib::modules::job_processor::core::job_processor_tests::test_job_processor_single_file_end_to_end
[INFO] [stdout]              at ./src/../modules/job_processor/core.rs:710:37
[INFO] [stdout]   28: ai_workbench_lib::modules::job_processor::core::job_processor_tests::test_job_processor_single_file_end_to_end::{{closure}}
[INFO] [stdout]              at ./src/../modules/job_processor/core.rs:638:57
[INFO] [stdout]   29: core::ops::function::FnOnce::call_once
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/ops/function.rs:250:5
[INFO] [stdout]   30: core::ops::function::FnOnce::call_once
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/ops/function.rs:250:5
[INFO] [stdout]   31: test::__rust_begin_short_backtrace
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/test/src/lib.rs:663:18
[INFO] [stdout]   32: test::run_test_in_process::{{closure}}
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/test/src/lib.rs:686:74
[INFO] [stdout]   33: <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/panic/unwind_safe.rs:274:9
[INFO] [stdout]   34: std::panicking::catch_unwind::do_call
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panicking.rs:590:40
[INFO] [stdout]   35: std::panicking::catch_unwind
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panicking.rs:553:19
[INFO] [stdout]   36: std::panic::catch_unwind
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panic.rs:359:14
[INFO] [stdout]   37: test::run_test_in_process
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/test/src/lib.rs:686:27
[INFO] [stdout]   38: test::run_test::{{closure}}
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/test/src/lib.rs:607:43
[INFO] [stdout]   39: test::run_test::{{closure}}
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/test/src/lib.rs:637:41
[INFO] [stdout]   40: std::sys::backtrace::__rust_begin_short_backtrace
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/sys/backtrace.rs:158:18
[INFO] [stdout]   41: std::thread::Builder::spawn_unchecked_::{{closure}}::{{closure}}
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/thread/mod.rs:562:17
[INFO] [stdout]   42: <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/panic/unwind_safe.rs:274:9
[INFO] [stdout]   43: std::panicking::catch_unwind::do_call
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panicking.rs:590:40
[INFO] [stdout]   44: std::panicking::catch_unwind
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panicking.rs:553:19
[INFO] [stdout]   45: std::panic::catch_unwind
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panic.rs:359:14
[INFO] [stdout]   46: std::thread::Builder::spawn_unchecked_::{{closure}}
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/thread/mod.rs:560:30
[INFO] [stdout]   47: core::ops::function::FnOnce::call_once{{vtable.shim}}
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/ops/function.rs:250:5
[INFO] [stdout]   48: <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/alloc/src/boxed.rs:2085:9
[INFO] [stdout]   49: std::sys::thread::unix::Thread::new::thread_start
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/sys/thread/unix.rs:124:17
[INFO] [stdout]   50: <unknown>
[INFO] [stdout]   51: clone
[INFO] [stdout] [2m2025-11-11T07:22:52.384393Z[0m [32m INFO[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m === FileDiscovery::discover_files DEBUG END ===
[INFO] [stdout] [2m2025-11-11T07:22:52.384431Z[0m [31mERROR[0m [2mai_workbench_lib::modules::job_processor::core[0m[2m:[0m File discovery failed with error: Failed to get metadata for single file: text_files/0c472776-7c22-464b-93bf-3714fd229b01.txt
[INFO] [stdout] [2m2025-11-11T07:22:52.384277Z[0m [31mERROR[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m Error details: Failed to get metadata for single file: test-data/data.csv
[INFO] [stdout] 
[INFO] [stdout] Caused by:
[INFO] [stdout]     0: Failed to get metadata for test-data/data.csv
[INFO] [stdout]     1: S3 HEAD request failed for 'test-data/data.csv': dispatch failure
[INFO] [stdout] 
[INFO] [stdout] Stack backtrace:
[INFO] [stdout]    0: anyhow::error::<impl anyhow::Error>::msg
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/anyhow-1.0.99/src/backtrace.rs:27:14
[INFO] [stdout]    1: ai_workbench_lib::modules::file_discovery::FileDiscovery::get_file_metadata::{{closure}}
[INFO] [stdout]              at ./src/../modules/file_discovery.rs:267:28
[INFO] [stdout]    2: ai_workbench_lib::modules::file_discovery::FileDiscovery::process_single_file::{{closure}}
[INFO] [stdout]              at ./src/../modules/file_discovery.rs:59:64
[INFO] [stdout]    3: ai_workbench_lib::modules::file_discovery::FileDiscovery::discover_files::{{closure}}
[INFO] [stdout]              at ./src/../modules/file_discovery.rs:35:50
[INFO] [stdout]    4: ai_workbench_lib::modules::file_discovery::file_discovery_tests::test_file_type_detection::{{closure}}
[INFO] [stdout]              at ./src/../modules/file_discovery.rs:516:61
[INFO] [stdout]    5: <core::pin::Pin<P> as core::future::future::Future>::poll
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/future/future.rs:133:9
[INFO] [stdout]    6: <core::pin::Pin<P> as core::future::future::Future>::poll
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/future/future.rs:133:9
[INFO] [stdout]    7: tokio::runtime::scheduler::current_thread::CoreGuard::block_on::{{closure}}::{{closure}}::{{closure}}
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:742:70
[INFO] [stdout]    8: tokio::task::coop::with_budget
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/task/coop/mod.rs:167:5
[INFO] [stdout]    9: tokio::task::coop::budget
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/task/coop/mod.rs:133:5
[INFO] [stdout]   10: tokio::runtime::scheduler::current_thread::CoreGuard::block_on::{{closure}}::{{closure}}
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:742:25
[INFO] [stdout]   11: tokio::runtime::scheduler::current_thread::Context::enter
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:432:19
[INFO] [stdout]   12: tokio::runtime::scheduler::current_thread::CoreGuard::block_on::{{closure}}
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:741:44
[INFO] [stdout]   13: tokio::runtime::scheduler::current_thread::CoreGuard::enter::{{closure}}
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:829:68
[INFO] [stdout]   14: tokio::runtime::context::scoped::Scoped<T>::set
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/context/scoped.rs:40:9
[INFO] [stdout]   15: tokio::runtime::context::set_scheduler::{{closure}}
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/context.rs:176:38
[INFO] [stdout]   16: std::thread::local::LocalKey<T>::try_with
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/thread/local.rs:508:12
[INFO] [stdout]   17: std::thread::local::LocalKey<T>::with
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/thread/local.rs:472:20
[INFO] [stdout]   18: tokio::runtime::context::set_scheduler
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/context.rs:176:17
[INFO] [stdout]   19: tokio::runtime::scheduler::current_thread::CoreGuard::enter
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:829:27
[INFO] [stdout]   20: tokio::runtime::scheduler::current_thread::CoreGuard::block_on
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:729:24
[INFO] [stdout]   21: tokio::runtime::scheduler::current_thread::CurrentThread::block_on::{{closure}}
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:200:33
[INFO] [stdout]   22: tokio::runtime::context::runtime::enter_runtime
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/context/runtime.rs:65:16
[INFO] [stdout]   23: tokio::runtime::scheduler::current_thread::CurrentThread::block_on
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:188:9
[INFO] [stdout]   24: tokio::runtime::runtime::Runtime::block_on_inner
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/runtime.rs:356:52
[INFO] [stdout]   25: tokio::runtime::runtime::Runtime::block_on
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/runtime.rs:330:18
[INFO] [stdout]   26: ai_workbench_lib::modules::file_discovery::file_discovery_tests::test_file_type_detection
[INFO] [stdout]              at ./src/../modules/file_discovery.rs:515:36
[INFO] [stdout]   27: ai_workbench_lib::modules::file_discovery::file_discovery_tests::test_file_type_detection::{{closure}}
[INFO] [stdout]              at ./src/../modules/file_discovery.rs:498:40
[INFO] [stdout]   28: core::ops::function::FnOnce::call_once
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/ops/function.rs:250:5
[INFO] [stdout]   29: core::ops::function::FnOnce::call_once
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/ops/function.rs:250:5
[INFO] [stdout]   30: test::__rust_begin_short_backtrace
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/test/src/lib.rs:663:18
[INFO] [stdout]   31: test::run_test_in_process::{{closure}}
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/test/src/lib.rs:686:74
[INFO] [stdout]   32: <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/panic/unwind_safe.rs:274:9
[INFO] [stdout]   33: std::panicking::catch_unwind::do_call
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panicking.rs:590:40
[INFO] [stdout]   34: std::panicking::catch_unwind
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panicking.rs:553:19
[INFO] [stdout]   35: std::panic::catch_unwind
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panic.rs:359:14
[INFO] [stdout]   36: test::run_test_in_process
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/test/src/lib.rs:686:27
[INFO] [stdout]   37: test::run_test::{{closure}}
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/test/src/lib.rs:607:43
[INFO] [stdout]   38: test::run_test::{{closure}}
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/test/src/lib.rs:637:41
[INFO] [stdout]   39: std::sys::backtrace::__rust_begin_short_backtrace
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/sys/backtrace.rs:158:18
[INFO] [stdout]   40: std::thread::Builder::spawn_unchecked_::{{closure}}::{{closure}}
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/thread/mod.rs:562:17
[INFO] [stdout]   41: <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/panic/unwind_safe.rs:274:9
[INFO] [stdout]   42: std::panicking::catch_unwind::do_call
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panicking.rs:590:40
[INFO] [stdout]   43: std::panicking::catch_unwind
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panicking.rs:553:19
[INFO] [stdout]   44: std::panic::catch_unwind
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panic.rs:359:14
[INFO] [stdout]   45: std::thread::Builder::spawn_unchecked_::{{closure}}
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/thread/mod.rs:560:30
[INFO] [stdout]   46: core::ops::function::FnOnce::call_once{{vtable.shim}}
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/ops/function.rs:250:5
[INFO] [stdout]   47: <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/alloc/src/boxed.rs:2085:9
[INFO] [stdout]   48: std::sys::thread::unix::Thread::new::thread_start
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/sys/thread/unix.rs:124:17
[INFO] [stdout]   49: <unknown>
[INFO] [stdout]   50: clone
[INFO] [stdout] [2m2025-11-11T07:22:52.384928Z[0m [32m INFO[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m === FileDiscovery::discover_files DEBUG END ===
[INFO] [stdout] [2m2025-11-11T07:22:52.384447Z[0m [31mERROR[0m [2mai_workbench_lib::modules::job_processor::core[0m[2m:[0m Error context: Failed to get metadata for single file: text_files/0c472776-7c22-464b-93bf-3714fd229b01.txt
[INFO] [stdout] 
[INFO] [stdout] Caused by:
[INFO] [stdout]     0: Failed to get metadata for text_files/0c472776-7c22-464b-93bf-3714fd229b01.txt
[INFO] [stdout]     1: S3 HEAD request failed for 'text_files/0c472776-7c22-464b-93bf-3714fd229b01.txt': dispatch failure
[INFO] [stdout] 
[INFO] [stdout] Stack backtrace:
[INFO] [stdout]    0: anyhow::error::<impl anyhow::Error>::msg
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/anyhow-1.0.99/src/backtrace.rs:27:14
[INFO] [stdout]    1: ai_workbench_lib::modules::file_discovery::FileDiscovery::get_file_metadata::{{closure}}
[INFO] [stdout]              at ./src/../modules/file_discovery.rs:267:28
[INFO] [stdout]    2: ai_workbench_lib::modules::file_discovery::FileDiscovery::process_single_file::{{closure}}
[INFO] [stdout]              at ./src/../modules/file_discovery.rs:59:64
[INFO] [stdout]    3: ai_workbench_lib::modules::file_discovery::FileDiscovery::discover_files::{{closure}}
[INFO] [stdout]              at ./src/../modules/file_discovery.rs:35:50
[INFO] [stdout]    4: ai_workbench_lib::modules::job_processor::core::JobProcessor::discover_files::{{closure}}
[INFO] [stdout]              at ./src/../modules/job_processor/core.rs:102:83
[INFO] [stdout]    5: ai_workbench_lib::modules::job_processor::core::job_processor_tests::test_job_processor_single_file_end_to_end::{{closure}}
[INFO] [stdout]              at ./src/../modules/job_processor/core.rs:673:55
[INFO] [stdout]    6: <core::pin::Pin<P> as core::future::future::Future>::poll
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/future/future.rs:133:9
[INFO] [stdout]    7: <core::pin::Pin<P> as core::future::future::Future>::poll
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/future/future.rs:133:9
[INFO] [stdout]    8: tokio::runtime::scheduler::current_thread::CoreGuard::block_on::{{closure}}::{{closure}}::{{closure}}
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:742:70
[INFO] [stdout]    9: tokio::task::coop::with_budget
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/task/coop/mod.rs:167:5
[INFO] [stdout]   10: tokio::task::coop::budget
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/task/coop/mod.rs:133:5
[INFO] [stdout]   11: tokio::runtime::scheduler::current_thread::CoreGuard::block_on::{{closure}}::{{closure}}
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:742:25
[INFO] [stdout]   12: tokio::runtime::scheduler::current_thread::Context::enter
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:432:19
[INFO] [stdout]   13: tokio::runtime::scheduler::current_thread::CoreGuard::block_on::{{closure}}
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:741:44
[INFO] [stdout]   14: tokio::runtime::scheduler::current_thread::CoreGuard::enter::{{closure}}
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:829:68
[INFO] [stdout]   15: tokio::runtime::context::scoped::Scoped<T>::set
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/context/scoped.rs:40:9
[INFO] [stdout]   16: tokio::runtime::context::set_scheduler::{{closure}}
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/context.rs:176:38
[INFO] [stdout]   17: std::thread::local::LocalKey<T>::try_with
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/thread/local.rs:508:12
[INFO] [stdout]   18: std::thread::local::LocalKey<T>::with
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/thread/local.rs:472:20
[INFO] [stdout]   19: tokio::runtime::context::set_scheduler
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/context.rs:176:17
[INFO] [stdout]   20: tokio::runtime::scheduler::current_thread::CoreGuard::enter
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:829:27
[INFO] [stdout]   21: tokio::runtime::scheduler::current_thread::CoreGuard::block_on
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:729:24
[INFO] [stdout]   22: tokio::runtime::scheduler::current_thread::CurrentThread::block_on::{{closure}}
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:200:33
[INFO] [stdout]   23: tokio::runtime::context::runtime::enter_runtime
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/context/runtime.rs:65:16
[INFO] [stdout]   24: tokio::runtime::scheduler::current_thread::CurrentThread::block_on
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:188:9
[INFO] [stdout]   25: tokio::runtime::runtime::Runtime::block_on_inner
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/runtime.rs:356:52
[INFO] [stdout]   26: tokio::runtime::runtime::Runtime::block_on
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/runtime.rs:330:18
[INFO] [stdout]   27: ai_workbench_lib::modules::job_processor::core::job_processor_tests::test_job_processor_single_file_end_to_end
[INFO] [stdout]              at ./src/../modules/job_processor/core.rs:710:37
[INFO] [stdout]   28: ai_workbench_lib::modules::job_processor::core::job_processor_tests::test_job_processor_single_file_end_to_end::{{closure}}
[INFO] [stdout]              at ./src/../modules/job_processor/core.rs:638:57
[INFO] [stdout]   29: core::ops::function::FnOnce::call_once
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/ops/function.rs:250:5
[INFO] [stdout]   30: core::ops::function::FnOnce::call_once
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/ops/function.rs:250:5
[INFO] [stdout]   31: test::__rust_begin_short_backtrace
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/test/src/lib.rs:663:18
[INFO] [stdout]   32: test::run_test_in_process::{{closure}}
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/test/src/lib.rs:686:74
[INFO] [stdout]   33: <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/panic/unwind_safe.rs:274:9
[INFO] [stdout]   34: std::panicking::catch_unwind::do_call
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panicking.rs:590:40
[INFO] [stdout]   35: std::panicking::catch_unwind
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panicking.rs:553:19
[INFO] [stdout]   36: std::panic::catch_unwind
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panic.rs:359:14
[INFO] [stdout]   37: test::run_test_in_process
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/test/src/lib.rs:686:27
[INFO] [stdout]   38: test::run_test::{{closure}}
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/test/src/lib.rs:607:43
[INFO] [stdout]   39: test::run_test::{{closure}}
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/test/src/lib.rs:637:41
[INFO] [stdout]   40: std::sys::backtrace::__rust_begin_short_backtrace
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/sys/backtrace.rs:158:18
[INFO] [stdout]   41: std::thread::Builder::spawn_unchecked_::{{closure}}::{{closure}}
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/thread/mod.rs:562:17
[INFO] [stdout]   42: <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/panic/unwind_safe.rs:274:9
[INFO] [stdout]   43: std::panicking::catch_unwind::do_call
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panicking.rs:590:40
[INFO] [stdout]   44: std::panicking::catch_unwind
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panicking.rs:553:19
[INFO] [stdout]   45: std::panic::catch_unwind
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panic.rs:359:14
[INFO] [stdout]   46: std::thread::Builder::spawn_unchecked_::{{closure}}
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/thread/mod.rs:560:30
[INFO] [stdout]   47: core::ops::function::FnOnce::call_once{{vtable.shim}}
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/ops/function.rs:250:5
[INFO] [stdout]   48: <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/alloc/src/boxed.rs:2085:9
[INFO] [stdout]   49: std::sys::thread::unix::Thread::new::thread_start
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/sys/thread/unix.rs:124:17
[INFO] [stdout]   50: <unknown>
[INFO] [stdout]   51: clone
[INFO] [stdout] [2m2025-11-11T07:22:52.384957Z[0m [32m INFO[0m [2mai_workbench_lib::modules::job_processor::core[0m[2m:[0m === FILE DISCOVERY DEBUG END (FAILED) ===
[INFO] [stdout] [2m2025-11-11T07:22:52.384961Z[0m [32m INFO[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m === FileDiscovery::discover_files DEBUG START ===
[INFO] [stdout] [2m2025-11-11T07:22:52.384973Z[0m [32m INFO[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m Input spec: 'test-data/config.json'
[INFO] [stdout] [2m2025-11-11T07:22:52.384981Z[0m [32m INFO[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m Workspace bucket: 'ai-workbench-6c9c43db-7fe6-42f1-8b11-8f82323f83f0-eu-west-2'
[INFO] [stdout] [2m2025-11-11T07:22:52.384991Z[0m [32m INFO[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m Detected single file input (no trailing '/') - processing as single file
[INFO] [stdout] [2m2025-11-11T07:22:52.385002Z[0m [32m INFO[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m === process_single_file DEBUG START ===
[INFO] [stdout] [2m2025-11-11T07:22:52.385010Z[0m [32m INFO[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m Processing single file: 'test-data/config.json'
[INFO] [stdout] [2m2025-11-11T07:22:52.385018Z[0m [32m INFO[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m Full S3 path: s3://ai-workbench-6c9c43db-7fe6-42f1-8b11-8f82323f83f0-eu-west-2/test-data/config.json
[INFO] [stdout] [2m2025-11-11T07:22:52.385029Z[0m [32m INFO[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m Attempting to get file metadata...
[INFO] [stdout] [2m2025-11-11T07:22:52.385038Z[0m [32m INFO[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m === get_file_metadata DEBUG START ===
[INFO] [stdout] [2m2025-11-11T07:22:52.385046Z[0m [32m INFO[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m Getting metadata for file: 'test-data/config.json'
[INFO] [stdout] [2m2025-11-11T07:22:52.385056Z[0m [32m INFO[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m Bucket: 'ai-workbench-6c9c43db-7fe6-42f1-8b11-8f82323f83f0-eu-west-2'
[INFO] [stdout] test modules::file_discovery::file_discovery_tests::test_folder_discovery ... ok
[INFO] [stdout] [2m2025-11-11T07:22:52.379185Z[0m [31mERROR[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m Error details: Failed to list objects in S3
[INFO] [stdout] 
[INFO] [stdout] Caused by:
[INFO] [stdout]     S3 ListObjectsV2 failed for folder 'large-dataset/': dispatch failure
[INFO] [stdout] 
[INFO] [stdout] Stack backtrace:
[INFO] [stdout]    0: anyhow::error::<impl anyhow::Error>::msg
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/anyhow-1.0.99/src/backtrace.rs:27:14
[INFO] [stdout]    1: ai_workbench_lib::modules::file_discovery::FileDiscovery::process_folder::{{closure}}
[INFO] [stdout]              at ./src/../modules/file_discovery.rs:157:32
[INFO] [stdout]    2: ai_workbench_lib::modules::file_discovery::FileDiscovery::discover_files::{{closure}}
[INFO] [stdout]              at ./src/../modules/file_discovery.rs:31:45
[INFO] [stdout]    3: ai_workbench_lib::modules::file_discovery::file_discovery_tests::test_large_folder_discovery::{{closure}}
[INFO] [stdout]              at ./src/../modules/file_discovery.rs:551:61
[INFO] [stdout]    4: <core::pin::Pin<P> as core::future::future::Future>::poll
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/future/future.rs:133:9
[INFO] [stdout]    5: <core::pin::Pin<P> as core::future::future::Future>::poll
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/future/future.rs:133:9
[INFO] [stdout]    6: tokio::runtime::scheduler::current_thread::CoreGuard::block_on::{{closure}}::{{closure}}::{{closure}}
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:742:70
[INFO] [stdout]    7: tokio::task::coop::with_budget
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/task/coop/mod.rs:167:5
[INFO] [stdout]    8: tokio::task::coop::budget
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/task/coop/mod.rs:133:5
[INFO] [stdout]    9: tokio::runtime::scheduler::current_thread::CoreGuard::block_on::{{closure}}::{{closure}}
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:742:25
[INFO] [stdout]   10: tokio::runtime::scheduler::current_thread::Context::enter
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:432:19
[INFO] [stdout]   11: tokio::runtime::scheduler::current_thread::CoreGuard::block_on::{{closure}}
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:741:44
[INFO] [stdout]   12: tokio::runtime::scheduler::current_thread::CoreGuard::enter::{{closure}}
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:829:68
[INFO] [stdout]   13: tokio::runtime::context::scoped::Scoped<T>::set
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/context/scoped.rs:40:9
[INFO] [stdout]   14: tokio::runtime::context::set_scheduler::{{closure}}
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/context.rs:176:38
[INFO] [stdout]   15: std::thread::local::LocalKey<T>::try_with
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/thread/local.rs:508:12
[INFO] [stdout]   16: std::thread::local::LocalKey<T>::with
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/thread/local.rs:472:20
[INFO] [stdout]   17: tokio::runtime::context::set_scheduler
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/context.rs:176:17
[INFO] [stdout]   18: tokio::runtime::scheduler::current_thread::CoreGuard::enter
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:829:27
[INFO] [stdout]   19: tokio::runtime::scheduler::current_thread::CoreGuard::block_on
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:729:24
[INFO] [stdout]   20: tokio::runtime::scheduler::current_thread::CurrentThread::block_on::{{closure}}
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:200:33
[INFO] [stdout]   21: tokio::runtime::context::runtime::enter_runtime
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/context/runtime.rs:65:16
[INFO] [stdout]   22: tokio::runtime::scheduler::current_thread::CurrentThread::block_on
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:188:9
[INFO] [stdout]   23: tokio::runtime::runtime::Runtime::block_on_inner
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/runtime.rs:356:52
[INFO] [stdout]   24: tokio::runtime::runtime::Runtime::block_on
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/runtime.rs:330:18
[INFO] [stdout]   25: ai_workbench_lib::modules::file_discovery::file_discovery_tests::test_large_folder_discovery
[INFO] [stdout]              at ./src/../modules/file_discovery.rs:553:22
[INFO] [stdout]   26: ai_workbench_lib::modules::file_discovery::file_discovery_tests::test_large_folder_discovery::{{closure}}
[INFO] [stdout]              at ./src/../modules/file_discovery.rs:543:43
[INFO] [stdout]   27: core::ops::function::FnOnce::call_once
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/ops/function.rs:250:5
[INFO] [stdout]   28: core::ops::function::FnOnce::call_once
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/ops/function.rs:250:5
[INFO] [stdout]   29: test::__rust_begin_short_backtrace
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/test/src/lib.rs:663:18
[INFO] [stdout]   30: test::run_test_in_process::{{closure}}
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/test/src/lib.rs:686:74
[INFO] [stdout]   31: <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/panic/unwind_safe.rs:274:9
[INFO] [stdout]   32: std::panicking::catch_unwind::do_call
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panicking.rs:590:40
[INFO] [stdout]   33: std::panicking::catch_unwind
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panicking.rs:553:19
[INFO] [stdout]   34: std::panic::catch_unwind
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panic.rs:359:14
[INFO] [stdout]   35: test::run_test_in_process
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/test/src/lib.rs:686:27
[INFO] [stdout]   36: test::run_test::{{closure}}
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/test/src/lib.rs:607:43
[INFO] [stdout]   37: test::run_test::{{closure}}
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/test/src/lib.rs:637:41
[INFO] [stdout]   38: std::sys::backtrace::__rust_begin_short_backtrace
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/sys/backtrace.rs:158:18
[INFO] [stdout]   39: std::thread::Builder::spawn_unchecked_::{{closure}}::{{closure}}
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/thread/mod.rs:562:17
[INFO] [stdout]   40: <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/panic/unwind_safe.rs:274:9
[INFO] [stdout]   41: std::panicking::catch_unwind::do_call
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panicking.rs:590:40
[INFO] [stdout]   42: std::panicking::catch_unwind
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panicking.rs:553:19
[INFO] [stdout]   43: std::panic::catch_unwind
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panic.rs:359:14
[INFO] [stdout]   44: std::thread::Builder::spawn_unchecked_::{{closure}}
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/thread/mod.rs:560:30
[INFO] [stdout]   45: core::ops::function::FnOnce::call_once{{vtable.shim}}
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/ops/function.rs:250:5
[INFO] [stdout]   46: <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/alloc/src/boxed.rs:2085:9
[INFO] [stdout]   47: std::sys::thread::unix::Thread::new::thread_start
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/sys/thread/unix.rs:124:17
[INFO] [stdout]   48: <unknown>
[INFO] [stdout]   49: clone
[INFO] [stdout] [2m2025-11-11T07:22:52.385176Z[0m [32m INFO[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m === FileDiscovery::discover_files DEBUG END ===
[INFO] [stdout] test modules::job_processor::core::job_processor_tests::test_job_processor_config_validation ... ok
[INFO] [stdout] [2m2025-11-11T07:22:52.380621Z[0m [31mERROR[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m Error details: Failed to list objects in S3
[INFO] [stdout] 
[INFO] [stdout] Caused by:
[INFO] [stdout]     S3 ListObjectsV2 failed for folder 'text_files/': dispatch failure
[INFO] [stdout] 
[INFO] [stdout] Stack backtrace:
[INFO] [stdout]    0: anyhow::error::<impl anyhow::Error>::msg
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/anyhow-1.0.99/src/backtrace.rs:27:14
[INFO] [stdout]    1: ai_workbench_lib::modules::file_discovery::FileDiscovery::process_folder::{{closure}}
[INFO] [stdout]              at ./src/../modules/file_discovery.rs:157:32
[INFO] [stdout]    2: ai_workbench_lib::modules::file_discovery::FileDiscovery::discover_files::{{closure}}
[INFO] [stdout]              at ./src/../modules/file_discovery.rs:31:45
[INFO] [stdout]    3: ai_workbench_lib::modules::job_processor::core::JobProcessor::discover_files::{{closure}}
[INFO] [stdout]              at ./src/../modules/job_processor/core.rs:102:83
[INFO] [stdout]    4: ai_workbench_lib::modules::job_processor::core::job_processor_tests::test_job_processor_file_discovery::{{closure}}
[INFO] [stdout]              at ./src/../modules/job_processor/core.rs:955:42
[INFO] [stdout]    5: <core::pin::Pin<P> as core::future::future::Future>::poll
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/future/future.rs:133:9
[INFO] [stdout]    6: <core::pin::Pin<P> as core::future::future::Future>::poll
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/future/future.rs:133:9
[INFO] [stdout]    7: tokio::runtime::scheduler::current_thread::CoreGuard::block_on::{{closure}}::{{closure}}::{{closure}}
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:742:70
[INFO] [stdout]    8: tokio::task::coop::with_budget
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/task/coop/mod.rs:167:5
[INFO] [stdout]    9: tokio::task::coop::budget
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/task/coop/mod.rs:133:5
[INFO] [stdout]   10: tokio::runtime::scheduler::current_thread::CoreGuard::block_on::{{closure}}::{{closure}}
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:742:25
[INFO] [stdout]   11: tokio::runtime::scheduler::current_thread::Context::enter
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:432:19
[INFO] [stdout]   12: tokio::runtime::scheduler::current_thread::CoreGuard::block_on::{{closure}}
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:741:44
[INFO] [stdout]   13: tokio::runtime::scheduler::current_thread::CoreGuard::enter::{{closure}}
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:829:68
[INFO] [stdout]   14: tokio::runtime::context::scoped::Scoped<T>::set
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/context/scoped.rs:40:9
[INFO] [stdout]   15: tokio::runtime::context::set_scheduler::{{closure}}
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/context.rs:176:38
[INFO] [stdout]   16: std::thread::local::LocalKey<T>::try_with
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/thread/local.rs:508:12
[INFO] [stdout]   17: std::thread::local::LocalKey<T>::with
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/thread/local.rs:472:20
[INFO] [stdout]   18: tokio::runtime::context::set_scheduler
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/context.rs:176:17
[INFO] [stdout]   19: tokio::runtime::scheduler::current_thread::CoreGuard::enter
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:829:27
[INFO] [stdout]   20: tokio::runtime::scheduler::current_thread::CoreGuard::block_on
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:729:24
[INFO] [stdout]   21: tokio::runtime::scheduler::current_thread::CurrentThread::block_on::{{closure}}
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:200:33
[INFO] [stdout]   22: tokio::runtime::context::runtime::enter_runtime
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/context/runtime.rs:65:16
[INFO] [stdout]   23: tokio::runtime::scheduler::current_thread::CurrentThread::block_on
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:188:9
[INFO] [stdout]   24: tokio::runtime::runtime::Runtime::block_on_inner
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/runtime.rs:356:52
[INFO] [stdout]   25: tokio::runtime::runtime::Runtime::block_on
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/runtime.rs:330:18
[INFO] [stdout]   26: ai_workbench_lib::modules::job_processor::core::job_processor_tests::test_job_processor_file_discovery
[INFO] [stdout]              at ./src/../modules/job_processor/core.rs:955:48
[INFO] [stdout]   27: ai_workbench_lib::modules::job_processor::core::job_processor_tests::test_job_processor_file_discovery::{{closure}}
[INFO] [stdout]              at ./src/../modules/job_processor/core.rs:930:49
[INFO] [stdout]   28: core::ops::function::FnOnce::call_once
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/ops/function.rs:250:5
[INFO] [stdout]   29: core::ops::function::FnOnce::call_once
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/ops/function.rs:250:5
[INFO] [stdout]   30: test::__rust_begin_short_backtrace
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/test/src/lib.rs:663:18
[INFO] [stdout]   31: test::run_test_in_process::{{closure}}
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/test/src/lib.rs:686:74
[INFO] [stdout]   32: <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/panic/unwind_safe.rs:274:9
[INFO] [stdout]   33: std::panicking::catch_unwind::do_call
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panicking.rs:590:40
[INFO] [stdout]   34: std::panicking::catch_unwind
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panicking.rs:553:19
[INFO] [stdout]   35: std::panic::catch_unwind
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panic.rs:359:14
[INFO] [stdout]   36: test::run_test_in_process
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/test/src/lib.rs:686:27
[INFO] [stdout]   37: test::run_test::{{closure}}
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/test/src/lib.rs:607:43
[INFO] [stdout]   38: test::run_test::{{closure}}
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/test/src/lib.rs:637:41
[INFO] [stdout]   39: std::sys::backtrace::__rust_begin_short_backtrace
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/sys/backtrace.rs:158:18
[INFO] [stdout]   40: std::thread::Builder::spawn_unchecked_::{{closure}}::{{closure}}
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/thread/mod.rs:562:17
[INFO] [stdout]   41: <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/panic/unwind_safe.rs:274:9
[INFO] [stdout]   42: std::panicking::catch_unwind::do_call
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panicking.rs:590:40
[INFO] [stdout]   43: std::panicking::catch_unwind
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panicking.rs:553:19
[INFO] [stdout]   44: std::panic::catch_unwind
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panic.rs:359:14
[INFO] [stdout]   45: std::thread::Builder::spawn_unchecked_::{{closure}}
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/thread/mod.rs:560:30
[INFO] [stdout]   46: core::ops::function::FnOnce::call_once{{vtable.shim}}
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/ops/function.rs:250:5
[INFO] [stdout]   47: <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/alloc/src/boxed.rs:2085:9
[INFO] [stdout]   48: std::sys::thread::unix::Thread::new::thread_start
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/sys/thread/unix.rs:124:17
[INFO] [stdout]   49: <unknown>
[INFO] [stdout]   50: clone
[INFO] [stdout] [2m2025-11-11T07:22:52.385861Z[0m [32m INFO[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m === FileDiscovery::discover_files DEBUG END ===
[INFO] [stdout] test modules::file_discovery::file_discovery_tests::test_large_folder_discovery ... ok
[INFO] [stdout] [2m2025-11-11T07:22:52.385877Z[0m [31mERROR[0m [2mai_workbench_lib::modules::job_processor::core[0m[2m:[0m File discovery failed with error: Failed to list objects in S3
[INFO] [stdout] test modules::job_processor::core::job_processor_tests::test_job_processor_single_file_end_to_end ... ok
[INFO] [stdout] [2m2025-11-11T07:22:52.385890Z[0m [31mERROR[0m [2mai_workbench_lib::modules::job_processor::core[0m[2m:[0m Error context: Failed to list objects in S3
[INFO] [stdout] 
[INFO] [stdout] Caused by:
[INFO] [stdout]     S3 ListObjectsV2 failed for folder 'text_files/': dispatch failure
[INFO] [stdout] 
[INFO] [stdout] Stack backtrace:
[INFO] [stdout]    0: anyhow::error::<impl anyhow::Error>::msg
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/anyhow-1.0.99/src/backtrace.rs:27:14
[INFO] [stdout]    1: ai_workbench_lib::modules::file_discovery::FileDiscovery::process_folder::{{closure}}
[INFO] [stdout]              at ./src/../modules/file_discovery.rs:157:32
[INFO] [stdout]    2: ai_workbench_lib::modules::file_discovery::FileDiscovery::discover_files::{{closure}}
[INFO] [stdout]              at ./src/../modules/file_discovery.rs:31:45
[INFO] [stdout]    3: ai_workbench_lib::modules::job_processor::core::JobProcessor::discover_files::{{closure}}
[INFO] [stdout]              at ./src/../modules/job_processor/core.rs:102:83
[INFO] [stdout]    4: ai_workbench_lib::modules::job_processor::core::job_processor_tests::test_job_processor_file_discovery::{{closure}}
[INFO] [stdout]              at ./src/../modules/job_processor/core.rs:955:42
[INFO] [stdout]    5: <core::pin::Pin<P> as core::future::future::Future>::poll
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/future/future.rs:133:9
[INFO] [stdout]    6: <core::pin::Pin<P> as core::future::future::Future>::poll
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/future/future.rs:133:9
[INFO] [stdout]    7: tokio::runtime::scheduler::current_thread::CoreGuard::block_on::{{closure}}::{{closure}}::{{closure}}
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:742:70
[INFO] [stdout]    8: tokio::task::coop::with_budget
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/task/coop/mod.rs:167:5
[INFO] [stdout]    9: tokio::task::coop::budget
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/task/coop/mod.rs:133:5
[INFO] [stdout]   10: tokio::runtime::scheduler::current_thread::CoreGuard::block_on::{{closure}}::{{closure}}
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:742:25
[INFO] [stdout]   11: tokio::runtime::scheduler::current_thread::Context::enter
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:432:19
[INFO] [stdout]   12: tokio::runtime::scheduler::current_thread::CoreGuard::block_on::{{closure}}
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:741:44
[INFO] [stdout]   13: tokio::runtime::scheduler::current_thread::CoreGuard::enter::{{closure}}
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:829:68
[INFO] [stdout]   14: tokio::runtime::context::scoped::Scoped<T>::set
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/context/scoped.rs:40:9
[INFO] [stdout]   15: tokio::runtime::context::set_scheduler::{{closure}}
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/context.rs:176:38
[INFO] [stdout]   16: std::thread::local::LocalKey<T>::try_with
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/thread/local.rs:508:12
[INFO] [stdout]   17: std::thread::local::LocalKey<T>::with
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/thread/local.rs:472:20
[INFO] [stdout]   18: tokio::runtime::context::set_scheduler
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/context.rs:176:17
[INFO] [stdout]   19: tokio::runtime::scheduler::current_thread::CoreGuard::enter
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:829:27
[INFO] [stdout]   20: tokio::runtime::scheduler::current_thread::CoreGuard::block_on
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:729:24
[INFO] [stdout]   21: tokio::runtime::scheduler::current_thread::CurrentThread::block_on::{{closure}}
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:200:33
[INFO] [stdout]   22: tokio::runtime::context::runtime::enter_runtime
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/context/runtime.rs:65:16
[INFO] [stdout]   23: tokio::runtime::scheduler::current_thread::CurrentThread::block_on
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:188:9
[INFO] [stdout]   24: tokio::runtime::runtime::Runtime::block_on_inner
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/runtime.rs:356:52
[INFO] [stdout]   25: tokio::runtime::runtime::Runtime::block_on
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/runtime.rs:330:18
[INFO] [stdout]   26: ai_workbench_lib::modules::job_processor::core::job_processor_tests::test_job_processor_file_discovery
[INFO] [stdout]              at ./src/../modules/job_processor/core.rs:955:48
[INFO] [stdout]   27: ai_workbench_lib::modules::job_processor::core::job_processor_tests::test_job_processor_file_discovery::{{closure}}
[INFO] [stdout]              at ./src/../modules/job_processor/core.rs:930:49
[INFO] [stdout]   28: core::ops::function::FnOnce::call_once
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/ops/function.rs:250:5
[INFO] [stdout]   29: core::ops::function::FnOnce::call_once
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/ops/function.rs:250:5
[INFO] [stdout]   30: test::__rust_begin_short_backtrace
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/test/src/lib.rs:663:18
[INFO] [stdout]   31: test::run_test_in_process::{{closure}}
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/test/src/lib.rs:686:74
[INFO] [stdout]   32: <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/panic/unwind_safe.rs:274:9
[INFO] [stdout]   33: std::panicking::catch_unwind::do_call
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panicking.rs:590:40
[INFO] [stdout]   34: std::panicking::catch_unwind
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panicking.rs:553:19
[INFO] [stdout]   35: std::panic::catch_unwind
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panic.rs:359:14
[INFO] [stdout]   36: test::run_test_in_process
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/test/src/lib.rs:686:27
[INFO] [stdout]   37: test::run_test::{{closure}}
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/test/src/lib.rs:607:43
[INFO] [stdout]   38: test::run_test::{{closure}}
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/test/src/lib.rs:637:41
[INFO] [stdout]   39: std::sys::backtrace::__rust_begin_short_backtrace
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/sys/backtrace.rs:158:18
[INFO] [stdout]   40: std::thread::Builder::spawn_unchecked_::{{closure}}::{{closure}}
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/thread/mod.rs:562:17
[INFO] [stdout]   41: <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/panic/unwind_safe.rs:274:9
[INFO] [stdout]   42: std::panicking::catch_unwind::do_call
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panicking.rs:590:40
[INFO] [stdout]   43: std::panicking::catch_unwind
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panicking.rs:553:19
[INFO] [stdout]   44: std::panic::catch_unwind
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panic.rs:359:14
[INFO] [stdout]   45: std::thread::Builder::spawn_unchecked_::{{closure}}
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/thread/mod.rs:560:30
[INFO] [stdout]   46: core::ops::function::FnOnce::call_once{{vtable.shim}}
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/ops/function.rs:250:5
[INFO] [stdout]   47: <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/alloc/src/boxed.rs:2085:9
[INFO] [stdout]   48: std::sys::thread::unix::Thread::new::thread_start
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/sys/thread/unix.rs:124:17
[INFO] [stdout]   49: <unknown>
[INFO] [stdout]   50: clone
[INFO] [stdout] [2m2025-11-11T07:22:52.386438Z[0m [32m INFO[0m [2mai_workbench_lib::modules::job_processor::core[0m[2m:[0m === FILE DISCOVERY DEBUG END (FAILED) ===
[INFO] [stdout] test modules::job_processor::core::job_processor_tests::test_job_processor_file_discovery ... ok
[INFO] [stdout] [2m2025-11-11T07:22:52.385066Z[0m [32m INFO[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m Full S3 path: s3://ai-workbench-6c9c43db-7fe6-42f1-8b11-8f82323f83f0-eu-west-2/test-data/config.json
[INFO] [stdout] [2m2025-11-11T07:22:52.387236Z[0m [32m INFO[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m Sending HEAD request to S3...
[INFO] [stdout] [2m2025-11-11T07:22:52.390139Z[0m [31mERROR[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m S3 HEAD request failed for 'test-data/config.json': dispatch failure
[INFO] [stdout] [2m2025-11-11T07:22:52.390176Z[0m [31mERROR[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m Error details: DispatchFailure(DispatchFailure { source: ConnectorError { kind: Other(None), source: CredentialsNotLoaded(CredentialsNotLoaded { source: Some("no providers in chain provided credentials") }), connection: Unknown } })
[INFO] [stdout] [2m2025-11-11T07:22:52.390204Z[0m [31mERROR[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m Possible causes:
[INFO] [stdout] [2m2025-11-11T07:22:52.390213Z[0m [31mERROR[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m   1. File 'test-data/config.json' does not exist in bucket 'ai-workbench-6c9c43db-7fe6-42f1-8b11-8f82323f83f0-eu-west-2'
[INFO] [stdout] [2m2025-11-11T07:22:52.390229Z[0m [31mERROR[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m   2. No read permissions for this file/bucket
[INFO] [stdout] [2m2025-11-11T07:22:52.390237Z[0m [31mERROR[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m   3. AWS credentials are invalid or expired
[INFO] [stdout] [2m2025-11-11T07:22:52.390244Z[0m [31mERROR[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m   4. Network connectivity issues
[INFO] [stdout] [2m2025-11-11T07:22:52.390265Z[0m [31mERROR[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m   5. S3 service is temporarily unavailable
[INFO] [stdout] [2m2025-11-11T07:22:52.390329Z[0m [31mERROR[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m Failed to get metadata for file 'test-data/config.json': Failed to get metadata for test-data/config.json
[INFO] [stdout] [2m2025-11-11T07:22:52.390354Z[0m [31mERROR[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m This usually means:
[INFO] [stdout] [2m2025-11-11T07:22:52.390362Z[0m [31mERROR[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m   1. File does not exist at s3://ai-workbench-6c9c43db-7fe6-42f1-8b11-8f82323f83f0-eu-west-2/test-data/config.json
[INFO] [stdout] [2m2025-11-11T07:22:52.390374Z[0m [31mERROR[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m   2. No permissions to access the file
[INFO] [stdout] [2m2025-11-11T07:22:52.390382Z[0m [31mERROR[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m   3. S3 service is unavailable
[INFO] [stdout] [2m2025-11-11T07:22:52.390391Z[0m [31mERROR[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m   4. Incorrect bucket name or file path
[INFO] [stdout] [2m2025-11-11T07:22:52.390402Z[0m [31mERROR[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m discover_files failed: Failed to get metadata for single file: test-data/config.json
[INFO] [stdout] [2m2025-11-11T07:22:52.390424Z[0m [31mERROR[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m Error details: Failed to get metadata for single file: test-data/config.json
[INFO] [stdout] 
[INFO] [stdout] Caused by:
[INFO] [stdout]     0: Failed to get metadata for test-data/config.json
[INFO] [stdout]     1: S3 HEAD request failed for 'test-data/config.json': dispatch failure
[INFO] [stdout] 
[INFO] [stdout] Stack backtrace:
[INFO] [stdout]    0: anyhow::error::<impl anyhow::Error>::msg
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/anyhow-1.0.99/src/backtrace.rs:27:14
[INFO] [stdout]    1: ai_workbench_lib::modules::file_discovery::FileDiscovery::get_file_metadata::{{closure}}
[INFO] [stdout]              at ./src/../modules/file_discovery.rs:267:28
[INFO] [stdout]    2: ai_workbench_lib::modules::file_discovery::FileDiscovery::process_single_file::{{closure}}
[INFO] [stdout]              at ./src/../modules/file_discovery.rs:59:64
[INFO] [stdout]    3: ai_workbench_lib::modules::file_discovery::FileDiscovery::discover_files::{{closure}}
[INFO] [stdout]              at ./src/../modules/file_discovery.rs:35:50
[INFO] [stdout]    4: ai_workbench_lib::modules::file_discovery::file_discovery_tests::test_file_type_detection::{{closure}}
[INFO] [stdout]              at ./src/../modules/file_discovery.rs:516:61
[INFO] [stdout]    5: <core::pin::Pin<P> as core::future::future::Future>::poll
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/future/future.rs:133:9
[INFO] [stdout]    6: <core::pin::Pin<P> as core::future::future::Future>::poll
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/future/future.rs:133:9
[INFO] [stdout]    7: tokio::runtime::scheduler::current_thread::CoreGuard::block_on::{{closure}}::{{closure}}::{{closure}}
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:742:70
[INFO] [stdout]    8: tokio::task::coop::with_budget
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/task/coop/mod.rs:167:5
[INFO] [stdout]    9: tokio::task::coop::budget
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/task/coop/mod.rs:133:5
[INFO] [stdout]   10: tokio::runtime::scheduler::current_thread::CoreGuard::block_on::{{closure}}::{{closure}}
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:742:25
[INFO] [stdout]   11: tokio::runtime::scheduler::current_thread::Context::enter
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:432:19
[INFO] [stdout]   12: tokio::runtime::scheduler::current_thread::CoreGuard::block_on::{{closure}}
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:741:44
[INFO] [stdout]   13: tokio::runtime::scheduler::current_thread::CoreGuard::enter::{{closure}}
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:829:68
[INFO] [stdout]   14: tokio::runtime::context::scoped::Scoped<T>::set
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/context/scoped.rs:40:9
[INFO] [stdout]   15: tokio::runtime::context::set_scheduler::{{closure}}
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/context.rs:176:38
[INFO] [stdout]   16: std::thread::local::LocalKey<T>::try_with
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/thread/local.rs:508:12
[INFO] [stdout]   17: std::thread::local::LocalKey<T>::with
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/thread/local.rs:472:20
[INFO] [stdout]   18: tokio::runtime::context::set_scheduler
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/context.rs:176:17
[INFO] [stdout]   19: tokio::runtime::scheduler::current_thread::CoreGuard::enter
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:829:27
[INFO] [stdout]   20: tokio::runtime::scheduler::current_thread::CoreGuard::block_on
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:729:24
[INFO] [stdout]   21: tokio::runtime::scheduler::current_thread::CurrentThread::block_on::{{closure}}
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:200:33
[INFO] [stdout]   22: tokio::runtime::context::runtime::enter_runtime
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/context/runtime.rs:65:16
[INFO] [stdout]   23: tokio::runtime::scheduler::current_thread::CurrentThread::block_on
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:188:9
[INFO] [stdout]   24: tokio::runtime::runtime::Runtime::block_on_inner
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/runtime.rs:356:52
[INFO] [stdout]   25: tokio::runtime::runtime::Runtime::block_on
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/runtime.rs:330:18
[INFO] [stdout]   26: ai_workbench_lib::modules::file_discovery::file_discovery_tests::test_file_type_detection
[INFO] [stdout]              at ./src/../modules/file_discovery.rs:515:36
[INFO] [stdout]   27: ai_workbench_lib::modules::file_discovery::file_discovery_tests::test_file_type_detection::{{closure}}
[INFO] [stdout]              at ./src/../modules/file_discovery.rs:498:40
[INFO] [stdout]   28: core::ops::function::FnOnce::call_once
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/ops/function.rs:250:5
[INFO] [stdout]   29: core::ops::function::FnOnce::call_once
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/ops/function.rs:250:5
[INFO] [stdout]   30: test::__rust_begin_short_backtrace
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/test/src/lib.rs:663:18
[INFO] [stdout]   31: test::run_test_in_process::{{closure}}
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/test/src/lib.rs:686:74
[INFO] [stdout]   32: <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/panic/unwind_safe.rs:274:9
[INFO] [stdout]   33: std::panicking::catch_unwind::do_call
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panicking.rs:590:40
[INFO] [stdout]   34: std::panicking::catch_unwind
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panicking.rs:553:19
[INFO] [stdout]   35: std::panic::catch_unwind
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panic.rs:359:14
[INFO] [stdout]   36: test::run_test_in_process
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/test/src/lib.rs:686:27
[INFO] [stdout]   37: test::run_test::{{closure}}
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/test/src/lib.rs:607:43
[INFO] [stdout]   38: test::run_test::{{closure}}
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/test/src/lib.rs:637:41
[INFO] [stdout]   39: std::sys::backtrace::__rust_begin_short_backtrace
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/sys/backtrace.rs:158:18
[INFO] [stdout]   40: std::thread::Builder::spawn_unchecked_::{{closure}}::{{closure}}
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/thread/mod.rs:562:17
[INFO] [stdout]   41: <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/panic/unwind_safe.rs:274:9
[INFO] [stdout]   42: std::panicking::catch_unwind::do_call
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panicking.rs:590:40
[INFO] [stdout]   43: std::panicking::catch_unwind
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panicking.rs:553:19
[INFO] [stdout]   44: std::panic::catch_unwind
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panic.rs:359:14
[INFO] [stdout]   45: std::thread::Builder::spawn_unchecked_::{{closure}}
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/thread/mod.rs:560:30
[INFO] [stdout]   46: core::ops::function::FnOnce::call_once{{vtable.shim}}
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/ops/function.rs:250:5
[INFO] [stdout]   47: <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/alloc/src/boxed.rs:2085:9
[INFO] [stdout]   48: std::sys::thread::unix::Thread::new::thread_start
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/sys/thread/unix.rs:124:17
[INFO] [stdout]   49: <unknown>
[INFO] [stdout]   50: clone
[INFO] [stdout] [2m2025-11-11T07:22:52.391091Z[0m [32m INFO[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m === FileDiscovery::discover_files DEBUG END ===
[INFO] [stdout] [2m2025-11-11T07:22:52.391124Z[0m [32m INFO[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m === FileDiscovery::discover_files DEBUG START ===
[INFO] [stdout] [2m2025-11-11T07:22:52.391135Z[0m [32m INFO[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m Input spec: 'test-data/document.md'
[INFO] [stdout] [2m2025-11-11T07:22:52.391144Z[0m [32m INFO[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m Workspace bucket: 'ai-workbench-6c9c43db-7fe6-42f1-8b11-8f82323f83f0-eu-west-2'
[INFO] [stdout] [2m2025-11-11T07:22:52.391155Z[0m [32m INFO[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m Detected single file input (no trailing '/') - processing as single file
[INFO] [stdout] [2m2025-11-11T07:22:52.391173Z[0m [32m INFO[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m === process_single_file DEBUG START ===
[INFO] [stdout] [2m2025-11-11T07:22:52.391182Z[0m [32m INFO[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m Processing single file: 'test-data/document.md'
[INFO] [stdout] [2m2025-11-11T07:22:52.391190Z[0m [32m INFO[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m Full S3 path: s3://ai-workbench-6c9c43db-7fe6-42f1-8b11-8f82323f83f0-eu-west-2/test-data/document.md
[INFO] [stdout] [2m2025-11-11T07:22:52.391201Z[0m [32m INFO[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m Attempting to get file metadata...
[INFO] [stdout] [2m2025-11-11T07:22:52.391211Z[0m [32m INFO[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m === get_file_metadata DEBUG START ===
[INFO] [stdout] [2m2025-11-11T07:22:52.391220Z[0m [32m INFO[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m Getting metadata for file: 'test-data/document.md'
[INFO] [stdout] [2m2025-11-11T07:22:52.391229Z[0m [32m INFO[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m Bucket: 'ai-workbench-6c9c43db-7fe6-42f1-8b11-8f82323f83f0-eu-west-2'
[INFO] [stdout] [2m2025-11-11T07:22:52.391239Z[0m [32m INFO[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m Full S3 path: s3://ai-workbench-6c9c43db-7fe6-42f1-8b11-8f82323f83f0-eu-west-2/test-data/document.md
[INFO] [stdout] [2m2025-11-11T07:22:52.391249Z[0m [32m INFO[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m Sending HEAD request to S3...
[INFO] [stdout] [2m2025-11-11T07:22:52.393768Z[0m [31mERROR[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m S3 HEAD request failed for 'test-data/document.md': dispatch failure
[INFO] [stdout] [2m2025-11-11T07:22:52.393802Z[0m [31mERROR[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m Error details: DispatchFailure(DispatchFailure { source: ConnectorError { kind: Other(None), source: CredentialsNotLoaded(CredentialsNotLoaded { source: Some("no providers in chain provided credentials") }), connection: Unknown } })
[INFO] [stdout] [2m2025-11-11T07:22:52.393827Z[0m [31mERROR[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m Possible causes:
[INFO] [stdout] [2m2025-11-11T07:22:52.393834Z[0m [31mERROR[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m   1. File 'test-data/document.md' does not exist in bucket 'ai-workbench-6c9c43db-7fe6-42f1-8b11-8f82323f83f0-eu-west-2'
[INFO] [stdout] [2m2025-11-11T07:22:52.393847Z[0m [31mERROR[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m   2. No read permissions for this file/bucket
[INFO] [stdout] [2m2025-11-11T07:22:52.393855Z[0m [31mERROR[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m   3. AWS credentials are invalid or expired
[INFO] [stdout] [2m2025-11-11T07:22:52.393863Z[0m [31mERROR[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m   4. Network connectivity issues
[INFO] [stdout] [2m2025-11-11T07:22:52.393871Z[0m [31mERROR[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m   5. S3 service is temporarily unavailable
[INFO] [stdout] [2m2025-11-11T07:22:52.393969Z[0m [31mERROR[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m Failed to get metadata for file 'test-data/document.md': Failed to get metadata for test-data/document.md
[INFO] [stdout] [2m2025-11-11T07:22:52.393991Z[0m [31mERROR[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m This usually means:
[INFO] [stdout] [2m2025-11-11T07:22:52.393999Z[0m [31mERROR[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m   1. File does not exist at s3://ai-workbench-6c9c43db-7fe6-42f1-8b11-8f82323f83f0-eu-west-2/test-data/document.md
[INFO] [stdout] [2m2025-11-11T07:22:52.394011Z[0m [31mERROR[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m   2. No permissions to access the file
[INFO] [stdout] [2m2025-11-11T07:22:52.394019Z[0m [31mERROR[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m   3. S3 service is unavailable
[INFO] [stdout] [2m2025-11-11T07:22:52.394027Z[0m [31mERROR[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m   4. Incorrect bucket name or file path
[INFO] [stdout] [2m2025-11-11T07:22:52.394036Z[0m [31mERROR[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m discover_files failed: Failed to get metadata for single file: test-data/document.md
[INFO] [stdout] [2m2025-11-11T07:22:52.394046Z[0m [31mERROR[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m Error details: Failed to get metadata for single file: test-data/document.md
[INFO] [stdout] 
[INFO] [stdout] Caused by:
[INFO] [stdout]     0: Failed to get metadata for test-data/document.md
[INFO] [stdout]     1: S3 HEAD request failed for 'test-data/document.md': dispatch failure
[INFO] [stdout] 
[INFO] [stdout] Stack backtrace:
[INFO] [stdout]    0: anyhow::error::<impl anyhow::Error>::msg
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/anyhow-1.0.99/src/backtrace.rs:27:14
[INFO] [stdout]    1: ai_workbench_lib::modules::file_discovery::FileDiscovery::get_file_metadata::{{closure}}
[INFO] [stdout]              at ./src/../modules/file_discovery.rs:267:28
[INFO] [stdout]    2: ai_workbench_lib::modules::file_discovery::FileDiscovery::process_single_file::{{closure}}
[INFO] [stdout]              at ./src/../modules/file_discovery.rs:59:64
[INFO] [stdout]    3: ai_workbench_lib::modules::file_discovery::FileDiscovery::discover_files::{{closure}}
[INFO] [stdout]              at ./src/../modules/file_discovery.rs:35:50
[INFO] [stdout]    4: ai_workbench_lib::modules::file_discovery::file_discovery_tests::test_file_type_detection::{{closure}}
[INFO] [stdout]              at ./src/../modules/file_discovery.rs:516:61
[INFO] [stdout]    5: <core::pin::Pin<P> as core::future::future::Future>::poll
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/future/future.rs:133:9
[INFO] [stdout]    6: <core::pin::Pin<P> as core::future::future::Future>::poll
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/future/future.rs:133:9
[INFO] [stdout]    7: tokio::runtime::scheduler::current_thread::CoreGuard::block_on::{{closure}}::{{closure}}::{{closure}}
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:742:70
[INFO] [stdout]    8: tokio::task::coop::with_budget
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/task/coop/mod.rs:167:5
[INFO] [stdout]    9: tokio::task::coop::budget
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/task/coop/mod.rs:133:5
[INFO] [stdout]   10: tokio::runtime::scheduler::current_thread::CoreGuard::block_on::{{closure}}::{{closure}}
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:742:25
[INFO] [stdout]   11: tokio::runtime::scheduler::current_thread::Context::enter
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:432:19
[INFO] [stdout]   12: tokio::runtime::scheduler::current_thread::CoreGuard::block_on::{{closure}}
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:741:44
[INFO] [stdout]   13: tokio::runtime::scheduler::current_thread::CoreGuard::enter::{{closure}}
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:829:68
[INFO] [stdout]   14: tokio::runtime::context::scoped::Scoped<T>::set
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/context/scoped.rs:40:9
[INFO] [stdout]   15: tokio::runtime::context::set_scheduler::{{closure}}
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/context.rs:176:38
[INFO] [stdout]   16: std::thread::local::LocalKey<T>::try_with
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/thread/local.rs:508:12
[INFO] [stdout]   17: std::thread::local::LocalKey<T>::with
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/thread/local.rs:472:20
[INFO] [stdout]   18: tokio::runtime::context::set_scheduler
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/context.rs:176:17
[INFO] [stdout]   19: tokio::runtime::scheduler::current_thread::CoreGuard::enter
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:829:27
[INFO] [stdout]   20: tokio::runtime::scheduler::current_thread::CoreGuard::block_on
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:729:24
[INFO] [stdout]   21: tokio::runtime::scheduler::current_thread::CurrentThread::block_on::{{closure}}
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:200:33
[INFO] [stdout]   22: tokio::runtime::context::runtime::enter_runtime
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/context/runtime.rs:65:16
[INFO] [stdout]   23: tokio::runtime::scheduler::current_thread::CurrentThread::block_on
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:188:9
[INFO] [stdout]   24: tokio::runtime::runtime::Runtime::block_on_inner
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/runtime.rs:356:52
[INFO] [stdout]   25: tokio::runtime::runtime::Runtime::block_on
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/runtime.rs:330:18
[INFO] [stdout]   26: ai_workbench_lib::modules::file_discovery::file_discovery_tests::test_file_type_detection
[INFO] [stdout]              at ./src/../modules/file_discovery.rs:515:36
[INFO] [stdout]   27: ai_workbench_lib::modules::file_discovery::file_discovery_tests::test_file_type_detection::{{closure}}
[INFO] [stdout]              at ./src/../modules/file_discovery.rs:498:40
[INFO] [stdout]   28: core::ops::function::FnOnce::call_once
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/ops/function.rs:250:5
[INFO] [stdout]   29: core::ops::function::FnOnce::call_once
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/ops/function.rs:250:5
[INFO] [stdout]   30: test::__rust_begin_short_backtrace
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/test/src/lib.rs:663:18
[INFO] [stdout]   31: test::run_test_in_process::{{closure}}
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/test/src/lib.rs:686:74
[INFO] [stdout]   32: <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/panic/unwind_safe.rs:274:9
[INFO] [stdout]   33: std::panicking::catch_unwind::do_call
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panicking.rs:590:40
[INFO] [stdout]   34: std::panicking::catch_unwind
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panicking.rs:553:19
[INFO] [stdout]   35: std::panic::catch_unwind
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panic.rs:359:14
[INFO] [stdout]   36: test::run_test_in_process
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/test/src/lib.rs:686:27
[INFO] [stdout]   37: test::run_test::{{closure}}
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/test/src/lib.rs:607:43
[INFO] [stdout]   38: test::run_test::{{closure}}
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/test/src/lib.rs:637:41
[INFO] [stdout]   39: std::sys::backtrace::__rust_begin_short_backtrace
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/sys/backtrace.rs:158:18
[INFO] [stdout]   40: std::thread::Builder::spawn_unchecked_::{{closure}}::{{closure}}
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/thread/mod.rs:562:17
[INFO] [stdout]   41: <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/panic/unwind_safe.rs:274:9
[INFO] [stdout]   42: std::panicking::catch_unwind::do_call
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panicking.rs:590:40
[INFO] [stdout]   43: std::panicking::catch_unwind
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panicking.rs:553:19
[INFO] [stdout]   44: std::panic::catch_unwind
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panic.rs:359:14
[INFO] [stdout]   45: std::thread::Builder::spawn_unchecked_::{{closure}}
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/thread/mod.rs:560:30
[INFO] [stdout]   46: core::ops::function::FnOnce::call_once{{vtable.shim}}
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/ops/function.rs:250:5
[INFO] [stdout]   47: <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/alloc/src/boxed.rs:2085:9
[INFO] [stdout]   48: std::sys::thread::unix::Thread::new::thread_start
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/sys/thread/unix.rs:124:17
[INFO] [stdout]   49: <unknown>
[INFO] [stdout]   50: clone
[INFO] [stdout] [2m2025-11-11T07:22:52.394675Z[0m [32m INFO[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m === FileDiscovery::discover_files DEBUG END ===
[INFO] [stdout] [2m2025-11-11T07:22:52.394706Z[0m [32m INFO[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m === FileDiscovery::discover_files DEBUG START ===
[INFO] [stdout] [2m2025-11-11T07:22:52.394717Z[0m [32m INFO[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m Input spec: 'test-data/script.py'
[INFO] [stdout] [2m2025-11-11T07:22:52.394725Z[0m [32m INFO[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m Workspace bucket: 'ai-workbench-6c9c43db-7fe6-42f1-8b11-8f82323f83f0-eu-west-2'
[INFO] [stdout] [2m2025-11-11T07:22:52.394736Z[0m [32m INFO[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m Detected single file input (no trailing '/') - processing as single file
[INFO] [stdout] [2m2025-11-11T07:22:52.394748Z[0m [32m INFO[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m === process_single_file DEBUG START ===
[INFO] [stdout] [2m2025-11-11T07:22:52.394755Z[0m [32m INFO[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m Processing single file: 'test-data/script.py'
[INFO] [stdout] [2m2025-11-11T07:22:52.394763Z[0m [32m INFO[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m Full S3 path: s3://ai-workbench-6c9c43db-7fe6-42f1-8b11-8f82323f83f0-eu-west-2/test-data/script.py
[INFO] [stdout] [2m2025-11-11T07:22:52.394773Z[0m [32m INFO[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m Attempting to get file metadata...
[INFO] [stdout] [2m2025-11-11T07:22:52.394783Z[0m [32m INFO[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m === get_file_metadata DEBUG START ===
[INFO] [stdout] [2m2025-11-11T07:22:52.394791Z[0m [32m INFO[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m Getting metadata for file: 'test-data/script.py'
[INFO] [stdout] [2m2025-11-11T07:22:52.394799Z[0m [32m INFO[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m Bucket: 'ai-workbench-6c9c43db-7fe6-42f1-8b11-8f82323f83f0-eu-west-2'
[INFO] [stdout] [2m2025-11-11T07:22:52.394808Z[0m [32m INFO[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m Full S3 path: s3://ai-workbench-6c9c43db-7fe6-42f1-8b11-8f82323f83f0-eu-west-2/test-data/script.py
[INFO] [stdout] [2m2025-11-11T07:22:52.394818Z[0m [32m INFO[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m Sending HEAD request to S3...
[INFO] [stdout] [2m2025-11-11T07:22:52.397570Z[0m [31mERROR[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m S3 HEAD request failed for 'test-data/script.py': dispatch failure
[INFO] [stdout] [2m2025-11-11T07:22:52.397640Z[0m [31mERROR[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m Error details: DispatchFailure(DispatchFailure { source: ConnectorError { kind: Other(None), source: CredentialsNotLoaded(CredentialsNotLoaded { source: Some("no providers in chain provided credentials") }), connection: Unknown } })
[INFO] [stdout] [2m2025-11-11T07:22:52.397707Z[0m [31mERROR[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m Possible causes:
[INFO] [stdout] [2m2025-11-11T07:22:52.397741Z[0m [31mERROR[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m   1. File 'test-data/script.py' does not exist in bucket 'ai-workbench-6c9c43db-7fe6-42f1-8b11-8f82323f83f0-eu-west-2'
[INFO] [stdout] [2m2025-11-11T07:22:52.397776Z[0m [31mERROR[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m   2. No read permissions for this file/bucket
[INFO] [stdout] [2m2025-11-11T07:22:52.397803Z[0m [31mERROR[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m   3. AWS credentials are invalid or expired
[INFO] [stdout] [2m2025-11-11T07:22:52.397828Z[0m [31mERROR[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m   4. Network connectivity issues
[INFO] [stdout] [2m2025-11-11T07:22:52.397852Z[0m [31mERROR[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m   5. S3 service is temporarily unavailable
[INFO] [stdout] [2m2025-11-11T07:22:52.397932Z[0m [31mERROR[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m Failed to get metadata for file 'test-data/script.py': Failed to get metadata for test-data/script.py
[INFO] [stdout] [2m2025-11-11T07:22:52.397985Z[0m [31mERROR[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m This usually means:
[INFO] [stdout] [2m2025-11-11T07:22:52.398014Z[0m [31mERROR[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m   1. File does not exist at s3://ai-workbench-6c9c43db-7fe6-42f1-8b11-8f82323f83f0-eu-west-2/test-data/script.py
[INFO] [stdout] [2m2025-11-11T07:22:52.398048Z[0m [31mERROR[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m   2. No permissions to access the file
[INFO] [stdout] [2m2025-11-11T07:22:52.398092Z[0m [31mERROR[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m   3. S3 service is unavailable
[INFO] [stdout] [2m2025-11-11T07:22:52.398128Z[0m [31mERROR[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m   4. Incorrect bucket name or file path
[INFO] [stdout] [2m2025-11-11T07:22:52.398157Z[0m [31mERROR[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m discover_files failed: Failed to get metadata for single file: test-data/script.py
[INFO] [stdout] [2m2025-11-11T07:22:52.398205Z[0m [31mERROR[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m Error details: Failed to get metadata for single file: test-data/script.py
[INFO] [stdout] 
[INFO] [stdout] Caused by:
[INFO] [stdout]     0: Failed to get metadata for test-data/script.py
[INFO] [stdout]     1: S3 HEAD request failed for 'test-data/script.py': dispatch failure
[INFO] [stdout] 
[INFO] [stdout] Stack backtrace:
[INFO] [stdout]    0: anyhow::error::<impl anyhow::Error>::msg
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/anyhow-1.0.99/src/backtrace.rs:27:14
[INFO] [stdout]    1: ai_workbench_lib::modules::file_discovery::FileDiscovery::get_file_metadata::{{closure}}
[INFO] [stdout]              at ./src/../modules/file_discovery.rs:267:28
[INFO] [stdout]    2: ai_workbench_lib::modules::file_discovery::FileDiscovery::process_single_file::{{closure}}
[INFO] [stdout]              at ./src/../modules/file_discovery.rs:59:64
[INFO] [stdout]    3: ai_workbench_lib::modules::file_discovery::FileDiscovery::discover_files::{{closure}}
[INFO] [stdout]              at ./src/../modules/file_discovery.rs:35:50
[INFO] [stdout]    4: ai_workbench_lib::modules::file_discovery::file_discovery_tests::test_file_type_detection::{{closure}}
[INFO] [stdout]              at ./src/../modules/file_discovery.rs:516:61
[INFO] [stdout]    5: <core::pin::Pin<P> as core::future::future::Future>::poll
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/future/future.rs:133:9
[INFO] [stdout]    6: <core::pin::Pin<P> as core::future::future::Future>::poll
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/future/future.rs:133:9
[INFO] [stdout]    7: tokio::runtime::scheduler::current_thread::CoreGuard::block_on::{{closure}}::{{closure}}::{{closure}}
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:742:70
[INFO] [stdout]    8: tokio::task::coop::with_budget
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/task/coop/mod.rs:167:5
[INFO] [stdout]    9: tokio::task::coop::budget
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/task/coop/mod.rs:133:5
[INFO] [stdout]   10: tokio::runtime::scheduler::current_thread::CoreGuard::block_on::{{closure}}::{{closure}}
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:742:25
[INFO] [stdout]   11: tokio::runtime::scheduler::current_thread::Context::enter
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:432:19
[INFO] [stdout]   12: tokio::runtime::scheduler::current_thread::CoreGuard::block_on::{{closure}}
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:741:44
[INFO] [stdout]   13: tokio::runtime::scheduler::current_thread::CoreGuard::enter::{{closure}}
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:829:68
[INFO] [stdout]   14: tokio::runtime::context::scoped::Scoped<T>::set
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/context/scoped.rs:40:9
[INFO] [stdout]   15: tokio::runtime::context::set_scheduler::{{closure}}
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/context.rs:176:38
[INFO] [stdout]   16: std::thread::local::LocalKey<T>::try_with
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/thread/local.rs:508:12
[INFO] [stdout]   17: std::thread::local::LocalKey<T>::with
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/thread/local.rs:472:20
[INFO] [stdout]   18: tokio::runtime::context::set_scheduler
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/context.rs:176:17
[INFO] [stdout]   19: tokio::runtime::scheduler::current_thread::CoreGuard::enter
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:829:27
[INFO] [stdout]   20: tokio::runtime::scheduler::current_thread::CoreGuard::block_on
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:729:24
[INFO] [stdout]   21: tokio::runtime::scheduler::current_thread::CurrentThread::block_on::{{closure}}
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:200:33
[INFO] [stdout]   22: tokio::runtime::context::runtime::enter_runtime
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/context/runtime.rs:65:16
[INFO] [stdout]   23: tokio::runtime::scheduler::current_thread::CurrentThread::block_on
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:188:9
[INFO] [stdout]   24: tokio::runtime::runtime::Runtime::block_on_inner
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/runtime.rs:356:52
[INFO] [stdout]   25: tokio::runtime::runtime::Runtime::block_on
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/runtime.rs:330:18
[INFO] [stdout]   26: ai_workbench_lib::modules::file_discovery::file_discovery_tests::test_file_type_detection
[INFO] [stdout]              at ./src/../modules/file_discovery.rs:515:36
[INFO] [stdout]   27: ai_workbench_lib::modules::file_discovery::file_discovery_tests::test_file_type_detection::{{closure}}
[INFO] [stdout]              at ./src/../modules/file_discovery.rs:498:40
[INFO] [stdout]   28: core::ops::function::FnOnce::call_once
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/ops/function.rs:250:5
[INFO] [stdout]   29: core::ops::function::FnOnce::call_once
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/ops/function.rs:250:5
[INFO] [stdout]   30: test::__rust_begin_short_backtrace
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/test/src/lib.rs:663:18
[INFO] [stdout]   31: test::run_test_in_process::{{closure}}
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/test/src/lib.rs:686:74
[INFO] [stdout]   32: <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/panic/unwind_safe.rs:274:9
[INFO] [stdout]   33: std::panicking::catch_unwind::do_call
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panicking.rs:590:40
[INFO] [stdout]   34: std::panicking::catch_unwind
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panicking.rs:553:19
[INFO] [stdout]   35: std::panic::catch_unwind
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panic.rs:359:14
[INFO] [stdout]   36: test::run_test_in_process
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/test/src/lib.rs:686:27
[INFO] [stdout]   37: test::run_test::{{closure}}
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/test/src/lib.rs:607:43
[INFO] [stdout]   38: test::run_test::{{closure}}
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/test/src/lib.rs:637:41
[INFO] [stdout]   39: std::sys::backtrace::__rust_begin_short_backtrace
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/sys/backtrace.rs:158:18
[INFO] [stdout]   40: std::thread::Builder::spawn_unchecked_::{{closure}}::{{closure}}
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/thread/mod.rs:562:17
[INFO] [stdout]   41: <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/panic/unwind_safe.rs:274:9
[INFO] [stdout]   42: std::panicking::catch_unwind::do_call
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panicking.rs:590:40
[INFO] [stdout]   43: std::panicking::catch_unwind
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panicking.rs:553:19
[INFO] [stdout]   44: std::panic::catch_unwind
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panic.rs:359:14
[INFO] [stdout]   45: std::thread::Builder::spawn_unchecked_::{{closure}}
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/thread/mod.rs:560:30
[INFO] [stdout]   46: core::ops::function::FnOnce::call_once{{vtable.shim}}
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/ops/function.rs:250:5
[INFO] [stdout]   47: <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/alloc/src/boxed.rs:2085:9
[INFO] [stdout]   48: std::sys::thread::unix::Thread::new::thread_start
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/sys/thread/unix.rs:124:17
[INFO] [stdout]   49: <unknown>
[INFO] [stdout]   50: clone
[INFO] [stdout] [2m2025-11-11T07:22:52.398887Z[0m [32m INFO[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m === FileDiscovery::discover_files DEBUG END ===
[INFO] [stdout] [2m2025-11-11T07:22:52.398937Z[0m [32m INFO[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m === FileDiscovery::discover_files DEBUG START ===
[INFO] [stdout] [2m2025-11-11T07:22:52.399004Z[0m [32m INFO[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m Input spec: 'test-data/code.rs'
[INFO] [stdout] [2m2025-11-11T07:22:52.399040Z[0m [32m INFO[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m Workspace bucket: 'ai-workbench-6c9c43db-7fe6-42f1-8b11-8f82323f83f0-eu-west-2'
[INFO] [stdout] [2m2025-11-11T07:22:52.399105Z[0m [32m INFO[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m Detected single file input (no trailing '/') - processing as single file
[INFO] [stdout] [2m2025-11-11T07:22:52.399140Z[0m [32m INFO[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m === process_single_file DEBUG START ===
[INFO] [stdout] [2m2025-11-11T07:22:52.399189Z[0m [32m INFO[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m Processing single file: 'test-data/code.rs'
[INFO] [stdout] [2m2025-11-11T07:22:52.399244Z[0m [32m INFO[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m Full S3 path: s3://ai-workbench-6c9c43db-7fe6-42f1-8b11-8f82323f83f0-eu-west-2/test-data/code.rs
[INFO] [stdout] [2m2025-11-11T07:22:52.399280Z[0m [32m INFO[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m Attempting to get file metadata...
[INFO] [stdout] [2m2025-11-11T07:22:52.399330Z[0m [32m INFO[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m === get_file_metadata DEBUG START ===
[INFO] [stdout] [2m2025-11-11T07:22:52.399386Z[0m [32m INFO[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m Getting metadata for file: 'test-data/code.rs'
[INFO] [stdout] [2m2025-11-11T07:22:52.399423Z[0m [32m INFO[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m Bucket: 'ai-workbench-6c9c43db-7fe6-42f1-8b11-8f82323f83f0-eu-west-2'
[INFO] [stdout] [2m2025-11-11T07:22:52.399474Z[0m [32m INFO[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m Full S3 path: s3://ai-workbench-6c9c43db-7fe6-42f1-8b11-8f82323f83f0-eu-west-2/test-data/code.rs
[INFO] [stdout] [2m2025-11-11T07:22:52.399536Z[0m [32m INFO[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m Sending HEAD request to S3...
[INFO] [stdout] [2m2025-11-11T07:22:52.402430Z[0m [31mERROR[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m S3 HEAD request failed for 'test-data/code.rs': dispatch failure
[INFO] [stdout] [2m2025-11-11T07:22:52.402543Z[0m [31mERROR[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m Error details: DispatchFailure(DispatchFailure { source: ConnectorError { kind: Other(None), source: CredentialsNotLoaded(CredentialsNotLoaded { source: Some("no providers in chain provided credentials") }), connection: Unknown } })
[INFO] [stdout] [2m2025-11-11T07:22:52.402621Z[0m [31mERROR[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m Possible causes:
[INFO] [stdout] [2m2025-11-11T07:22:52.402671Z[0m [31mERROR[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m   1. File 'test-data/code.rs' does not exist in bucket 'ai-workbench-6c9c43db-7fe6-42f1-8b11-8f82323f83f0-eu-west-2'
[INFO] [stdout] [2m2025-11-11T07:22:52.402780Z[0m [31mERROR[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m   2. No read permissions for this file/bucket
[INFO] [stdout] [2m2025-11-11T07:22:52.402828Z[0m [31mERROR[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m   3. AWS credentials are invalid or expired
[INFO] [stdout] [2m2025-11-11T07:22:52.402859Z[0m [31mERROR[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m   4. Network connectivity issues
[INFO] [stdout] [2m2025-11-11T07:22:52.402882Z[0m [31mERROR[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m   5. S3 service is temporarily unavailable
[INFO] [stdout] [2m2025-11-11T07:22:52.403018Z[0m [31mERROR[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m Failed to get metadata for file 'test-data/code.rs': Failed to get metadata for test-data/code.rs
[INFO] [stdout] [2m2025-11-11T07:22:52.403081Z[0m [31mERROR[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m This usually means:
[INFO] [stdout] [2m2025-11-11T07:22:52.403111Z[0m [31mERROR[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m   1. File does not exist at s3://ai-workbench-6c9c43db-7fe6-42f1-8b11-8f82323f83f0-eu-west-2/test-data/code.rs
[INFO] [stdout] [2m2025-11-11T07:22:52.403171Z[0m [31mERROR[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m   2. No permissions to access the file
[INFO] [stdout] [2m2025-11-11T07:22:52.403207Z[0m [31mERROR[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m   3. S3 service is unavailable
[INFO] [stdout] [2m2025-11-11T07:22:52.403253Z[0m [31mERROR[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m   4. Incorrect bucket name or file path
[INFO] [stdout] [2m2025-11-11T07:22:52.403310Z[0m [31mERROR[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m discover_files failed: Failed to get metadata for single file: test-data/code.rs
[INFO] [stdout] [2m2025-11-11T07:22:52.403344Z[0m [31mERROR[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m Error details: Failed to get metadata for single file: test-data/code.rs
[INFO] [stdout] 
[INFO] [stdout] Caused by:
[INFO] [stdout]     0: Failed to get metadata for test-data/code.rs
[INFO] [stdout]     1: S3 HEAD request failed for 'test-data/code.rs': dispatch failure
[INFO] [stdout] 
[INFO] [stdout] Stack backtrace:
[INFO] [stdout]    0: anyhow::error::<impl anyhow::Error>::msg
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/anyhow-1.0.99/src/backtrace.rs:27:14
[INFO] [stdout]    1: ai_workbench_lib::modules::file_discovery::FileDiscovery::get_file_metadata::{{closure}}
[INFO] [stdout]              at ./src/../modules/file_discovery.rs:267:28
[INFO] [stdout]    2: ai_workbench_lib::modules::file_discovery::FileDiscovery::process_single_file::{{closure}}
[INFO] [stdout]              at ./src/../modules/file_discovery.rs:59:64
[INFO] [stdout]    3: ai_workbench_lib::modules::file_discovery::FileDiscovery::discover_files::{{closure}}
[INFO] [stdout]              at ./src/../modules/file_discovery.rs:35:50
[INFO] [stdout]    4: ai_workbench_lib::modules::file_discovery::file_discovery_tests::test_file_type_detection::{{closure}}
[INFO] [stdout]              at ./src/../modules/file_discovery.rs:516:61
[INFO] [stdout]    5: <core::pin::Pin<P> as core::future::future::Future>::poll
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/future/future.rs:133:9
[INFO] [stdout]    6: <core::pin::Pin<P> as core::future::future::Future>::poll
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/future/future.rs:133:9
[INFO] [stdout]    7: tokio::runtime::scheduler::current_thread::CoreGuard::block_on::{{closure}}::{{closure}}::{{closure}}
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:742:70
[INFO] [stdout]    8: tokio::task::coop::with_budget
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/task/coop/mod.rs:167:5
[INFO] [stdout]    9: tokio::task::coop::budget
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/task/coop/mod.rs:133:5
[INFO] [stdout]   10: tokio::runtime::scheduler::current_thread::CoreGuard::block_on::{{closure}}::{{closure}}
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:742:25
[INFO] [stdout]   11: tokio::runtime::scheduler::current_thread::Context::enter
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:432:19
[INFO] [stdout]   12: tokio::runtime::scheduler::current_thread::CoreGuard::block_on::{{closure}}
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:741:44
[INFO] [stdout]   13: tokio::runtime::scheduler::current_thread::CoreGuard::enter::{{closure}}
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:829:68
[INFO] [stdout]   14: tokio::runtime::context::scoped::Scoped<T>::set
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/context/scoped.rs:40:9
[INFO] [stdout]   15: tokio::runtime::context::set_scheduler::{{closure}}
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/context.rs:176:38
[INFO] [stdout]   16: std::thread::local::LocalKey<T>::try_with
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/thread/local.rs:508:12
[INFO] [stdout]   17: std::thread::local::LocalKey<T>::with
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/thread/local.rs:472:20
[INFO] [stdout]   18: tokio::runtime::context::set_scheduler
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/context.rs:176:17
[INFO] [stdout]   19: tokio::runtime::scheduler::current_thread::CoreGuard::enter
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:829:27
[INFO] [stdout]   20: tokio::runtime::scheduler::current_thread::CoreGuard::block_on
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:729:24
[INFO] [stdout]   21: tokio::runtime::scheduler::current_thread::CurrentThread::block_on::{{closure}}
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:200:33
[INFO] [stdout]   22: tokio::runtime::context::runtime::enter_runtime
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/context/runtime.rs:65:16
[INFO] [stdout]   23: tokio::runtime::scheduler::current_thread::CurrentThread::block_on
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:188:9
[INFO] [stdout]   24: tokio::runtime::runtime::Runtime::block_on_inner
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/runtime.rs:356:52
[INFO] [stdout]   25: tokio::runtime::runtime::Runtime::block_on
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/runtime.rs:330:18
[INFO] [stdout]   26: ai_workbench_lib::modules::file_discovery::file_discovery_tests::test_file_type_detection
[INFO] [stdout]              at ./src/../modules/file_discovery.rs:515:36
[INFO] [stdout]   27: ai_workbench_lib::modules::file_discovery::file_discovery_tests::test_file_type_detection::{{closure}}
[INFO] [stdout]              at ./src/../modules/file_discovery.rs:498:40
[INFO] [stdout]   28: core::ops::function::FnOnce::call_once
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/ops/function.rs:250:5
[INFO] [stdout]   29: core::ops::function::FnOnce::call_once
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/ops/function.rs:250:5
[INFO] [stdout]   30: test::__rust_begin_short_backtrace
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/test/src/lib.rs:663:18
[INFO] [stdout]   31: test::run_test_in_process::{{closure}}
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/test/src/lib.rs:686:74
[INFO] [stdout]   32: <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/panic/unwind_safe.rs:274:9
[INFO] [stdout]   33: std::panicking::catch_unwind::do_call
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panicking.rs:590:40
[INFO] [stdout]   34: std::panicking::catch_unwind
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panicking.rs:553:19
[INFO] [stdout]   35: std::panic::catch_unwind
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panic.rs:359:14
[INFO] [stdout]   36: test::run_test_in_process
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/test/src/lib.rs:686:27
[INFO] [stdout]   37: test::run_test::{{closure}}
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/test/src/lib.rs:607:43
[INFO] [stdout]   38: test::run_test::{{closure}}
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/test/src/lib.rs:637:41
[INFO] [stdout]   39: std::sys::backtrace::__rust_begin_short_backtrace
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/sys/backtrace.rs:158:18
[INFO] [stdout]   40: std::thread::Builder::spawn_unchecked_::{{closure}}::{{closure}}
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/thread/mod.rs:562:17
[INFO] [stdout]   41: <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/panic/unwind_safe.rs:274:9
[INFO] [stdout]   42: std::panicking::catch_unwind::do_call
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panicking.rs:590:40
[INFO] [stdout]   43: std::panicking::catch_unwind
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panicking.rs:553:19
[INFO] [stdout]   44: std::panic::catch_unwind
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panic.rs:359:14
[INFO] [stdout]   45: std::thread::Builder::spawn_unchecked_::{{closure}}
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/thread/mod.rs:560:30
[INFO] [stdout]   46: core::ops::function::FnOnce::call_once{{vtable.shim}}
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/ops/function.rs:250:5
[INFO] [stdout]   47: <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/alloc/src/boxed.rs:2085:9
[INFO] [stdout]   48: std::sys::thread::unix::Thread::new::thread_start
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/sys/thread/unix.rs:124:17
[INFO] [stdout]   49: <unknown>
[INFO] [stdout]   50: clone
[INFO] [stdout] [2m2025-11-11T07:22:52.404085Z[0m [32m INFO[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m === FileDiscovery::discover_files DEBUG END ===
[INFO] [stdout] [2m2025-11-11T07:22:52.404169Z[0m [32m INFO[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m === FileDiscovery::discover_files DEBUG START ===
[INFO] [stdout] [2m2025-11-11T07:22:52.404218Z[0m [32m INFO[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m Input spec: 'test-data/styles.css'
[INFO] [stdout] [2m2025-11-11T07:22:52.404244Z[0m [32m INFO[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m Workspace bucket: 'ai-workbench-6c9c43db-7fe6-42f1-8b11-8f82323f83f0-eu-west-2'
[INFO] [stdout] [2m2025-11-11T07:22:52.404273Z[0m [32m INFO[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m Detected single file input (no trailing '/') - processing as single file
[INFO] [stdout] [2m2025-11-11T07:22:52.404331Z[0m [32m INFO[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m === process_single_file DEBUG START ===
[INFO] [stdout] [2m2025-11-11T07:22:52.404359Z[0m [32m INFO[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m Processing single file: 'test-data/styles.css'
[INFO] [stdout] [2m2025-11-11T07:22:52.404384Z[0m [32m INFO[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m Full S3 path: s3://ai-workbench-6c9c43db-7fe6-42f1-8b11-8f82323f83f0-eu-west-2/test-data/styles.css
[INFO] [stdout] [2m2025-11-11T07:22:52.404452Z[0m [32m INFO[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m Attempting to get file metadata...
[INFO] [stdout] [2m2025-11-11T07:22:52.404483Z[0m [32m INFO[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m === get_file_metadata DEBUG START ===
[INFO] [stdout] [2m2025-11-11T07:22:52.404510Z[0m [32m INFO[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m Getting metadata for file: 'test-data/styles.css'
[INFO] [stdout] [2m2025-11-11T07:22:52.404570Z[0m [32m INFO[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m Bucket: 'ai-workbench-6c9c43db-7fe6-42f1-8b11-8f82323f83f0-eu-west-2'
[INFO] [stdout] [2m2025-11-11T07:22:52.404597Z[0m [32m INFO[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m Full S3 path: s3://ai-workbench-6c9c43db-7fe6-42f1-8b11-8f82323f83f0-eu-west-2/test-data/styles.css
[INFO] [stdout] [2m2025-11-11T07:22:52.404626Z[0m [32m INFO[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m Sending HEAD request to S3...
[INFO] [stdout] [2m2025-11-11T07:22:52.407658Z[0m [31mERROR[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m S3 HEAD request failed for 'test-data/styles.css': dispatch failure
[INFO] [stdout] [2m2025-11-11T07:22:52.407766Z[0m [31mERROR[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m Error details: DispatchFailure(DispatchFailure { source: ConnectorError { kind: Other(None), source: CredentialsNotLoaded(CredentialsNotLoaded { source: Some("no providers in chain provided credentials") }), connection: Unknown } })
[INFO] [stdout] [2m2025-11-11T07:22:52.407818Z[0m [31mERROR[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m Possible causes:
[INFO] [stdout] [2m2025-11-11T07:22:52.407889Z[0m [31mERROR[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m   1. File 'test-data/styles.css' does not exist in bucket 'ai-workbench-6c9c43db-7fe6-42f1-8b11-8f82323f83f0-eu-west-2'
[INFO] [stdout] [2m2025-11-11T07:22:52.407925Z[0m [31mERROR[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m   2. No read permissions for this file/bucket
[INFO] [stdout] [2m2025-11-11T07:22:52.407981Z[0m [31mERROR[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m   3. AWS credentials are invalid or expired
[INFO] [stdout] [2m2025-11-11T07:22:52.408024Z[0m [31mERROR[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m   4. Network connectivity issues
[INFO] [stdout] [2m2025-11-11T07:22:52.408055Z[0m [31mERROR[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m   5. S3 service is temporarily unavailable
[INFO] [stdout] [2m2025-11-11T07:22:52.408210Z[0m [31mERROR[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m Failed to get metadata for file 'test-data/styles.css': Failed to get metadata for test-data/styles.css
[INFO] [stdout] [2m2025-11-11T07:22:52.408272Z[0m [31mERROR[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m This usually means:
[INFO] [stdout] [2m2025-11-11T07:22:52.408300Z[0m [31mERROR[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m   1. File does not exist at s3://ai-workbench-6c9c43db-7fe6-42f1-8b11-8f82323f83f0-eu-west-2/test-data/styles.css
[INFO] [stdout] [2m2025-11-11T07:22:52.408354Z[0m [31mERROR[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m   2. No permissions to access the file
[INFO] [stdout] [2m2025-11-11T07:22:52.408398Z[0m [31mERROR[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m   3. S3 service is unavailable
[INFO] [stdout] [2m2025-11-11T07:22:52.408431Z[0m [31mERROR[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m   4. Incorrect bucket name or file path
[INFO] [stdout] [2m2025-11-11T07:22:52.408484Z[0m [31mERROR[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m discover_files failed: Failed to get metadata for single file: test-data/styles.css
[INFO] [stdout] [2m2025-11-11T07:22:52.408536Z[0m [31mERROR[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m Error details: Failed to get metadata for single file: test-data/styles.css
[INFO] [stdout] 
[INFO] [stdout] Caused by:
[INFO] [stdout]     0: Failed to get metadata for test-data/styles.css
[INFO] [stdout]     1: S3 HEAD request failed for 'test-data/styles.css': dispatch failure
[INFO] [stdout] 
[INFO] [stdout] Stack backtrace:
[INFO] [stdout]    0: anyhow::error::<impl anyhow::Error>::msg
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/anyhow-1.0.99/src/backtrace.rs:27:14
[INFO] [stdout]    1: ai_workbench_lib::modules::file_discovery::FileDiscovery::get_file_metadata::{{closure}}
[INFO] [stdout]              at ./src/../modules/file_discovery.rs:267:28
[INFO] [stdout]    2: ai_workbench_lib::modules::file_discovery::FileDiscovery::process_single_file::{{closure}}
[INFO] [stdout]              at ./src/../modules/file_discovery.rs:59:64
[INFO] [stdout]    3: ai_workbench_lib::modules::file_discovery::FileDiscovery::discover_files::{{closure}}
[INFO] [stdout]              at ./src/../modules/file_discovery.rs:35:50
[INFO] [stdout]    4: ai_workbench_lib::modules::file_discovery::file_discovery_tests::test_file_type_detection::{{closure}}
[INFO] [stdout]              at ./src/../modules/file_discovery.rs:516:61
[INFO] [stdout]    5: <core::pin::Pin<P> as core::future::future::Future>::poll
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/future/future.rs:133:9
[INFO] [stdout]    6: <core::pin::Pin<P> as core::future::future::Future>::poll
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/future/future.rs:133:9
[INFO] [stdout]    7: tokio::runtime::scheduler::current_thread::CoreGuard::block_on::{{closure}}::{{closure}}::{{closure}}
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:742:70
[INFO] [stdout]    8: tokio::task::coop::with_budget
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/task/coop/mod.rs:167:5
[INFO] [stdout]    9: tokio::task::coop::budget
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/task/coop/mod.rs:133:5
[INFO] [stdout]   10: tokio::runtime::scheduler::current_thread::CoreGuard::block_on::{{closure}}::{{closure}}
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:742:25
[INFO] [stdout]   11: tokio::runtime::scheduler::current_thread::Context::enter
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:432:19
[INFO] [stdout]   12: tokio::runtime::scheduler::current_thread::CoreGuard::block_on::{{closure}}
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:741:44
[INFO] [stdout]   13: tokio::runtime::scheduler::current_thread::CoreGuard::enter::{{closure}}
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:829:68
[INFO] [stdout]   14: tokio::runtime::context::scoped::Scoped<T>::set
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/context/scoped.rs:40:9
[INFO] [stdout]   15: tokio::runtime::context::set_scheduler::{{closure}}
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/context.rs:176:38
[INFO] [stdout]   16: std::thread::local::LocalKey<T>::try_with
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/thread/local.rs:508:12
[INFO] [stdout]   17: std::thread::local::LocalKey<T>::with
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/thread/local.rs:472:20
[INFO] [stdout]   18: tokio::runtime::context::set_scheduler
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/context.rs:176:17
[INFO] [stdout]   19: tokio::runtime::scheduler::current_thread::CoreGuard::enter
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:829:27
[INFO] [stdout]   20: tokio::runtime::scheduler::current_thread::CoreGuard::block_on
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:729:24
[INFO] [stdout]   21: tokio::runtime::scheduler::current_thread::CurrentThread::block_on::{{closure}}
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:200:33
[INFO] [stdout]   22: tokio::runtime::context::runtime::enter_runtime
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/context/runtime.rs:65:16
[INFO] [stdout]   23: tokio::runtime::scheduler::current_thread::CurrentThread::block_on
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:188:9
[INFO] [stdout]   24: tokio::runtime::runtime::Runtime::block_on_inner
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/runtime.rs:356:52
[INFO] [stdout]   25: tokio::runtime::runtime::Runtime::block_on
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/runtime.rs:330:18
[INFO] [stdout]   26: ai_workbench_lib::modules::file_discovery::file_discovery_tests::test_file_type_detection
[INFO] [stdout]              at ./src/../modules/file_discovery.rs:515:36
[INFO] [stdout]   27: ai_workbench_lib::modules::file_discovery::file_discovery_tests::test_file_type_detection::{{closure}}
[INFO] [stdout]              at ./src/../modules/file_discovery.rs:498:40
[INFO] [stdout]   28: core::ops::function::FnOnce::call_once
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/ops/function.rs:250:5
[INFO] [stdout]   29: core::ops::function::FnOnce::call_once
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/ops/function.rs:250:5
[INFO] [stdout]   30: test::__rust_begin_short_backtrace
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/test/src/lib.rs:663:18
[INFO] [stdout]   31: test::run_test_in_process::{{closure}}
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/test/src/lib.rs:686:74
[INFO] [stdout]   32: <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/panic/unwind_safe.rs:274:9
[INFO] [stdout]   33: std::panicking::catch_unwind::do_call
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panicking.rs:590:40
[INFO] [stdout]   34: std::panicking::catch_unwind
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panicking.rs:553:19
[INFO] [stdout]   35: std::panic::catch_unwind
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panic.rs:359:14
[INFO] [stdout]   36: test::run_test_in_process
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/test/src/lib.rs:686:27
[INFO] [stdout]   37: test::run_test::{{closure}}
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/test/src/lib.rs:607:43
[INFO] [stdout]   38: test::run_test::{{closure}}
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/test/src/lib.rs:637:41
[INFO] [stdout]   39: std::sys::backtrace::__rust_begin_short_backtrace
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/sys/backtrace.rs:158:18
[INFO] [stdout]   40: std::thread::Builder::spawn_unchecked_::{{closure}}::{{closure}}
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/thread/mod.rs:562:17
[INFO] [stdout]   41: <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/panic/unwind_safe.rs:274:9
[INFO] [stdout]   42: std::panicking::catch_unwind::do_call
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panicking.rs:590:40
[INFO] [stdout]   43: std::panicking::catch_unwind
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panicking.rs:553:19
[INFO] [stdout]   44: std::panic::catch_unwind
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panic.rs:359:14
[INFO] [stdout]   45: std::thread::Builder::spawn_unchecked_::{{closure}}
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/thread/mod.rs:560:30
[INFO] [stdout]   46: core::ops::function::FnOnce::call_once{{vtable.shim}}
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/ops/function.rs:250:5
[INFO] [stdout]   47: <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/alloc/src/boxed.rs:2085:9
[INFO] [stdout]   48: std::sys::thread::unix::Thread::new::thread_start
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/sys/thread/unix.rs:124:17
[INFO] [stdout]   49: <unknown>
[INFO] [stdout]   50: clone
[INFO] [stdout] [2m2025-11-11T07:22:52.409300Z[0m [32m INFO[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m === FileDiscovery::discover_files DEBUG END ===
[INFO] [stdout] [2m2025-11-11T07:22:52.409377Z[0m [32m INFO[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m === FileDiscovery::discover_files DEBUG START ===
[INFO] [stdout] [2m2025-11-11T07:22:52.409425Z[0m [32m INFO[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m Input spec: 'test-data/markup.html'
[INFO] [stdout] [2m2025-11-11T07:22:52.409450Z[0m [32m INFO[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m Workspace bucket: 'ai-workbench-6c9c43db-7fe6-42f1-8b11-8f82323f83f0-eu-west-2'
[INFO] [stdout] [2m2025-11-11T07:22:52.409478Z[0m [32m INFO[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m Detected single file input (no trailing '/') - processing as single file
[INFO] [stdout] [2m2025-11-11T07:22:52.409539Z[0m [32m INFO[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m === process_single_file DEBUG START ===
[INFO] [stdout] [2m2025-11-11T07:22:52.409570Z[0m [32m INFO[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m Processing single file: 'test-data/markup.html'
[INFO] [stdout] [2m2025-11-11T07:22:52.409622Z[0m [32m INFO[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m Full S3 path: s3://ai-workbench-6c9c43db-7fe6-42f1-8b11-8f82323f83f0-eu-west-2/test-data/markup.html
[INFO] [stdout] [2m2025-11-11T07:22:52.409675Z[0m [32m INFO[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m Attempting to get file metadata...
[INFO] [stdout] [2m2025-11-11T07:22:52.409706Z[0m [32m INFO[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m === get_file_metadata DEBUG START ===
[INFO] [stdout] [2m2025-11-11T07:22:52.409755Z[0m [32m INFO[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m Getting metadata for file: 'test-data/markup.html'
[INFO] [stdout] [2m2025-11-11T07:22:52.409799Z[0m [32m INFO[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m Bucket: 'ai-workbench-6c9c43db-7fe6-42f1-8b11-8f82323f83f0-eu-west-2'
[INFO] [stdout] [2m2025-11-11T07:22:52.409829Z[0m [32m INFO[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m Full S3 path: s3://ai-workbench-6c9c43db-7fe6-42f1-8b11-8f82323f83f0-eu-west-2/test-data/markup.html
[INFO] [stdout] [2m2025-11-11T07:22:52.409887Z[0m [32m INFO[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m Sending HEAD request to S3...
[INFO] [stdout] [2m2025-11-11T07:22:52.412784Z[0m [31mERROR[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m S3 HEAD request failed for 'test-data/markup.html': dispatch failure
[INFO] [stdout] [2m2025-11-11T07:22:52.412890Z[0m [31mERROR[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m Error details: DispatchFailure(DispatchFailure { source: ConnectorError { kind: Other(None), source: CredentialsNotLoaded(CredentialsNotLoaded { source: Some("no providers in chain provided credentials") }), connection: Unknown } })
[INFO] [stdout] [2m2025-11-11T07:22:52.412972Z[0m [31mERROR[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m Possible causes:
[INFO] [stdout] [2m2025-11-11T07:22:52.413017Z[0m [31mERROR[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m   1. File 'test-data/markup.html' does not exist in bucket 'ai-workbench-6c9c43db-7fe6-42f1-8b11-8f82323f83f0-eu-west-2'
[INFO] [stdout] [2m2025-11-11T07:22:52.413050Z[0m [31mERROR[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m   2. No read permissions for this file/bucket
[INFO] [stdout] [2m2025-11-11T07:22:52.413113Z[0m [31mERROR[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m   3. AWS credentials are invalid or expired
[INFO] [stdout] [2m2025-11-11T07:22:52.413153Z[0m [31mERROR[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m   4. Network connectivity issues
[INFO] [stdout] [2m2025-11-11T07:22:52.413225Z[0m [31mERROR[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m   5. S3 service is temporarily unavailable
[INFO] [stdout] [2m2025-11-11T07:22:52.413344Z[0m [31mERROR[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m Failed to get metadata for file 'test-data/markup.html': Failed to get metadata for test-data/markup.html
[INFO] [stdout] [2m2025-11-11T07:22:52.413404Z[0m [31mERROR[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m This usually means:
[INFO] [stdout] [2m2025-11-11T07:22:52.413443Z[0m [31mERROR[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m   1. File does not exist at s3://ai-workbench-6c9c43db-7fe6-42f1-8b11-8f82323f83f0-eu-west-2/test-data/markup.html
[INFO] [stdout] [2m2025-11-11T07:22:52.413504Z[0m [31mERROR[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m   2. No permissions to access the file
[INFO] [stdout] [2m2025-11-11T07:22:52.413534Z[0m [31mERROR[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m   3. S3 service is unavailable
[INFO] [stdout] [2m2025-11-11T07:22:52.413589Z[0m [31mERROR[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m   4. Incorrect bucket name or file path
[INFO] [stdout] [2m2025-11-11T07:22:52.413621Z[0m [31mERROR[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m discover_files failed: Failed to get metadata for single file: test-data/markup.html
[INFO] [stdout] [2m2025-11-11T07:22:52.413682Z[0m [31mERROR[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m Error details: Failed to get metadata for single file: test-data/markup.html
[INFO] [stdout] 
[INFO] [stdout] Caused by:
[INFO] [stdout]     0: Failed to get metadata for test-data/markup.html
[INFO] [stdout]     1: S3 HEAD request failed for 'test-data/markup.html': dispatch failure
[INFO] [stdout] 
[INFO] [stdout] Stack backtrace:
[INFO] [stdout]    0: anyhow::error::<impl anyhow::Error>::msg
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/anyhow-1.0.99/src/backtrace.rs:27:14
[INFO] [stdout]    1: ai_workbench_lib::modules::file_discovery::FileDiscovery::get_file_metadata::{{closure}}
[INFO] [stdout]              at ./src/../modules/file_discovery.rs:267:28
[INFO] [stdout]    2: ai_workbench_lib::modules::file_discovery::FileDiscovery::process_single_file::{{closure}}
[INFO] [stdout]              at ./src/../modules/file_discovery.rs:59:64
[INFO] [stdout]    3: ai_workbench_lib::modules::file_discovery::FileDiscovery::discover_files::{{closure}}
[INFO] [stdout]              at ./src/../modules/file_discovery.rs:35:50
[INFO] [stdout]    4: ai_workbench_lib::modules::file_discovery::file_discovery_tests::test_file_type_detection::{{closure}}
[INFO] [stdout]              at ./src/../modules/file_discovery.rs:516:61
[INFO] [stdout]    5: <core::pin::Pin<P> as core::future::future::Future>::poll
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/future/future.rs:133:9
[INFO] [stdout]    6: <core::pin::Pin<P> as core::future::future::Future>::poll
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/future/future.rs:133:9
[INFO] [stdout]    7: tokio::runtime::scheduler::current_thread::CoreGuard::block_on::{{closure}}::{{closure}}::{{closure}}
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:742:70
[INFO] [stdout]    8: tokio::task::coop::with_budget
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/task/coop/mod.rs:167:5
[INFO] [stdout]    9: tokio::task::coop::budget
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/task/coop/mod.rs:133:5
[INFO] [stdout]   10: tokio::runtime::scheduler::current_thread::CoreGuard::block_on::{{closure}}::{{closure}}
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:742:25
[INFO] [stdout]   11: tokio::runtime::scheduler::current_thread::Context::enter
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:432:19
[INFO] [stdout]   12: tokio::runtime::scheduler::current_thread::CoreGuard::block_on::{{closure}}
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:741:44
[INFO] [stdout]   13: tokio::runtime::scheduler::current_thread::CoreGuard::enter::{{closure}}
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:829:68
[INFO] [stdout]   14: tokio::runtime::context::scoped::Scoped<T>::set
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/context/scoped.rs:40:9
[INFO] [stdout]   15: tokio::runtime::context::set_scheduler::{{closure}}
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/context.rs:176:38
[INFO] [stdout]   16: std::thread::local::LocalKey<T>::try_with
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/thread/local.rs:508:12
[INFO] [stdout]   17: std::thread::local::LocalKey<T>::with
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/thread/local.rs:472:20
[INFO] [stdout]   18: tokio::runtime::context::set_scheduler
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/context.rs:176:17
[INFO] [stdout]   19: tokio::runtime::scheduler::current_thread::CoreGuard::enter
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:829:27
[INFO] [stdout]   20: tokio::runtime::scheduler::current_thread::CoreGuard::block_on
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:729:24
[INFO] [stdout]   21: tokio::runtime::scheduler::current_thread::CurrentThread::block_on::{{closure}}
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:200:33
[INFO] [stdout]   22: tokio::runtime::context::runtime::enter_runtime
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/context/runtime.rs:65:16
[INFO] [stdout]   23: tokio::runtime::scheduler::current_thread::CurrentThread::block_on
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/current_thread/mod.rs:188:9
[INFO] [stdout]   24: tokio::runtime::runtime::Runtime::block_on_inner
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/runtime.rs:356:52
[INFO] [stdout]   25: tokio::runtime::runtime::Runtime::block_on
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/runtime.rs:330:18
[INFO] [stdout]   26: ai_workbench_lib::modules::file_discovery::file_discovery_tests::test_file_type_detection
[INFO] [stdout]              at ./src/../modules/file_discovery.rs:515:36
[INFO] [stdout]   27: ai_workbench_lib::modules::file_discovery::file_discovery_tests::test_file_type_detection::{{closure}}
[INFO] [stdout]              at ./src/../modules/file_discovery.rs:498:40
[INFO] [stdout]   28: core::ops::function::FnOnce::call_once
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/ops/function.rs:250:5
[INFO] [stdout]   29: core::ops::function::FnOnce::call_once
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/ops/function.rs:250:5
[INFO] [stdout]   30: test::__rust_begin_short_backtrace
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/test/src/lib.rs:663:18
[INFO] [stdout]   31: test::run_test_in_process::{{closure}}
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/test/src/lib.rs:686:74
[INFO] [stdout]   32: <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/panic/unwind_safe.rs:274:9
[INFO] [stdout]   33: std::panicking::catch_unwind::do_call
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panicking.rs:590:40
[INFO] [stdout]   34: std::panicking::catch_unwind
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panicking.rs:553:19
[INFO] [stdout]   35: std::panic::catch_unwind
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panic.rs:359:14
[INFO] [stdout]   36: test::run_test_in_process
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/test/src/lib.rs:686:27
[INFO] [stdout]   37: test::run_test::{{closure}}
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/test/src/lib.rs:607:43
[INFO] [stdout]   38: test::run_test::{{closure}}
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/test/src/lib.rs:637:41
[INFO] [stdout]   39: std::sys::backtrace::__rust_begin_short_backtrace
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/sys/backtrace.rs:158:18
[INFO] [stdout]   40: std::thread::Builder::spawn_unchecked_::{{closure}}::{{closure}}
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/thread/mod.rs:562:17
[INFO] [stdout]   41: <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/panic/unwind_safe.rs:274:9
[INFO] [stdout]   42: std::panicking::catch_unwind::do_call
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panicking.rs:590:40
[INFO] [stdout]   43: std::panicking::catch_unwind
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panicking.rs:553:19
[INFO] [stdout]   44: std::panic::catch_unwind
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panic.rs:359:14
[INFO] [stdout]   45: std::thread::Builder::spawn_unchecked_::{{closure}}
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/thread/mod.rs:560:30
[INFO] [stdout]   46: core::ops::function::FnOnce::call_once{{vtable.shim}}
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/ops/function.rs:250:5
[INFO] [stdout]   47: <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/alloc/src/boxed.rs:2085:9
[INFO] [stdout]   48: std::sys::thread::unix::Thread::new::thread_start
[INFO] [stdout]              at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/sys/thread/unix.rs:124:17
[INFO] [stdout]   49: <unknown>
[INFO] [stdout]   50: clone
[INFO] [stdout] [2m2025-11-11T07:22:52.414405Z[0m [32m INFO[0m [2mai_workbench_lib::modules::file_discovery[0m[2m:[0m === FileDiscovery::discover_files DEBUG END ===
[INFO] [stdout] test modules::file_discovery::file_discovery_tests::test_file_type_detection ... ok
[INFO] [stdout] 
[INFO] [stdout] failures:
[INFO] [stdout] 
[INFO] [stdout] ---- modules::file_splitter::splitter::integration_tests::test_mixed_file_types stdout ----
[INFO] [stdout] 
[INFO] [stdout] thread 'modules::file_splitter::splitter::integration_tests::test_mixed_file_types' (34) panicked at src/../modules/file_splitter/splitter.rs:245:13:
[INFO] [stdout] assertion `left == right` failed
[INFO] [stdout]   left: Text
[INFO] [stdout]  right: Xml
[INFO] [stdout] stack backtrace:
[INFO] [stdout]    0:     0x56b582df1172 - std::backtrace_rs::backtrace::libunwind::trace::h9ea1e07ac77a25d7
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/../../backtrace/src/backtrace/libunwind.rs:117:9
[INFO] [stdout]    1:     0x56b582df1172 - std::backtrace_rs::backtrace::trace_unsynchronized::h72b75eb83b53d15a
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/../../backtrace/src/backtrace/mod.rs:66:14
[INFO] [stdout]    2:     0x56b582df1172 - std::sys::backtrace::_print_fmt::h255777b7ec902439
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/sys/backtrace.rs:66:9
[INFO] [stdout]    3:     0x56b582df1172 - <std::sys::backtrace::BacktraceLock::print::DisplayBacktrace as core::fmt::Display>::fmt::h8a1ac49fcd7ee8ce
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/sys/backtrace.rs:39:26
[INFO] [stdout]    4:     0x56b582e047cf - core::fmt::rt::Argument::fmt::h635cbf3c6754cc90
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/fmt/rt.rs:173:76
[INFO] [stdout]    5:     0x56b582e047cf - core::fmt::write::hbdcc0ec4f1bab1db
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/fmt/mod.rs:1469:25
[INFO] [stdout]    6:     0x56b582dbb323 - std::io::default_write_fmt::h2580ece0d4b58e51
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/io/mod.rs:639:11
[INFO] [stdout]    7:     0x56b582dbb323 - std::io::Write::write_fmt::hac9e42858f273b87
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/io/mod.rs:1954:13
[INFO] [stdout]    8:     0x56b582dc81f2 - std::sys::backtrace::BacktraceLock::print::h052febbd86f7bf07
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/sys/backtrace.rs:42:9
[INFO] [stdout]    9:     0x56b582dcdd9f - std::panicking::default_hook::{{closure}}::h66aeee20dd470abb
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panicking.rs:301:27
[INFO] [stdout]   10:     0x56b582dcdc31 - std::panicking::default_hook::h9d66b9642014ce48
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panicking.rs:325:9
[INFO] [stdout]   11:     0x56b581b5a77e - <alloc::boxed::Box<F,A> as core::ops::function::Fn<Args>>::call::h8c698701e444e564
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/alloc/src/boxed.rs:2099:9
[INFO] [stdout]   12:     0x56b581b5a77e - test::test_main_with_exit_callback::{{closure}}::ha1c98e7b69c4defb
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/test/src/lib.rs:145:21
[INFO] [stdout]   13:     0x56b582dce45f - <alloc::boxed::Box<F,A> as core::ops::function::Fn<Args>>::call::hc4f222a8a57c637a
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/alloc/src/boxed.rs:2099:9
[INFO] [stdout]   14:     0x56b582dce45f - std::panicking::panic_with_hook::h6d5921a1a1fa5a8e
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panicking.rs:842:13
[INFO] [stdout]   15:     0x56b582dce20a - std::panicking::panic_handler::{{closure}}::hafeef6f707d43542
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panicking.rs:707:13
[INFO] [stdout]   16:     0x56b582dc8329 - std::sys::backtrace::__rust_end_short_backtrace::h08cbc3319a3d0120
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/sys/backtrace.rs:174:18
[INFO] [stdout]   17:     0x56b582daef3d - __rustc[ce5c411ff86ab1b2]::rust_begin_unwind
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panicking.rs:698:5
[INFO] [stdout]   18:     0x56b582e0f630 - core::panicking::panic_fmt::h774fb860369a0f7b
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/panicking.rs:80:14
[INFO] [stdout]   19:     0x56b582e0f3e3 - core::panicking::assert_failed_inner::hbdde3a71af798a31
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/panicking.rs:444:17
[INFO] [stdout]   20:     0x56b581ac1793 - core::panicking::assert_failed::h76a660de6296b1e9
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/panicking.rs:399:5
[INFO] [stdout]   21:     0x56b581a6b93c - ai_workbench_lib::modules::file_splitter::splitter::integration_tests::test_mixed_file_types::h047f5bfa0ccc43ef
[INFO] [stdout]                                at /opt/rustwide/workdir/src/../modules/file_splitter/splitter.rs:245:13
[INFO] [stdout]   22:     0x56b581a6ba17 - ai_workbench_lib::modules::file_splitter::splitter::integration_tests::test_mixed_file_types::{{closure}}::h8743ba9eae5155a0
[INFO] [stdout]                                at /opt/rustwide/workdir/src/../modules/file_splitter/splitter.rs:231:31
[INFO] [stdout]   23:     0x56b581a96576 - core::ops::function::FnOnce::call_once::ha11b70c710cdee31
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/ops/function.rs:250:5
[INFO] [stdout]   24:     0x56b581b5a5cb - core::ops::function::FnOnce::call_once::h4f6eabe90cccb47d
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/ops/function.rs:250:5
[INFO] [stdout]   25:     0x56b581b5a5cb - test::__rust_begin_short_backtrace::h94208530f2b4c8c7
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/test/src/lib.rs:663:18
[INFO] [stdout]   26:     0x56b581b6e5dd - test::run_test_in_process::{{closure}}::hbdc6a45a78c46404
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/test/src/lib.rs:686:74
[INFO] [stdout]   27:     0x56b581b6e5dd - <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once::h7f9d8efc7eedd165
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/panic/unwind_safe.rs:274:9
[INFO] [stdout]   28:     0x56b581b6e5dd - std::panicking::catch_unwind::do_call::he5edbaf126c13b99
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panicking.rs:590:40
[INFO] [stdout]   29:     0x56b581b6e5dd - std::panicking::catch_unwind::h3d6a8652eceeabce
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panicking.rs:553:19
[INFO] [stdout]   30:     0x56b581b6e5dd - std::panic::catch_unwind::hbef0c501dd7bb498
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panic.rs:359:14
[INFO] [stdout]   31:     0x56b581b6e5dd - test::run_test_in_process::h400008a46a5006a0
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/test/src/lib.rs:686:27
[INFO] [stdout]   32:     0x56b581b6e5dd - test::run_test::{{closure}}::h9a131655b31d9427
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/test/src/lib.rs:607:43
[INFO] [stdout]   33:     0x56b581b47b24 - test::run_test::{{closure}}::he4a7177c216308c0
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/test/src/lib.rs:637:41
[INFO] [stdout]   34:     0x56b581b47b24 - std::sys::backtrace::__rust_begin_short_backtrace::hd3980e9c55d6c539
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/sys/backtrace.rs:158:18
[INFO] [stdout]   35:     0x56b581b4b45a - std::thread::Builder::spawn_unchecked_::{{closure}}::{{closure}}::h18552df67c7aa9b8
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/thread/mod.rs:562:17
[INFO] [stdout]   36:     0x56b581b4b45a - <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once::h3d7b285234641a08
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/panic/unwind_safe.rs:274:9
[INFO] [stdout]   37:     0x56b581b4b45a - std::panicking::catch_unwind::do_call::h5a2b5cf36528a5b2
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panicking.rs:590:40
[INFO] [stdout]   38:     0x56b581b4b45a - std::panicking::catch_unwind::h7e8b531bbda77d20
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panicking.rs:553:19
[INFO] [stdout]   39:     0x56b581b4b45a - std::panic::catch_unwind::hf9bbb2040b4f1e6c
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panic.rs:359:14
[INFO] [stdout]   40:     0x56b581b4b45a - std::thread::Builder::spawn_unchecked_::{{closure}}::h600f92cf68549574
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/thread/mod.rs:560:30
[INFO] [stdout]   41:     0x56b581b4b45a - core::ops::function::FnOnce::call_once{{vtable.shim}}::hb6607307df8a1847
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/ops/function.rs:250:5
[INFO] [stdout]   42:     0x56b582dc364f - <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once::h6891ad53a5ed6f52
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/alloc/src/boxed.rs:2085:9
[INFO] [stdout]   43:     0x56b582dc364f - std::sys::thread::unix::Thread::new::thread_start::h7f3e77fa86df70af
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/sys/thread/unix.rs:124:17
[INFO] [stdout]   44:     0x70def9591aa4 - <unknown>
[INFO] [stdout]   45:     0x70def961ea64 - clone
[INFO] [stdout]   46:                0x0 - <unknown>
[INFO] [stdout] 
[INFO] [stdout] ---- modules::file_splitter::splitter::tests::test_binary_file_rejection stdout ----
[INFO] [stdout] 
[INFO] [stdout] thread 'modules::file_splitter::splitter::tests::test_binary_file_rejection' (36) panicked at src/../modules/file_splitter/splitter.rs:137:9:
[INFO] [stdout] assertion failed: result.is_err()
[INFO] [stdout] stack backtrace:
[INFO] [stdout]    0:     0x56b582df1172 - std::backtrace_rs::backtrace::libunwind::trace::h9ea1e07ac77a25d7
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/../../backtrace/src/backtrace/libunwind.rs:117:9
[INFO] [stdout]    1:     0x56b582df1172 - std::backtrace_rs::backtrace::trace_unsynchronized::h72b75eb83b53d15a
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/../../backtrace/src/backtrace/mod.rs:66:14
[INFO] [stdout]    2:     0x56b582df1172 - std::sys::backtrace::_print_fmt::h255777b7ec902439
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/sys/backtrace.rs:66:9
[INFO] [stdout]    3:     0x56b582df1172 - <std::sys::backtrace::BacktraceLock::print::DisplayBacktrace as core::fmt::Display>::fmt::h8a1ac49fcd7ee8ce
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/sys/backtrace.rs:39:26
[INFO] [stdout]    4:     0x56b582e047cf - core::fmt::rt::Argument::fmt::h635cbf3c6754cc90
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/fmt/rt.rs:173:76
[INFO] [stdout]    5:     0x56b582e047cf - core::fmt::write::hbdcc0ec4f1bab1db
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/fmt/mod.rs:1469:25
[INFO] [stdout]    6:     0x56b582dbb323 - std::io::default_write_fmt::h2580ece0d4b58e51
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/io/mod.rs:639:11
[INFO] [stdout]    7:     0x56b582dbb323 - std::io::Write::write_fmt::hac9e42858f273b87
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/io/mod.rs:1954:13
[INFO] [stdout]    8:     0x56b582dc81f2 - std::sys::backtrace::BacktraceLock::print::h052febbd86f7bf07
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/sys/backtrace.rs:42:9
[INFO] [stdout]    9:     0x56b582dcdd9f - std::panicking::default_hook::{{closure}}::h66aeee20dd470abb
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panicking.rs:301:27
[INFO] [stdout]   10:     0x56b582dcdc31 - std::panicking::default_hook::h9d66b9642014ce48
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panicking.rs:325:9
[INFO] [stdout]   11:     0x56b581b5a77e - <alloc::boxed::Box<F,A> as core::ops::function::Fn<Args>>::call::h8c698701e444e564
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/alloc/src/boxed.rs:2099:9
[INFO] [stdout]   12:     0x56b581b5a77e - test::test_main_with_exit_callback::{{closure}}::ha1c98e7b69c4defb
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/test/src/lib.rs:145:21
[INFO] [stdout]   13:     0x56b582dce45f - <alloc::boxed::Box<F,A> as core::ops::function::Fn<Args>>::call::hc4f222a8a57c637a
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/alloc/src/boxed.rs:2099:9
[INFO] [stdout]   14:     0x56b582dce45f - std::panicking::panic_with_hook::h6d5921a1a1fa5a8e
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panicking.rs:842:13
[INFO] [stdout]   15:     0x56b582dce1d6 - std::panicking::panic_handler::{{closure}}::hafeef6f707d43542
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panicking.rs:700:13
[INFO] [stdout]   16:     0x56b582dc8329 - std::sys::backtrace::__rust_end_short_backtrace::h08cbc3319a3d0120
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/sys/backtrace.rs:174:18
[INFO] [stdout]   17:     0x56b582daef3d - __rustc[ce5c411ff86ab1b2]::rust_begin_unwind
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panicking.rs:698:5
[INFO] [stdout]   18:     0x56b582e0f630 - core::panicking::panic_fmt::h774fb860369a0f7b
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/panicking.rs:80:14
[INFO] [stdout]   19:     0x56b582e0f60c - core::panicking::panic::h2a6fe4ac012a7582
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/panicking.rs:150:5
[INFO] [stdout]   20:     0x56b581a6dd60 - ai_workbench_lib::modules::file_splitter::splitter::tests::test_binary_file_rejection::hdf5c313d6d6f58ad
[INFO] [stdout]                                at /opt/rustwide/workdir/src/../modules/file_splitter/splitter.rs:137:9
[INFO] [stdout]   21:     0x56b581a6dda7 - ai_workbench_lib::modules::file_splitter::splitter::tests::test_binary_file_rejection::{{closure}}::h2bac494174188f01
[INFO] [stdout]                                at /opt/rustwide/workdir/src/../modules/file_splitter/splitter.rs:130:36
[INFO] [stdout]   22:     0x56b581a96216 - core::ops::function::FnOnce::call_once::h71199977bc1bef86
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/ops/function.rs:250:5
[INFO] [stdout]   23:     0x56b581b5a5cb - core::ops::function::FnOnce::call_once::h4f6eabe90cccb47d
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/ops/function.rs:250:5
[INFO] [stdout]   24:     0x56b581b5a5cb - test::__rust_begin_short_backtrace::h94208530f2b4c8c7
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/test/src/lib.rs:663:18
[INFO] [stdout]   25:     0x56b581b6e5dd - test::run_test_in_process::{{closure}}::hbdc6a45a78c46404
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/test/src/lib.rs:686:74
[INFO] [stdout]   26:     0x56b581b6e5dd - <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once::h7f9d8efc7eedd165
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/panic/unwind_safe.rs:274:9
[INFO] [stdout]   27:     0x56b581b6e5dd - std::panicking::catch_unwind::do_call::he5edbaf126c13b99
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panicking.rs:590:40
[INFO] [stdout]   28:     0x56b581b6e5dd - std::panicking::catch_unwind::h3d6a8652eceeabce
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panicking.rs:553:19
[INFO] [stdout]   29:     0x56b581b6e5dd - std::panic::catch_unwind::hbef0c501dd7bb498
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panic.rs:359:14
[INFO] [stdout]   30:     0x56b581b6e5dd - test::run_test_in_process::h400008a46a5006a0
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/test/src/lib.rs:686:27
[INFO] [stdout]   31:     0x56b581b6e5dd - test::run_test::{{closure}}::h9a131655b31d9427
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/test/src/lib.rs:607:43
[INFO] [stdout]   32:     0x56b581b47b24 - test::run_test::{{closure}}::he4a7177c216308c0
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/test/src/lib.rs:637:41
[INFO] [stdout]   33:     0x56b581b47b24 - std::sys::backtrace::__rust_begin_short_backtrace::hd3980e9c55d6c539
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/sys/backtrace.rs:158:18
[INFO] [stdout]   34:     0x56b581b4b45a - std::thread::Builder::spawn_unchecked_::{{closure}}::{{closure}}::h18552df67c7aa9b8
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/thread/mod.rs:562:17
[INFO] [stdout]   35:     0x56b581b4b45a - <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once::h3d7b285234641a08
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/panic/unwind_safe.rs:274:9
[INFO] [stdout]   36:     0x56b581b4b45a - std::panicking::catch_unwind::do_call::h5a2b5cf36528a5b2
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panicking.rs:590:40
[INFO] [stdout]   37:     0x56b581b4b45a - std::panicking::catch_unwind::h7e8b531bbda77d20
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panicking.rs:553:19
[INFO] [stdout]   38:     0x56b581b4b45a - std::panic::catch_unwind::hf9bbb2040b4f1e6c
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panic.rs:359:14
[INFO] [stdout]   39:     0x56b581b4b45a - std::thread::Builder::spawn_unchecked_::{{closure}}::h600f92cf68549574
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/thread/mod.rs:560:30
[INFO] [stdout]   40:     0x56b581b4b45a - core::ops::function::FnOnce::call_once{{vtable.shim}}::hb6607307df8a1847
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/ops/function.rs:250:5
[INFO] [stdout]   41:     0x56b582dc364f - <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once::h6891ad53a5ed6f52
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/alloc/src/boxed.rs:2085:9
[INFO] [stdout]   42:     0x56b582dc364f - std::sys::thread::unix::Thread::new::thread_start::h7f3e77fa86df70af
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/sys/thread/unix.rs:124:17
[INFO] [stdout]   43:     0x70def9591aa4 - <unknown>
[INFO] [stdout]   44:     0x70def961ea64 - clone
[INFO] [stdout]   45:                0x0 - <unknown>
[INFO] [stdout] 
[INFO] [stdout] ---- modules::file_splitter::splitter_types::json_splitter::tests::test_json_array_detection stdout ----
[INFO] [stdout] 
[INFO] [stdout] thread 'modules::file_splitter::splitter_types::json_splitter::tests::test_json_array_detection' (49) panicked at src/../modules/file_splitter/splitter_types/json_splitter.rs:364:9:
[INFO] [stdout] assertion `left == right` failed
[INFO] [stdout]   left: JsonLines
[INFO] [stdout]  right: Standard
[INFO] [stdout] stack backtrace:
[INFO] [stdout]    0:     0x56b582df1172 - std::backtrace_rs::backtrace::libunwind::trace::h9ea1e07ac77a25d7
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/../../backtrace/src/backtrace/libunwind.rs:117:9
[INFO] [stdout]    1:     0x56b582df1172 - std::backtrace_rs::backtrace::trace_unsynchronized::h72b75eb83b53d15a
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/../../backtrace/src/backtrace/mod.rs:66:14
[INFO] [stdout]    2:     0x56b582df1172 - std::sys::backtrace::_print_fmt::h255777b7ec902439
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/sys/backtrace.rs:66:9
[INFO] [stdout]    3:     0x56b582df1172 - <std::sys::backtrace::BacktraceLock::print::DisplayBacktrace as core::fmt::Display>::fmt::h8a1ac49fcd7ee8ce
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/sys/backtrace.rs:39:26
[INFO] [stdout]    4:     0x56b582e047cf - core::fmt::rt::Argument::fmt::h635cbf3c6754cc90
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/fmt/rt.rs:173:76
[INFO] [stdout]    5:     0x56b582e047cf - core::fmt::write::hbdcc0ec4f1bab1db
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/fmt/mod.rs:1469:25
[INFO] [stdout]    6:     0x56b582dbb323 - std::io::default_write_fmt::h2580ece0d4b58e51
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/io/mod.rs:639:11
[INFO] [stdout]    7:     0x56b582dbb323 - std::io::Write::write_fmt::hac9e42858f273b87
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/io/mod.rs:1954:13
[INFO] [stdout]    8:     0x56b582dc81f2 - std::sys::backtrace::BacktraceLock::print::h052febbd86f7bf07
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/sys/backtrace.rs:42:9
[INFO] [stdout]    9:     0x56b582dcdd9f - std::panicking::default_hook::{{closure}}::h66aeee20dd470abb
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panicking.rs:301:27
[INFO] [stdout]   10:     0x56b582dcdc31 - std::panicking::default_hook::h9d66b9642014ce48
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panicking.rs:325:9
[INFO] [stdout]   11:     0x56b581b5a77e - <alloc::boxed::Box<F,A> as core::ops::function::Fn<Args>>::call::h8c698701e444e564
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/alloc/src/boxed.rs:2099:9
[INFO] [stdout]   12:     0x56b581b5a77e - test::test_main_with_exit_callback::{{closure}}::ha1c98e7b69c4defb
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/test/src/lib.rs:145:21
[INFO] [stdout]   13:     0x56b582dce45f - <alloc::boxed::Box<F,A> as core::ops::function::Fn<Args>>::call::hc4f222a8a57c637a
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/alloc/src/boxed.rs:2099:9
[INFO] [stdout]   14:     0x56b582dce45f - std::panicking::panic_with_hook::h6d5921a1a1fa5a8e
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panicking.rs:842:13
[INFO] [stdout]   15:     0x56b582dce20a - std::panicking::panic_handler::{{closure}}::hafeef6f707d43542
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panicking.rs:707:13
[INFO] [stdout]   16:     0x56b582dc8329 - std::sys::backtrace::__rust_end_short_backtrace::h08cbc3319a3d0120
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/sys/backtrace.rs:174:18
[INFO] [stdout]   17:     0x56b582daef3d - __rustc[ce5c411ff86ab1b2]::rust_begin_unwind
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panicking.rs:698:5
[INFO] [stdout]   18:     0x56b582e0f630 - core::panicking::panic_fmt::h774fb860369a0f7b
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/panicking.rs:80:14
[INFO] [stdout]   19:     0x56b582e0f3e3 - core::panicking::assert_failed_inner::hbdde3a71af798a31
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/panicking.rs:444:17
[INFO] [stdout]   20:     0x56b581ac17d2 - core::panicking::assert_failed::ha24034a1566b3333
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/panicking.rs:399:5
[INFO] [stdout]   21:     0x56b581a69059 - ai_workbench_lib::modules::file_splitter::splitter_types::json_splitter::tests::test_json_array_detection::h92b0a8da361c7288
[INFO] [stdout]                                at /opt/rustwide/workdir/src/../modules/file_splitter/splitter_types/json_splitter.rs:364:9
[INFO] [stdout]   22:     0x56b581a69087 - ai_workbench_lib::modules::file_splitter::splitter_types::json_splitter::tests::test_json_array_detection::{{closure}}::hb79ebdeed91abd8a
[INFO] [stdout]                                at /opt/rustwide/workdir/src/../modules/file_splitter/splitter_types/json_splitter.rs:358:35
[INFO] [stdout]   23:     0x56b581a96886 - core::ops::function::FnOnce::call_once::he67507a381cd727b
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/ops/function.rs:250:5
[INFO] [stdout]   24:     0x56b581b5a5cb - core::ops::function::FnOnce::call_once::h4f6eabe90cccb47d
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/ops/function.rs:250:5
[INFO] [stdout]   25:     0x56b581b5a5cb - test::__rust_begin_short_backtrace::h94208530f2b4c8c7
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/test/src/lib.rs:663:18
[INFO] [stdout]   26:     0x56b581b6e5dd - test::run_test_in_process::{{closure}}::hbdc6a45a78c46404
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/test/src/lib.rs:686:74
[INFO] [stdout]   27:     0x56b581b6e5dd - <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once::h7f9d8efc7eedd165
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/panic/unwind_safe.rs:274:9
[INFO] [stdout]   28:     0x56b581b6e5dd - std::panicking::catch_unwind::do_call::he5edbaf126c13b99
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panicking.rs:590:40
[INFO] [stdout]   29:     0x56b581b6e5dd - std::panicking::catch_unwind::h3d6a8652eceeabce
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panicking.rs:553:19
[INFO] [stdout]   30:     0x56b581b6e5dd - std::panic::catch_unwind::hbef0c501dd7bb498
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panic.rs:359:14
[INFO] [stdout]   31:     0x56b581b6e5dd - test::run_test_in_process::h400008a46a5006a0
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/test/src/lib.rs:686:27
[INFO] [stdout]   32:     0x56b581b6e5dd - test::run_test::{{closure}}::h9a131655b31d9427
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/test/src/lib.rs:607:43
[INFO] [stdout]   33:     0x56b581b47b24 - test::run_test::{{closure}}::he4a7177c216308c0
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/test/src/lib.rs:637:41
[INFO] [stdout]   34:     0x56b581b47b24 - std::sys::backtrace::__rust_begin_short_backtrace::hd3980e9c55d6c539
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/sys/backtrace.rs:158:18
[INFO] [stdout]   35:     0x56b581b4b45a - std::thread::Builder::spawn_unchecked_::{{closure}}::{{closure}}::h18552df67c7aa9b8
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/thread/mod.rs:562:17
[INFO] [stdout]   36:     0x56b581b4b45a - <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once::h3d7b285234641a08
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/panic/unwind_safe.rs:274:9
[INFO] [stdout]   37:     0x56b581b4b45a - std::panicking::catch_unwind::do_call::h5a2b5cf36528a5b2
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panicking.rs:590:40
[INFO] [stdout]   38:     0x56b581b4b45a - std::panicking::catch_unwind::h7e8b531bbda77d20
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panicking.rs:553:19
[INFO] [stdout]   39:     0x56b581b4b45a - std::panic::catch_unwind::hf9bbb2040b4f1e6c
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panic.rs:359:14
[INFO] [stdout]   40:     0x56b581b4b45a - std::thread::Builder::spawn_unchecked_::{{closure}}::h600f92cf68549574
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/thread/mod.rs:560:30
[INFO] [stdout]   41:     0x56b581b4b45a - core::ops::function::FnOnce::call_once{{vtable.shim}}::hb6607307df8a1847
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/ops/function.rs:250:5
[INFO] [stdout]   42:     0x56b582dc364f - <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once::h6891ad53a5ed6f52
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/alloc/src/boxed.rs:2085:9
[INFO] [stdout]   43:     0x56b582dc364f - std::sys::thread::unix::Thread::new::thread_start::h7f3e77fa86df70af
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/sys/thread/unix.rs:124:17
[INFO] [stdout]   44:     0x70def9591aa4 - <unknown>
[INFO] [stdout]   45:     0x70def961ea64 - clone
[INFO] [stdout]   46:                0x0 - <unknown>
[INFO] [stdout] 
[INFO] [stdout] ---- modules::file_splitter::splitter_types::json_splitter::tests::test_standard_json_detection stdout ----
[INFO] [stdout] 
[INFO] [stdout] thread 'modules::file_splitter::splitter_types::json_splitter::tests::test_standard_json_detection' (52) panicked at src/../modules/file_splitter/splitter_types/json_splitter.rs:354:9:
[INFO] [stdout] assertion `left == right` failed
[INFO] [stdout]   left: JsonLines
[INFO] [stdout]  right: Standard
[INFO] [stdout] stack backtrace:
[INFO] [stdout]    0:     0x56b582df1172 - std::backtrace_rs::backtrace::libunwind::trace::h9ea1e07ac77a25d7
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/../../backtrace/src/backtrace/libunwind.rs:117:9
[INFO] [stdout]    1:     0x56b582df1172 - std::backtrace_rs::backtrace::trace_unsynchronized::h72b75eb83b53d15a
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/../../backtrace/src/backtrace/mod.rs:66:14
[INFO] [stdout]    2:     0x56b582df1172 - std::sys::backtrace::_print_fmt::h255777b7ec902439
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/sys/backtrace.rs:66:9
[INFO] [stdout]    3:     0x56b582df1172 - <std::sys::backtrace::BacktraceLock::print::DisplayBacktrace as core::fmt::Display>::fmt::h8a1ac49fcd7ee8ce
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/sys/backtrace.rs:39:26
[INFO] [stdout]    4:     0x56b582e047cf - core::fmt::rt::Argument::fmt::h635cbf3c6754cc90
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/fmt/rt.rs:173:76
[INFO] [stdout]    5:     0x56b582e047cf - core::fmt::write::hbdcc0ec4f1bab1db
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/fmt/mod.rs:1469:25
[INFO] [stdout]    6:     0x56b582dbb323 - std::io::default_write_fmt::h2580ece0d4b58e51
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/io/mod.rs:639:11
[INFO] [stdout]    7:     0x56b582dbb323 - std::io::Write::write_fmt::hac9e42858f273b87
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/io/mod.rs:1954:13
[INFO] [stdout]    8:     0x56b582dc81f2 - std::sys::backtrace::BacktraceLock::print::h052febbd86f7bf07
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/sys/backtrace.rs:42:9
[INFO] [stdout]    9:     0x56b582dcdd9f - std::panicking::default_hook::{{closure}}::h66aeee20dd470abb
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panicking.rs:301:27
[INFO] [stdout]   10:     0x56b582dcdc31 - std::panicking::default_hook::h9d66b9642014ce48
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panicking.rs:325:9
[INFO] [stdout]   11:     0x56b581b5a77e - <alloc::boxed::Box<F,A> as core::ops::function::Fn<Args>>::call::h8c698701e444e564
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/alloc/src/boxed.rs:2099:9
[INFO] [stdout]   12:     0x56b581b5a77e - test::test_main_with_exit_callback::{{closure}}::ha1c98e7b69c4defb
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/test/src/lib.rs:145:21
[INFO] [stdout]   13:     0x56b582dce45f - <alloc::boxed::Box<F,A> as core::ops::function::Fn<Args>>::call::hc4f222a8a57c637a
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/alloc/src/boxed.rs:2099:9
[INFO] [stdout]   14:     0x56b582dce45f - std::panicking::panic_with_hook::h6d5921a1a1fa5a8e
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panicking.rs:842:13
[INFO] [stdout]   15:     0x56b582dce20a - std::panicking::panic_handler::{{closure}}::hafeef6f707d43542
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panicking.rs:707:13
[INFO] [stdout]   16:     0x56b582dc8329 - std::sys::backtrace::__rust_end_short_backtrace::h08cbc3319a3d0120
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/sys/backtrace.rs:174:18
[INFO] [stdout]   17:     0x56b582daef3d - __rustc[ce5c411ff86ab1b2]::rust_begin_unwind
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panicking.rs:698:5
[INFO] [stdout]   18:     0x56b582e0f630 - core::panicking::panic_fmt::h774fb860369a0f7b
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/panicking.rs:80:14
[INFO] [stdout]   19:     0x56b582e0f3e3 - core::panicking::assert_failed_inner::hbdde3a71af798a31
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/panicking.rs:444:17
[INFO] [stdout]   20:     0x56b581ac17d2 - core::panicking::assert_failed::ha24034a1566b3333
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/panicking.rs:399:5
[INFO] [stdout]   21:     0x56b581a69ba9 - ai_workbench_lib::modules::file_splitter::splitter_types::json_splitter::tests::test_standard_json_detection::hb110d6fdf5ab9bbc
[INFO] [stdout]                                at /opt/rustwide/workdir/src/../modules/file_splitter/splitter_types/json_splitter.rs:354:9
[INFO] [stdout]   22:     0x56b581a69bd7 - ai_workbench_lib::modules::file_splitter::splitter_types::json_splitter::tests::test_standard_json_detection::{{closure}}::hf0132aaacd6f39f2
[INFO] [stdout]                                at /opt/rustwide/workdir/src/../modules/file_splitter/splitter_types/json_splitter.rs:348:38
[INFO] [stdout]   23:     0x56b581a95fa6 - core::ops::function::FnOnce::call_once::h590cdf75a98f2f24
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/ops/function.rs:250:5
[INFO] [stdout]   24:     0x56b581b5a5cb - core::ops::function::FnOnce::call_once::h4f6eabe90cccb47d
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/ops/function.rs:250:5
[INFO] [stdout]   25:     0x56b581b5a5cb - test::__rust_begin_short_backtrace::h94208530f2b4c8c7
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/test/src/lib.rs:663:18
[INFO] [stdout]   26:     0x56b581b6e5dd - test::run_test_in_process::{{closure}}::hbdc6a45a78c46404
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/test/src/lib.rs:686:74
[INFO] [stdout]   27:     0x56b581b6e5dd - <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once::h7f9d8efc7eedd165
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/panic/unwind_safe.rs:274:9
[INFO] [stdout]   28:     0x56b581b6e5dd - std::panicking::catch_unwind::do_call::he5edbaf126c13b99
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panicking.rs:590:40
[INFO] [stdout]   29:     0x56b581b6e5dd - std::panicking::catch_unwind::h3d6a8652eceeabce
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panicking.rs:553:19
[INFO] [stdout]   30:     0x56b581b6e5dd - std::panic::catch_unwind::hbef0c501dd7bb498
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panic.rs:359:14
[INFO] [stdout]   31:     0x56b581b6e5dd - test::run_test_in_process::h400008a46a5006a0
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/test/src/lib.rs:686:27
[INFO] [stdout]   32:     0x56b581b6e5dd - test::run_test::{{closure}}::h9a131655b31d9427
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/test/src/lib.rs:607:43
[INFO] [stdout]   33:     0x56b581b47b24 - test::run_test::{{closure}}::he4a7177c216308c0
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/test/src/lib.rs:637:41
[INFO] [stdout]   34:     0x56b581b47b24 - std::sys::backtrace::__rust_begin_short_backtrace::hd3980e9c55d6c539
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/sys/backtrace.rs:158:18
[INFO] [stdout]   35:     0x56b581b4b45a - std::thread::Builder::spawn_unchecked_::{{closure}}::{{closure}}::h18552df67c7aa9b8
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/thread/mod.rs:562:17
[INFO] [stdout]   36:     0x56b581b4b45a - <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once::h3d7b285234641a08
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/panic/unwind_safe.rs:274:9
[INFO] [stdout]   37:     0x56b581b4b45a - std::panicking::catch_unwind::do_call::h5a2b5cf36528a5b2
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panicking.rs:590:40
[INFO] [stdout]   38:     0x56b581b4b45a - std::panicking::catch_unwind::h7e8b531bbda77d20
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panicking.rs:553:19
[INFO] [stdout]   39:     0x56b581b4b45a - std::panic::catch_unwind::hf9bbb2040b4f1e6c
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panic.rs:359:14
[INFO] [stdout]   40:     0x56b581b4b45a - std::thread::Builder::spawn_unchecked_::{{closure}}::h600f92cf68549574
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/thread/mod.rs:560:30
[INFO] [stdout]   41:     0x56b581b4b45a - core::ops::function::FnOnce::call_once{{vtable.shim}}::hb6607307df8a1847
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/ops/function.rs:250:5
[INFO] [stdout]   42:     0x56b582dc364f - <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once::h6891ad53a5ed6f52
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/alloc/src/boxed.rs:2085:9
[INFO] [stdout]   43:     0x56b582dc364f - std::sys::thread::unix::Thread::new::thread_start::h7f3e77fa86df70af
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/sys/thread/unix.rs:124:17
[INFO] [stdout]   44:     0x70def9591aa4 - <unknown>
[INFO] [stdout]   45:     0x70def961ea64 - clone
[INFO] [stdout]   46:                0x0 - <unknown>
[INFO] [stdout] 
[INFO] [stdout] ---- modules::file_splitter::splitter_types::json_splitter::tests::test_invalid_json_rejection stdout ----
[INFO] [stdout] 
[INFO] [stdout] thread 'modules::file_splitter::splitter_types::json_splitter::tests::test_invalid_json_rejection' (48) panicked at src/../modules/file_splitter/splitter_types/json_splitter.rs:400:9:
[INFO] [stdout] assertion `left == right` failed
[INFO] [stdout]   left: JsonLines
[INFO] [stdout]  right: Invalid
[INFO] [stdout] stack backtrace:
[INFO] [stdout]    0:     0x56b582df1172 - std::backtrace_rs::backtrace::libunwind::trace::h9ea1e07ac77a25d7
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/../../backtrace/src/backtrace/libunwind.rs:117:9
[INFO] [stdout]    1:     0x56b582df1172 - std::backtrace_rs::backtrace::trace_unsynchronized::h72b75eb83b53d15a
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/../../backtrace/src/backtrace/mod.rs:66:14
[INFO] [stdout]    2:     0x56b582df1172 - std::sys::backtrace::_print_fmt::h255777b7ec902439
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/sys/backtrace.rs:66:9
[INFO] [stdout]    3:     0x56b582df1172 - <std::sys::backtrace::BacktraceLock::print::DisplayBacktrace as core::fmt::Display>::fmt::h8a1ac49fcd7ee8ce
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/sys/backtrace.rs:39:26
[INFO] [stdout]    4:     0x56b582e047cf - core::fmt::rt::Argument::fmt::h635cbf3c6754cc90
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/fmt/rt.rs:173:76
[INFO] [stdout]    5:     0x56b582e047cf - core::fmt::write::hbdcc0ec4f1bab1db
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/fmt/mod.rs:1469:25
[INFO] [stdout]    6:     0x56b582dbb323 - std::io::default_write_fmt::h2580ece0d4b58e51
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/io/mod.rs:639:11
[INFO] [stdout]    7:     0x56b582dbb323 - std::io::Write::write_fmt::hac9e42858f273b87
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/io/mod.rs:1954:13
[INFO] [stdout]    8:     0x56b582dc81f2 - std::sys::backtrace::BacktraceLock::print::h052febbd86f7bf07
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/sys/backtrace.rs:42:9
[INFO] [stdout]    9:     0x56b582dcdd9f - std::panicking::default_hook::{{closure}}::h66aeee20dd470abb
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panicking.rs:301:27
[INFO] [stdout]   10:     0x56b582dcdc31 - std::panicking::default_hook::h9d66b9642014ce48
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panicking.rs:325:9
[INFO] [stdout]   11:     0x56b581b5a77e - <alloc::boxed::Box<F,A> as core::ops::function::Fn<Args>>::call::h8c698701e444e564
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/alloc/src/boxed.rs:2099:9
[INFO] [stdout]   12:     0x56b581b5a77e - test::test_main_with_exit_callback::{{closure}}::ha1c98e7b69c4defb
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/test/src/lib.rs:145:21
[INFO] [stdout]   13:     0x56b582dce45f - <alloc::boxed::Box<F,A> as core::ops::function::Fn<Args>>::call::hc4f222a8a57c637a
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/alloc/src/boxed.rs:2099:9
[INFO] [stdout]   14:     0x56b582dce45f - std::panicking::panic_with_hook::h6d5921a1a1fa5a8e
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panicking.rs:842:13
[INFO] [stdout]   15:     0x56b582dce20a - std::panicking::panic_handler::{{closure}}::hafeef6f707d43542
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panicking.rs:707:13
[INFO] [stdout]   16:     0x56b582dc8329 - std::sys::backtrace::__rust_end_short_backtrace::h08cbc3319a3d0120
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/sys/backtrace.rs:174:18
[INFO] [stdout]   17:     0x56b582daef3d - __rustc[ce5c411ff86ab1b2]::rust_begin_unwind
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panicking.rs:698:5
[INFO] [stdout]   18:     0x56b582e0f630 - core::panicking::panic_fmt::h774fb860369a0f7b
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/panicking.rs:80:14
[INFO] [stdout]   19:     0x56b582e0f3e3 - core::panicking::assert_failed_inner::hbdde3a71af798a31
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/panicking.rs:444:17
[INFO] [stdout]   20:     0x56b581ac17d2 - core::panicking::assert_failed::ha24034a1566b3333
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/panicking.rs:399:5
[INFO] [stdout]   21:     0x56b581a69a29 - ai_workbench_lib::modules::file_splitter::splitter_types::json_splitter::tests::test_invalid_json_rejection::h47c51a2ca9cac10e
[INFO] [stdout]                                at /opt/rustwide/workdir/src/../modules/file_splitter/splitter_types/json_splitter.rs:400:9
[INFO] [stdout]   22:     0x56b581a69a57 - ai_workbench_lib::modules::file_splitter::splitter_types::json_splitter::tests::test_invalid_json_rejection::{{closure}}::h856f04f18f9f65b6
[INFO] [stdout]                                at /opt/rustwide/workdir/src/../modules/file_splitter/splitter_types/json_splitter.rs:395:37
[INFO] [stdout]   23:     0x56b581a96376 - core::ops::function::FnOnce::call_once::h879f36ddbed5df18
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/ops/function.rs:250:5
[INFO] [stdout]   24:     0x56b581b5a5cb - core::ops::function::FnOnce::call_once::h4f6eabe90cccb47d
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/ops/function.rs:250:5
[INFO] [stdout]   25:     0x56b581b5a5cb - test::__rust_begin_short_backtrace::h94208530f2b4c8c7
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/test/src/lib.rs:663:18
[INFO] [stdout]   26:     0x56b581b6e5dd - test::run_test_in_process::{{closure}}::hbdc6a45a78c46404
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/test/src/lib.rs:686:74
[INFO] [stdout]   27:     0x56b581b6e5dd - <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once::h7f9d8efc7eedd165
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/panic/unwind_safe.rs:274:9
[INFO] [stdout]   28:     0x56b581b6e5dd - std::panicking::catch_unwind::do_call::he5edbaf126c13b99
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panicking.rs:590:40
[INFO] [stdout]   29:     0x56b581b6e5dd - std::panicking::catch_unwind::h3d6a8652eceeabce
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panicking.rs:553:19
[INFO] [stdout]   30:     0x56b581b6e5dd - std::panic::catch_unwind::hbef0c501dd7bb498
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panic.rs:359:14
[INFO] [stdout]   31:     0x56b581b6e5dd - test::run_test_in_process::h400008a46a5006a0
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/test/src/lib.rs:686:27
[INFO] [stdout]   32:     0x56b581b6e5dd - test::run_test::{{closure}}::h9a131655b31d9427
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/test/src/lib.rs:607:43
[INFO] [stdout]   33:     0x56b581b47b24 - test::run_test::{{closure}}::he4a7177c216308c0
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/test/src/lib.rs:637:41
[INFO] [stdout]   34:     0x56b581b47b24 - std::sys::backtrace::__rust_begin_short_backtrace::hd3980e9c55d6c539
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/sys/backtrace.rs:158:18
[INFO] [stdout]   35:     0x56b581b4b45a - std::thread::Builder::spawn_unchecked_::{{closure}}::{{closure}}::h18552df67c7aa9b8
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/thread/mod.rs:562:17
[INFO] [stdout]   36:     0x56b581b4b45a - <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once::h3d7b285234641a08
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/panic/unwind_safe.rs:274:9
[INFO] [stdout]   37:     0x56b581b4b45a - std::panicking::catch_unwind::do_call::h5a2b5cf36528a5b2
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panicking.rs:590:40
[INFO] [stdout]   38:     0x56b581b4b45a - std::panicking::catch_unwind::h7e8b531bbda77d20
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panicking.rs:553:19
[INFO] [stdout]   39:     0x56b581b4b45a - std::panic::catch_unwind::hf9bbb2040b4f1e6c
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panic.rs:359:14
[INFO] [stdout]   40:     0x56b581b4b45a - std::thread::Builder::spawn_unchecked_::{{closure}}::h600f92cf68549574
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/thread/mod.rs:560:30
[INFO] [stdout]   41:     0x56b581b4b45a - core::ops::function::FnOnce::call_once{{vtable.shim}}::hb6607307df8a1847
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/ops/function.rs:250:5
[INFO] [stdout]   42:     0x56b582dc364f - <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once::h6891ad53a5ed6f52
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/alloc/src/boxed.rs:2085:9
[INFO] [stdout]   43:     0x56b582dc364f - std::sys::thread::unix::Thread::new::thread_start::h7f3e77fa86df70af
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/sys/thread/unix.rs:124:17
[INFO] [stdout]   44:     0x70def9591aa4 - <unknown>
[INFO] [stdout]   45:     0x70def961ea64 - clone
[INFO] [stdout]   46:                0x0 - <unknown>
[INFO] [stdout] 
[INFO] [stdout] ---- modules::file_splitter::splitter::integration_tests::test_edge_cases stdout ----
[INFO] [stdout] 
[INFO] [stdout] thread 'modules::file_splitter::splitter::integration_tests::test_edge_cases' (31) panicked at src/../modules/file_splitter/splitter.rs:263:9:
[INFO] [stdout] assertion failed: chunks[0].metadata.has_headers
[INFO] [stdout] stack backtrace:
[INFO] [stdout]    0:     0x56b582df1172 - std::backtrace_rs::backtrace::libunwind::trace::h9ea1e07ac77a25d7
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/../../backtrace/src/backtrace/libunwind.rs:117:9
[INFO] [stdout]    1:     0x56b582df1172 - std::backtrace_rs::backtrace::trace_unsynchronized::h72b75eb83b53d15a
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/../../backtrace/src/backtrace/mod.rs:66:14
[INFO] [stdout]    2:     0x56b582df1172 - std::sys::backtrace::_print_fmt::h255777b7ec902439
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/sys/backtrace.rs:66:9
[INFO] [stdout]    3:     0x56b582df1172 - <std::sys::backtrace::BacktraceLock::print::DisplayBacktrace as core::fmt::Display>::fmt::h8a1ac49fcd7ee8ce
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/sys/backtrace.rs:39:26
[INFO] [stdout]    4:     0x56b582e047cf - core::fmt::rt::Argument::fmt::h635cbf3c6754cc90
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/fmt/rt.rs:173:76
[INFO] [stdout]    5:     0x56b582e047cf - core::fmt::write::hbdcc0ec4f1bab1db
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/fmt/mod.rs:1469:25
[INFO] [stdout]    6:     0x56b582dbb323 - std::io::default_write_fmt::h2580ece0d4b58e51
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/io/mod.rs:639:11
[INFO] [stdout]    7:     0x56b582dbb323 - std::io::Write::write_fmt::hac9e42858f273b87
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/io/mod.rs:1954:13
[INFO] [stdout]    8:     0x56b582dc81f2 - std::sys::backtrace::BacktraceLock::print::h052febbd86f7bf07
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/sys/backtrace.rs:42:9
[INFO] [stdout]    9:     0x56b582dcdd9f - std::panicking::default_hook::{{closure}}::h66aeee20dd470abb
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panicking.rs:301:27
[INFO] [stdout]   10:     0x56b582dcdc31 - std::panicking::default_hook::h9d66b9642014ce48
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panicking.rs:325:9
[INFO] [stdout]   11:     0x56b581b5a77e - <alloc::boxed::Box<F,A> as core::ops::function::Fn<Args>>::call::h8c698701e444e564
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/alloc/src/boxed.rs:2099:9
[INFO] [stdout]   12:     0x56b581b5a77e - test::test_main_with_exit_callback::{{closure}}::ha1c98e7b69c4defb
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/test/src/lib.rs:145:21
[INFO] [stdout]   13:     0x56b582dce45f - <alloc::boxed::Box<F,A> as core::ops::function::Fn<Args>>::call::hc4f222a8a57c637a
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/alloc/src/boxed.rs:2099:9
[INFO] [stdout]   14:     0x56b582dce45f - std::panicking::panic_with_hook::h6d5921a1a1fa5a8e
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panicking.rs:842:13
[INFO] [stdout]   15:     0x56b582dce1d6 - std::panicking::panic_handler::{{closure}}::hafeef6f707d43542
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panicking.rs:700:13
[INFO] [stdout]   16:     0x56b582dc8329 - std::sys::backtrace::__rust_end_short_backtrace::h08cbc3319a3d0120
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/sys/backtrace.rs:174:18
[INFO] [stdout]   17:     0x56b582daef3d - __rustc[ce5c411ff86ab1b2]::rust_begin_unwind
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panicking.rs:698:5
[INFO] [stdout]   18:     0x56b582e0f630 - core::panicking::panic_fmt::h774fb860369a0f7b
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/panicking.rs:80:14
[INFO] [stdout]   19:     0x56b582e0f60c - core::panicking::panic::h2a6fe4ac012a7582
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/panicking.rs:150:5
[INFO] [stdout]   20:     0x56b581a6aff2 - ai_workbench_lib::modules::file_splitter::splitter::integration_tests::test_edge_cases::h3fcf007f6fcd2372
[INFO] [stdout]                                at /opt/rustwide/workdir/src/../modules/file_splitter/splitter.rs:263:9
[INFO] [stdout]   21:     0x56b581a6b2f7 - ai_workbench_lib::modules::file_splitter::splitter::integration_tests::test_edge_cases::{{closure}}::h3ffb540227baf696
[INFO] [stdout]                                at /opt/rustwide/workdir/src/../modules/file_splitter/splitter.rs:251:25
[INFO] [stdout]   22:     0x56b581a95d16 - core::ops::function::FnOnce::call_once::h37d2cfe369b810a8
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/ops/function.rs:250:5
[INFO] [stdout]   23:     0x56b581b5a5cb - core::ops::function::FnOnce::call_once::h4f6eabe90cccb47d
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/ops/function.rs:250:5
[INFO] [stdout]   24:     0x56b581b5a5cb - test::__rust_begin_short_backtrace::h94208530f2b4c8c7
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/test/src/lib.rs:663:18
[INFO] [stdout]   25:     0x56b581b6e5dd - test::run_test_in_process::{{closure}}::hbdc6a45a78c46404
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/test/src/lib.rs:686:74
[INFO] [stdout]   26:     0x56b581b6e5dd - <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once::h7f9d8efc7eedd165
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/panic/unwind_safe.rs:274:9
[INFO] [stdout]   27:     0x56b581b6e5dd - std::panicking::catch_unwind::do_call::he5edbaf126c13b99
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panicking.rs:590:40
[INFO] [stdout]   28:     0x56b581b6e5dd - std::panicking::catch_unwind::h3d6a8652eceeabce
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panicking.rs:553:19
[INFO] [stdout]   29:     0x56b581b6e5dd - std::panic::catch_unwind::hbef0c501dd7bb498
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panic.rs:359:14
[INFO] [stdout]   30:     0x56b581b6e5dd - test::run_test_in_process::h400008a46a5006a0
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/test/src/lib.rs:686:27
[INFO] [stdout]   31:     0x56b581b6e5dd - test::run_test::{{closure}}::h9a131655b31d9427
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/test/src/lib.rs:607:43
[INFO] [stdout]   32:     0x56b581b47b24 - test::run_test::{{closure}}::he4a7177c216308c0
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/test/src/lib.rs:637:41
[INFO] [stdout]   33:     0x56b581b47b24 - std::sys::backtrace::__rust_begin_short_backtrace::hd3980e9c55d6c539
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/sys/backtrace.rs:158:18
[INFO] [stdout]   34:     0x56b581b4b45a - std::thread::Builder::spawn_unchecked_::{{closure}}::{{closure}}::h18552df67c7aa9b8
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/thread/mod.rs:562:17
[INFO] [stdout]   35:     0x56b581b4b45a - <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once::h3d7b285234641a08
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/panic/unwind_safe.rs:274:9
[INFO] [stdout]   36:     0x56b581b4b45a - std::panicking::catch_unwind::do_call::h5a2b5cf36528a5b2
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panicking.rs:590:40
[INFO] [stdout]   37:     0x56b581b4b45a - std::panicking::catch_unwind::h7e8b531bbda77d20
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panicking.rs:553:19
[INFO] [stdout]   38:     0x56b581b4b45a - std::panic::catch_unwind::hf9bbb2040b4f1e6c
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panic.rs:359:14
[INFO] [stdout]   39:     0x56b581b4b45a - std::thread::Builder::spawn_unchecked_::{{closure}}::h600f92cf68549574
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/thread/mod.rs:560:30
[INFO] [stdout]   40:     0x56b581b4b45a - core::ops::function::FnOnce::call_once{{vtable.shim}}::hb6607307df8a1847
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/ops/function.rs:250:5
[INFO] [stdout]   41:     0x56b582dc364f - <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once::h6891ad53a5ed6f52
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/alloc/src/boxed.rs:2085:9
[INFO] [stdout]   42:     0x56b582dc364f - std::sys::thread::unix::Thread::new::thread_start::h7f3e77fa86df70af
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/sys/thread/unix.rs:124:17
[INFO] [stdout]   43:     0x70def9591aa4 - <unknown>
[INFO] [stdout]   44:     0x70def961ea64 - clone
[INFO] [stdout]   45:                0x0 - <unknown>
[INFO] [stdout] 
[INFO] [stdout] ---- modules::file_splitter::splitter::integration_tests::test_performance_characteristics stdout ----
[INFO] [stdout] 
[INFO] [stdout] thread 'modules::file_splitter::splitter::integration_tests::test_performance_characteristics' (35) panicked at src/../modules/file_splitter/splitter.rs:288:9:
[INFO] [stdout] assertion failed: chunks.len() > 1
[INFO] [stdout] stack backtrace:
[INFO] [stdout]    0:     0x56b582df1172 - std::backtrace_rs::backtrace::libunwind::trace::h9ea1e07ac77a25d7
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/../../backtrace/src/backtrace/libunwind.rs:117:9
[INFO] [stdout]    1:     0x56b582df1172 - std::backtrace_rs::backtrace::trace_unsynchronized::h72b75eb83b53d15a
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/../../backtrace/src/backtrace/mod.rs:66:14
[INFO] [stdout]    2:     0x56b582df1172 - std::sys::backtrace::_print_fmt::h255777b7ec902439
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/sys/backtrace.rs:66:9
[INFO] [stdout]    3:     0x56b582df1172 - <std::sys::backtrace::BacktraceLock::print::DisplayBacktrace as core::fmt::Display>::fmt::h8a1ac49fcd7ee8ce
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/sys/backtrace.rs:39:26
[INFO] [stdout]    4:     0x56b582e047cf - core::fmt::rt::Argument::fmt::h635cbf3c6754cc90
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/fmt/rt.rs:173:76
[INFO] [stdout]    5:     0x56b582e047cf - core::fmt::write::hbdcc0ec4f1bab1db
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/fmt/mod.rs:1469:25
[INFO] [stdout]    6:     0x56b582dbb323 - std::io::default_write_fmt::h2580ece0d4b58e51
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/io/mod.rs:639:11
[INFO] [stdout]    7:     0x56b582dbb323 - std::io::Write::write_fmt::hac9e42858f273b87
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/io/mod.rs:1954:13
[INFO] [stdout]    8:     0x56b582dc81f2 - std::sys::backtrace::BacktraceLock::print::h052febbd86f7bf07
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/sys/backtrace.rs:42:9
[INFO] [stdout]    9:     0x56b582dcdd9f - std::panicking::default_hook::{{closure}}::h66aeee20dd470abb
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panicking.rs:301:27
[INFO] [stdout]   10:     0x56b582dcdc31 - std::panicking::default_hook::h9d66b9642014ce48
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panicking.rs:325:9
[INFO] [stdout]   11:     0x56b581b5a77e - <alloc::boxed::Box<F,A> as core::ops::function::Fn<Args>>::call::h8c698701e444e564
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/alloc/src/boxed.rs:2099:9
[INFO] [stdout]   12:     0x56b581b5a77e - test::test_main_with_exit_callback::{{closure}}::ha1c98e7b69c4defb
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/test/src/lib.rs:145:21
[INFO] [stdout]   13:     0x56b582dce45f - <alloc::boxed::Box<F,A> as core::ops::function::Fn<Args>>::call::hc4f222a8a57c637a
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/alloc/src/boxed.rs:2099:9
[INFO] [stdout]   14:     0x56b582dce45f - std::panicking::panic_with_hook::h6d5921a1a1fa5a8e
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panicking.rs:842:13
[INFO] [stdout]   15:     0x56b582dce1d6 - std::panicking::panic_handler::{{closure}}::hafeef6f707d43542
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panicking.rs:700:13
[INFO] [stdout]   16:     0x56b582dc8329 - std::sys::backtrace::__rust_end_short_backtrace::h08cbc3319a3d0120
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/sys/backtrace.rs:174:18
[INFO] [stdout]   17:     0x56b582daef3d - __rustc[ce5c411ff86ab1b2]::rust_begin_unwind
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panicking.rs:698:5
[INFO] [stdout]   18:     0x56b582e0f630 - core::panicking::panic_fmt::h774fb860369a0f7b
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/panicking.rs:80:14
[INFO] [stdout]   19:     0x56b582e0f60c - core::panicking::panic::h2a6fe4ac012a7582
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/panicking.rs:150:5
[INFO] [stdout]   20:     0x56b581a6d1be - ai_workbench_lib::modules::file_splitter::splitter::integration_tests::test_performance_characteristics::h9f1ad1f8dc26381d
[INFO] [stdout]                                at /opt/rustwide/workdir/src/../modules/file_splitter/splitter.rs:288:9
[INFO] [stdout]   21:     0x56b581a6d3f7 - ai_workbench_lib::modules::file_splitter::splitter::integration_tests::test_performance_characteristics::{{closure}}::h67c1aad7c32583c3
[INFO] [stdout]                                at /opt/rustwide/workdir/src/../modules/file_splitter/splitter.rs:273:42
[INFO] [stdout]   22:     0x56b581a964f6 - core::ops::function::FnOnce::call_once::h9cc3124650197451
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/ops/function.rs:250:5
[INFO] [stdout]   23:     0x56b581b5a5cb - core::ops::function::FnOnce::call_once::h4f6eabe90cccb47d
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/ops/function.rs:250:5
[INFO] [stdout]   24:     0x56b581b5a5cb - test::__rust_begin_short_backtrace::h94208530f2b4c8c7
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/test/src/lib.rs:663:18
[INFO] [stdout]   25:     0x56b581b6e5dd - test::run_test_in_process::{{closure}}::hbdc6a45a78c46404
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/test/src/lib.rs:686:74
[INFO] [stdout]   26:     0x56b581b6e5dd - <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once::h7f9d8efc7eedd165
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/panic/unwind_safe.rs:274:9
[INFO] [stdout]   27:     0x56b581b6e5dd - std::panicking::catch_unwind::do_call::he5edbaf126c13b99
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panicking.rs:590:40
[INFO] [stdout]   28:     0x56b581b6e5dd - std::panicking::catch_unwind::h3d6a8652eceeabce
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panicking.rs:553:19
[INFO] [stdout]   29:     0x56b581b6e5dd - std::panic::catch_unwind::hbef0c501dd7bb498
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panic.rs:359:14
[INFO] [stdout]   30:     0x56b581b6e5dd - test::run_test_in_process::h400008a46a5006a0
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/test/src/lib.rs:686:27
[INFO] [stdout]   31:     0x56b581b6e5dd - test::run_test::{{closure}}::h9a131655b31d9427
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/test/src/lib.rs:607:43
[INFO] [stdout]   32:     0x56b581b47b24 - test::run_test::{{closure}}::he4a7177c216308c0
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/test/src/lib.rs:637:41
[INFO] [stdout]   33:     0x56b581b47b24 - std::sys::backtrace::__rust_begin_short_backtrace::hd3980e9c55d6c539
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/sys/backtrace.rs:158:18
[INFO] [stdout]   34:     0x56b581b4b45a - std::thread::Builder::spawn_unchecked_::{{closure}}::{{closure}}::h18552df67c7aa9b8
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/thread/mod.rs:562:17
[INFO] [stdout]   35:     0x56b581b4b45a - <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once::h3d7b285234641a08
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/panic/unwind_safe.rs:274:9
[INFO] [stdout]   36:     0x56b581b4b45a - std::panicking::catch_unwind::do_call::h5a2b5cf36528a5b2
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panicking.rs:590:40
[INFO] [stdout]   37:     0x56b581b4b45a - std::panicking::catch_unwind::h7e8b531bbda77d20
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panicking.rs:553:19
[INFO] [stdout]   38:     0x56b581b4b45a - std::panic::catch_unwind::hf9bbb2040b4f1e6c
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/panic.rs:359:14
[INFO] [stdout]   39:     0x56b581b4b45a - std::thread::Builder::spawn_unchecked_::{{closure}}::h600f92cf68549574
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/thread/mod.rs:560:30
[INFO] [stdout]   40:     0x56b581b4b45a - core::ops::function::FnOnce::call_once{{vtable.shim}}::hb6607307df8a1847
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/core/src/ops/function.rs:250:5
[INFO] [stdout]   41:     0x56b582dc364f - <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once::h6891ad53a5ed6f52
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/alloc/src/boxed.rs:2085:9
[INFO] [stdout]   42:     0x56b582dc364f - std::sys::thread::unix::Thread::new::thread_start::h7f3e77fa86df70af
[INFO] [stdout]                                at /rustc/c2e32f1c9652b13ed99608599c1e855462f421f3/library/std/src/sys/thread/unix.rs:124:17
[INFO] [stdout]   43:     0x70def9591aa4 - <unknown>
[INFO] [stdout]   44:     0x70def961ea64 - clone
[INFO] [stdout]   45:                0x0 - <unknown>
[INFO] [stdout] 
[INFO] [stdout] 
[INFO] [stdout] failures:
[INFO] [stdout]     modules::file_splitter::splitter::integration_tests::test_edge_cases
[INFO] [stdout]     modules::file_splitter::splitter::integration_tests::test_mixed_file_types
[INFO] [stdout]     modules::file_splitter::splitter::integration_tests::test_performance_characteristics
[INFO] [stdout]     modules::file_splitter::splitter::tests::test_binary_file_rejection
[INFO] [stdout]     modules::file_splitter::splitter_types::json_splitter::tests::test_invalid_json_rejection
[INFO] [stdout]     modules::file_splitter::splitter_types::json_splitter::tests::test_json_array_detection
[INFO] [stdout]     modules::file_splitter::splitter_types::json_splitter::tests::test_standard_json_detection
[INFO] [stdout] 
[INFO] [stdout] test result: FAILED. 30 passed; 7 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.50s
[INFO] [stdout] 
[INFO] [stderr] error: test failed, to rerun pass `--lib`
[INFO] running `Command { std: "docker" "inspect" "295690670bb3ced118ab22ee038d27aff5e5d03ba003da87bfa33a012aa34748", kill_on_drop: false }`
[INFO] running `Command { std: "docker" "rm" "-f" "295690670bb3ced118ab22ee038d27aff5e5d03ba003da87bfa33a012aa34748", kill_on_drop: false }`
[INFO] [stdout] 295690670bb3ced118ab22ee038d27aff5e5d03ba003da87bfa33a012aa34748
