[INFO] cloning repository https://github.com/heManKuMAR6/evalforge
[INFO] running `Command { std: "git" "-c" "credential.helper=" "-c" "credential.helper=/workspace/cargo-home/bin/git-credential-null" "clone" "--bare" "https://github.com/heManKuMAR6/evalforge" "/workspace/cache/git-repos/https%3A%2F%2Fgithub.com%2FheManKuMAR6%2Fevalforge", kill_on_drop: false }`
[INFO] [stderr] Cloning into bare repository '/workspace/cache/git-repos/https%3A%2F%2Fgithub.com%2FheManKuMAR6%2Fevalforge'...
[INFO] running `Command { std: "git" "rev-parse" "HEAD", kill_on_drop: false }`
[INFO] [stdout] 7931647894572756517c815bcdbd8758487605cc
[INFO] testing heManKuMAR6/evalforge against 1.99.0-beta.8+cargoflags=--release for beta-release-1.100-2
[INFO] running `Command { std: "git" "clone" "/workspace/cache/git-repos/https%3A%2F%2Fgithub.com%2FheManKuMAR6%2Fevalforge" "/workspace/builds/worker-4-tc1/source", kill_on_drop: false }`
[INFO] [stderr] Cloning into '/workspace/builds/worker-4-tc1/source'...
[INFO] [stderr] done.
[INFO] started tweaking git repo https://github.com/heManKuMAR6/evalforge
[INFO] finished tweaking git repo https://github.com/heManKuMAR6/evalforge
[INFO] tweaked toml for git repo https://github.com/heManKuMAR6/evalforge written to /workspace/builds/worker-4-tc1/source/Cargo.toml
[INFO] validating manifest of git repo https://github.com/heManKuMAR6/evalforge on toolchain 1.99.0-beta.8
[INFO] running `Command { std: CARGO_HOME="/workspace/cargo-home" RUSTUP_HOME="/workspace/rustup-home" "/workspace/cargo-home/bin/cargo" "+1.99.0-beta.8" "metadata" "--manifest-path" "Cargo.toml" "--no-deps", kill_on_drop: false }`
[INFO] crate git repo https://github.com/heManKuMAR6/evalforge already has a lockfile, it will not be regenerated
[INFO] running `Command { std: CARGO_HOME="/workspace/cargo-home" RUSTUP_HOME="/workspace/rustup-home" "/workspace/cargo-home/bin/cargo" "+1.99.0-beta.8" "fetch" "--manifest-path" "Cargo.toml", kill_on_drop: false }`
[INFO] [stderr]  Downloading crates ...
[INFO] [stderr]   Downloaded wasm-bindgen-futures v0.4.67
[INFO] [stderr]   Downloaded js-sys v0.3.94
[INFO] [stderr]   Downloaded web-sys v0.3.94
[INFO] [stderr]   Downloaded openssl-sys v0.9.112
[INFO] [stderr]   Downloaded openssl v0.10.76
[INFO] [stderr]   Downloaded clap_derive v4.6.0
[INFO] [stderr]   Downloaded clap v4.6.0
[INFO] running `Command { std: "docker" "create" "-v" "/var/lib/crater-agent-workspace/builds/worker-4-tc1/source:/opt/rustwide/workdir:ro,Z" "-v" "/var/lib/crater-agent-workspace/builds/worker-4-tc1/target:/opt/rustwide/target:rw,Z" "-v" "/var/lib/crater-agent-workspace/cargo-home:/opt/rustwide/cargo-home:ro,Z" "-v" "/var/lib/crater-agent-workspace/rustup-home:/opt/rustwide/rustup-home:ro,Z" "-m" "1610612736" "--network" "none" "ghcr.io/rust-lang/crates-build-env/linux@sha256:77db811e55d90add9212f6832c23229e3de5f1c1f9905e8cb07319ea15df73ac" "sleep" "infinity", kill_on_drop: false }`
[INFO] [stdout] 8481537f0a4ff8394122190d39881f082239a57f66a202cc8a66462ea837b91b
[INFO] running `Command { std: "docker" "start" "8481537f0a4ff8394122190d39881f082239a57f66a202cc8a66462ea837b91b", kill_on_drop: false }`
[INFO] running `Command { std: "docker" "inspect" "8481537f0a4ff8394122190d39881f082239a57f66a202cc8a66462ea837b91b", kill_on_drop: false }`
[INFO] running `Command { std: "docker" "exec" "-e" "SOURCE_DIR=/opt/rustwide/workdir" "-e" "CARGO_HOME=/opt/rustwide/cargo-home" "-e" "RUSTUP_HOME=/opt/rustwide/rustup-home" "-e" "CARGO_TARGET_DIR=/opt/rustwide/target" "-w" "/opt/rustwide/workdir" "--user" "0:0" "8481537f0a4ff8394122190d39881f082239a57f66a202cc8a66462ea837b91b" "/opt/rustwide/cargo-home/bin/cargo" "+1.99.0-beta.8" "metadata" "--no-deps" "--format-version=1", kill_on_drop: false }`
[INFO] running `Command { std: "docker" "inspect" "8481537f0a4ff8394122190d39881f082239a57f66a202cc8a66462ea837b91b", kill_on_drop: false }`
[INFO] running `Command { std: "docker" "exec" "-e" "SOURCE_DIR=/opt/rustwide/workdir" "-e" "CARGO_HOME=/opt/rustwide/cargo-home" "-e" "RUSTUP_HOME=/opt/rustwide/rustup-home" "-e" "CARGO_TARGET_DIR=/opt/rustwide/target" "-e" "CARGO_INCREMENTAL=0" "-e" "RUST_BACKTRACE=full" "-e" "RUSTFLAGS=--cap-lints=warn" "-e" "RUSTDOCFLAGS=--cap-lints=warn" "-w" "/opt/rustwide/workdir" "--user" "0:0" "8481537f0a4ff8394122190d39881f082239a57f66a202cc8a66462ea837b91b" "/opt/rustwide/cargo-home/bin/cargo" "+1.99.0-beta.8" "build" "--frozen" "--message-format=json" "--release", kill_on_drop: false }`
[INFO] [stderr]    Compiling proc-macro2 v1.0.106
[INFO] [stderr]    Compiling quote v1.0.45
[INFO] [stderr]    Compiling libc v0.2.184
[INFO] [stderr]    Compiling smallvec v1.15.1
[INFO] [stderr]    Compiling bytes v1.11.1
[INFO] [stderr]    Compiling parking_lot_core v0.9.12
[INFO] [stderr]    Compiling futures-core v0.3.32
[INFO] [stderr]    Compiling find-msvc-tools v0.1.9
[INFO] [stderr]    Compiling lock_api v0.4.14
[INFO] [stderr]    Compiling futures-sink v0.3.32
[INFO] [stderr]    Compiling slab v0.4.12
[INFO] [stderr]    Compiling writeable v0.6.2
[INFO] [stderr]    Compiling serde_core v1.0.228
[INFO] [stderr]    Compiling memchr v2.8.0
[INFO] [stderr]    Compiling openssl v0.10.76
[INFO] [stderr]    Compiling bitflags v2.11.0
[INFO] [stderr]    Compiling hashbrown v0.16.1
[INFO] [stderr]    Compiling futures-io v0.3.32
[INFO] [stderr]    Compiling cc v1.2.58
[INFO] [stderr]    Compiling futures-task v0.3.32
[INFO] [stderr]    Compiling tracing v0.1.44
[INFO] [stderr]    Compiling native-tls v0.2.18
[INFO] [stderr]    Compiling serde v1.0.228
[INFO] [stderr]    Compiling futures-channel v0.3.32
[INFO] [stderr]    Compiling zmij v1.0.21
[INFO] [stderr]    Compiling autocfg v1.5.0
[INFO] [stderr]    Compiling openssl-probe v0.2.1
[INFO] [stderr]    Compiling log v0.4.29
[INFO] [stderr]    Compiling sync_wrapper v1.0.2
[INFO] [stderr]    Compiling ipnet v2.12.0
[INFO] [stderr]    Compiling serde_json v1.0.149
[INFO] [stderr]    Compiling anstyle-parse v1.0.0
[INFO] [stderr]    Compiling futures-util v0.3.32
[INFO] [stderr]    Compiling colorchoice v1.0.5
[INFO] [stderr]    Compiling anstyle-query v1.1.5
[INFO] [stderr]    Compiling zeroize v1.8.2
[INFO] [stderr]    Compiling anstyle v1.0.14
[INFO] [stderr]    Compiling iri-string v0.7.12
[INFO] [stderr]    Compiling is_terminal_polyfill v1.70.2
[INFO] [stderr]    Compiling num-traits v0.2.19
[INFO] [stderr]    Compiling thiserror v1.0.69
[INFO] [stderr]    Compiling rustls-pki-types v1.14.0
[INFO] [stderr]    Compiling syn v2.0.117
[INFO] [stderr]    Compiling http v1.4.0
[INFO] [stderr]    Compiling anstream v1.0.0
[INFO] [stderr]    Compiling encoding_rs v0.8.35
[INFO] [stderr]    Compiling clap_lex v1.1.0
[INFO] [stderr]    Compiling clap_builder v4.6.0
[INFO] [stderr]    Compiling indexmap v2.13.0
[INFO] [stderr]    Compiling errno v0.3.14
[INFO] [stderr]    Compiling socket2 v0.6.3
[INFO] [stderr]    Compiling signal-hook-registry v1.4.8
[INFO] [stderr]    Compiling mio v1.2.0
[INFO] [stderr]    Compiling parking_lot v0.12.5
[INFO] [stderr]    Compiling openssl-sys v0.9.112
[INFO] [stderr]    Compiling http-body v1.0.1
[INFO] [stderr]    Compiling http-body-util v0.1.3
[INFO] [stderr]    Compiling synstructure v0.13.2
[INFO] [stderr]    Compiling zerofrom-derive v0.1.7
[INFO] [stderr]    Compiling yoke-derive v0.8.2
[INFO] [stderr]    Compiling zerovec-derive v0.11.3
[INFO] [stderr]    Compiling displaydoc v0.2.5
[INFO] [stderr]    Compiling tokio-macros v2.6.1
[INFO] [stderr]    Compiling openssl-macros v0.1.1
[INFO] [stderr]    Compiling serde_derive v1.0.228
[INFO] [stderr]    Compiling thiserror-impl v1.0.69
[INFO] [stderr]    Compiling clap_derive v4.6.0
[INFO] [stderr]    Compiling tokio v1.50.0
[INFO] [stderr]    Compiling zerofrom v0.1.7
[INFO] [stderr]    Compiling yoke v0.8.2
[INFO] [stderr]    Compiling zerovec v0.11.6
[INFO] [stderr]    Compiling zerotrie v0.2.4
[INFO] [stderr]    Compiling tinystr v0.8.3
[INFO] [stderr]    Compiling potential_utf v0.1.5
[INFO] [stderr]    Compiling icu_collections v2.2.0
[INFO] [stderr]    Compiling icu_locale_core v2.2.0
[INFO] [stderr]    Compiling clap v4.6.0
[INFO] [stderr]    Compiling icu_provider v2.2.0
[INFO] [stderr]    Compiling icu_normalizer v2.2.0
[INFO] [stderr]    Compiling icu_properties v2.2.0
[INFO] [stderr]    Compiling serde_urlencoded v0.7.1
[INFO] [stderr]    Compiling chrono v0.4.44
[INFO] [stderr]    Compiling idna_adapter v1.2.1
[INFO] [stderr]    Compiling idna v1.1.0
[INFO] [stderr]    Compiling url v2.5.8
[INFO] [stderr]    Compiling tokio-util v0.7.18
[INFO] [stderr]    Compiling tower v0.5.3
[INFO] [stderr]    Compiling tokio-native-tls v0.3.1
[INFO] [stderr]    Compiling tower-http v0.6.8
[INFO] [stderr]    Compiling h2 v0.4.13
[INFO] [stderr]    Compiling hyper v1.9.0
[INFO] [stderr]    Compiling hyper-util v0.1.20
[INFO] [stderr]    Compiling hyper-tls v0.6.0
[INFO] [stderr]    Compiling reqwest v0.12.28
[INFO] [stderr]    Compiling evalforge-core v1.0.0 (/opt/rustwide/workdir/crates/evalforge-core)
[INFO] [stderr]    Compiling evalforge-cli v1.0.0 (/opt/rustwide/workdir/crates/evalforge-cli)
[INFO] [stderr]     Finished `release` profile [optimized] target(s) in 1m 28s
[INFO] running `Command { std: "docker" "inspect" "8481537f0a4ff8394122190d39881f082239a57f66a202cc8a66462ea837b91b", kill_on_drop: false }`
[INFO] running `Command { std: "docker" "exec" "-e" "SOURCE_DIR=/opt/rustwide/workdir" "-e" "CARGO_HOME=/opt/rustwide/cargo-home" "-e" "RUSTUP_HOME=/opt/rustwide/rustup-home" "-e" "CARGO_TARGET_DIR=/opt/rustwide/target" "-e" "CARGO_INCREMENTAL=0" "-e" "RUST_BACKTRACE=full" "-e" "RUSTFLAGS=--cap-lints=warn" "-e" "RUSTDOCFLAGS=--cap-lints=warn" "-w" "/opt/rustwide/workdir" "--user" "0:0" "8481537f0a4ff8394122190d39881f082239a57f66a202cc8a66462ea837b91b" "/opt/rustwide/cargo-home/bin/cargo" "+1.99.0-beta.8" "test" "--frozen" "--no-run" "--message-format=json" "--release", kill_on_drop: false }`
[INFO] [stderr]    Compiling evalforge-core v1.0.0 (/opt/rustwide/workdir/crates/evalforge-core)
[INFO] [stderr]    Compiling evalforge-cli v1.0.0 (/opt/rustwide/workdir/crates/evalforge-cli)
[INFO] [stderr]     Finished `release` profile [optimized] target(s) in 3.87s
[INFO] running `Command { std: "docker" "inspect" "8481537f0a4ff8394122190d39881f082239a57f66a202cc8a66462ea837b91b", kill_on_drop: false }`
[INFO] running `Command { std: "docker" "exec" "-e" "SOURCE_DIR=/opt/rustwide/workdir" "-e" "CARGO_HOME=/opt/rustwide/cargo-home" "-e" "RUSTUP_HOME=/opt/rustwide/rustup-home" "-e" "CARGO_TARGET_DIR=/opt/rustwide/target" "-e" "CARGO_INCREMENTAL=0" "-e" "RUST_BACKTRACE=full" "-e" "RUSTFLAGS=--cap-lints=warn" "-e" "RUSTDOCFLAGS=--cap-lints=warn" "-w" "/opt/rustwide/workdir" "--user" "0:0" "8481537f0a4ff8394122190d39881f082239a57f66a202cc8a66462ea837b91b" "/opt/rustwide/cargo-home/bin/cargo" "+1.99.0-beta.8" "test" "--frozen" "--release", kill_on_drop: false }`
[INFO] [stderr]     Finished `release` profile [optimized] target(s) in 0.19s
[INFO] [stderr]      Running unittests src/main.rs (/opt/rustwide/target/release/deps/evalforge-e9377dc17fda3da6)
[INFO] [stdout] 
[INFO] [stdout] running 27 tests
[INFO] [stdout] test tests::test_calibration_too_generous ... ok
[INFO] [stdout] test tests::test_calibration_too_harsh ... ok
[INFO] [stdout] test tests::test_batch_one_fail ... ok
[INFO] [stdout] test tests::test_compare_improvement ... ok
[INFO] [stdout] test tests::test_compare_regression ... ok
[INFO] [stdout] test tests::test_compare_unchanged ... ok
[INFO] [stdout] test tests::test_batch_pass_rate ... ok
[INFO] [stdout] test tests::test_model_best_value ... ok
[INFO] [stdout] test tests::test_model_best_quality ... ok
[INFO] [stdout] test tests::test_calibration_agreement ... ok
[INFO] [stdout] test tests::test_compare_delta_symbol ... ok
[INFO] [stdout] test tests::test_batch_all_pass ... ok
[INFO] [stdout] test tests::test_recommended_threshold ... ok
[INFO] [stdout] test tests::test_report_metric_average ... ok
[INFO] [stdout] test tests::test_model_cost_haiku ... ok
[INFO] [stdout] test tests::test_report_pass_rate ... ok
[INFO] [stdout] test tests::test_skill_argument_correctness ... ok
[INFO] [stdout] test tests::test_skill_invocation_missing ... ok
[INFO] [stdout] test tests::test_skill_result_utilization ... ok
[INFO] [stdout] test tests::test_slope_degrading ... ok
[INFO] [stdout] test tests::test_model_cost_gpt4o ... ok
[INFO] [stdout] test tests::test_slope_flat ... ok
[INFO] [stdout] test tests::test_slope_perfect_increase ... ok
[INFO] [stdout] test tests::test_slope_requires_two_points ... ok
[INFO] [stdout] test tests::test_skill_invocation_found ... ok
[INFO] [stdout] test tests::test_slope_two_points ... ok
[INFO] [stdout] test tests::test_skill_score_calculation ... ok
[INFO] [stdout] 
[INFO] [stdout] test result: ok. 27 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s
[INFO] [stdout] 
[INFO] [stderr]      Running unittests src/lib.rs (/opt/rustwide/target/release/deps/evalforge_core-64169997053c1881)
[INFO] [stdout] 
[INFO] [stdout] running 31 tests
[INFO] [stdout] test metrics::answer_relevance::tests::test_extract_answer_relevance_input ... ok
[INFO] [stdout] test metrics::code_quality::tests::test_extract_code_quality_input ... ok
[INFO] [stdout] test metrics::code_quality::tests::test_extract_code_quality_no_code ... ok
[INFO] [stdout] test metrics::context_precision::tests::test_extract_context_precision_input ... ok
[INFO] [stdout] test metrics::answer_relevance::tests::test_extract_simple_trace ... ok
[INFO] [stdout] test metrics::goal_completion::tests::test_extract_simple_trace ... ok
[INFO] [stdout] test metrics::code_correctness::tests::test_extract_code_correctness_input ... ok
[INFO] [stdout] test metrics::context_precision::tests::test_zero_chunks_score ... ok
[INFO] [stdout] test metrics::context_precision::tests::test_extract_no_tool_calls ... ok
[INFO] [stdout] test metrics::faithfulness::tests::test_extract_simple_trace_empty_context ... ok
[INFO] [stdout] test metrics::code_correctness::tests::test_extract_code_correctness_no_code ... ok
[INFO] [stdout] test metrics::hallucination::tests::test_hallucination_result_fields ... ok
[INFO] [stdout] test metrics::code_security::tests::test_extract_code_security_no_code ... ok
[INFO] [stdout] test metrics::tool_accuracy::tests::test_extract_from_sample_trace ... ok
[INFO] [stdout] test metrics::g_eval::tests::test_extract_g_eval_input ... ok
[INFO] [stdout] test metrics::tool_accuracy::tests::test_missing_tool ... ok
[INFO] [stdout] test metrics::g_eval::tests::test_extract_g_eval_simple_trace ... ok
[INFO] [stdout] test metrics::g_eval::tests::test_g_eval_result_fields ... ok
[INFO] [stdout] test metrics::tool_accuracy::tests::test_no_tools_expected_none_used ... ok
[INFO] [stdout] test metrics::goal_completion::tests::test_extract_goal_completion_input ... ok
[INFO] [stdout] test metrics::tool_accuracy::tests::test_perfect_score ... ok
[INFO] [stdout] test metrics::hallucination::tests::test_extract_hallucination_input ... ok
[INFO] [stdout] test metrics::hallucination::tests::test_extract_no_tool_calls ... ok
[INFO] [stdout] test metrics::tool_accuracy::tests::test_no_tools_expected_but_used ... ok
[INFO] [stdout] test metrics::faithfulness::tests::test_extract_context_only_from_tool_calls ... ok
[INFO] [stdout] test metrics::faithfulness::tests::test_extract_question_and_answer ... ok
[INFO] [stdout] test trace::tests::test_sample_trace_framework ... ok
[INFO] [stdout] test trace::tests::test_sample_trace_step_count ... ok
[INFO] [stdout] test trace::tests::test_simple_trace_id ... ok
[INFO] [stdout] test metrics::tool_accuracy::tests::test_unexpected_tool ... ok
[INFO] [stderr]    Doc-tests evalforge_core
[INFO] [stdout] test metrics::code_security::tests::test_extract_code_security_input ... ok
[INFO] [stdout] 
[INFO] [stdout] test result: ok. 31 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.02s
[INFO] [stdout] 
[INFO] [stdout] 
[INFO] [stdout] running 0 tests
[INFO] [stdout] 
[INFO] [stdout] test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s
[INFO] [stdout] 
[INFO] running `Command { std: "docker" "inspect" "8481537f0a4ff8394122190d39881f082239a57f66a202cc8a66462ea837b91b", kill_on_drop: false }`
[INFO] running `Command { std: "docker" "rm" "-f" "8481537f0a4ff8394122190d39881f082239a57f66a202cc8a66462ea837b91b", kill_on_drop: false }`
[INFO] [stdout] 8481537f0a4ff8394122190d39881f082239a57f66a202cc8a66462ea837b91b
