[INFO] fetching crate knee_scraper 0.1.8...
[INFO] testing knee_scraper-0.1.8 against try#bd7d74411512a3dd3b35d2f699c51dd2557c7e7e+cargoflags=-Zbuild-dir-new-layout for pr-149852-1
[INFO] extracting crate knee_scraper 0.1.8 into /workspace/builds/worker-0-tc2/source
[INFO] started tweaking crates.io crate knee_scraper 0.1.8
[INFO] finished tweaking crates.io crate knee_scraper 0.1.8
[INFO] tweaked toml for crates.io crate knee_scraper 0.1.8 written to /workspace/builds/worker-0-tc2/source/Cargo.toml
[INFO] validating manifest of crates.io crate knee_scraper 0.1.8 on toolchain bd7d74411512a3dd3b35d2f699c51dd2557c7e7e
[INFO] running `Command { std: CARGO_HOME="/workspace/cargo-home" RUSTUP_HOME="/workspace/rustup-home" "/workspace/cargo-home/bin/cargo" "+bd7d74411512a3dd3b35d2f699c51dd2557c7e7e" "metadata" "--manifest-path" "Cargo.toml" "--no-deps", kill_on_drop: false }`
[INFO] running `Command { std: CARGO_HOME="/workspace/cargo-home" RUSTUP_HOME="/workspace/rustup-home" "/workspace/cargo-home/bin/cargo" "+bd7d74411512a3dd3b35d2f699c51dd2557c7e7e" "generate-lockfile" "--manifest-path" "Cargo.toml", kill_on_drop: false }`
[INFO] [stderr]     Updating crates.io index
[INFO] [stderr]      Locking 225 packages to latest compatible versions
[INFO] [stderr]       Adding rand v0.8.5 (available: v0.9.2)
[INFO] [stderr]       Adding reqwest v0.12.28 (available: v0.13.1)
[INFO] [stderr]       Adding scraper v0.20.0 (available: v0.25.0)
[INFO] running `Command { std: CARGO_HOME="/workspace/cargo-home" RUSTUP_HOME="/workspace/rustup-home" "/workspace/cargo-home/bin/cargo" "+bd7d74411512a3dd3b35d2f699c51dd2557c7e7e" "fetch" "--manifest-path" "Cargo.toml", kill_on_drop: false }`
[INFO] running `Command { std: "docker" "create" "-v" "/var/lib/crater-agent-workspace/builds/worker-0-tc2/target:/opt/rustwide/target:rw,Z" "-v" "/var/lib/crater-agent-workspace/builds/worker-0-tc2/source:/opt/rustwide/workdir:ro,Z" "-v" "/var/lib/crater-agent-workspace/cargo-home:/opt/rustwide/cargo-home:ro,Z" "-v" "/var/lib/crater-agent-workspace/rustup-home:/opt/rustwide/rustup-home:ro,Z" "-e" "SOURCE_DIR=/opt/rustwide/workdir" "-e" "CARGO_TARGET_DIR=/opt/rustwide/target" "-e" "CARGO_HOME=/opt/rustwide/cargo-home" "-e" "RUSTUP_HOME=/opt/rustwide/rustup-home" "-w" "/opt/rustwide/workdir" "-m" "1610612736" "--user" "0:0" "--network" "none" "ghcr.io/rust-lang/crates-build-env/linux@sha256:aa71247004a7fa38d13ec170f48f06cdedf5bc50b2a8645e56ed7e992e6fa513" "/opt/rustwide/cargo-home/bin/cargo" "+bd7d74411512a3dd3b35d2f699c51dd2557c7e7e" "metadata" "--no-deps" "--format-version=1", kill_on_drop: false }`
[INFO] [stdout] 2ac39839e5475c99ae54a9621748b188fcd655f54120971645ceb0c266579d56
[INFO] running `Command { std: "docker" "start" "-a" "2ac39839e5475c99ae54a9621748b188fcd655f54120971645ceb0c266579d56", kill_on_drop: false }`
[INFO] running `Command { std: "docker" "inspect" "2ac39839e5475c99ae54a9621748b188fcd655f54120971645ceb0c266579d56", kill_on_drop: false }`
[INFO] running `Command { std: "docker" "rm" "-f" "2ac39839e5475c99ae54a9621748b188fcd655f54120971645ceb0c266579d56", kill_on_drop: false }`
[INFO] [stdout] 2ac39839e5475c99ae54a9621748b188fcd655f54120971645ceb0c266579d56
[INFO] running `Command { std: "docker" "create" "-v" "/var/lib/crater-agent-workspace/builds/worker-0-tc2/target:/opt/rustwide/target:rw,Z" "-v" "/var/lib/crater-agent-workspace/builds/worker-0-tc2/source:/opt/rustwide/workdir:ro,Z" "-v" "/var/lib/crater-agent-workspace/cargo-home:/opt/rustwide/cargo-home:ro,Z" "-v" "/var/lib/crater-agent-workspace/rustup-home:/opt/rustwide/rustup-home:ro,Z" "-e" "SOURCE_DIR=/opt/rustwide/workdir" "-e" "CARGO_TARGET_DIR=/opt/rustwide/target" "-e" "CARGO_INCREMENTAL=0" "-e" "RUST_BACKTRACE=full" "-e" "RUSTFLAGS=--cap-lints=forbid" "-e" "RUSTDOCFLAGS=--cap-lints=forbid" "-e" "CARGO_HOME=/opt/rustwide/cargo-home" "-e" "RUSTUP_HOME=/opt/rustwide/rustup-home" "-w" "/opt/rustwide/workdir" "-m" "1610612736" "--user" "0:0" "--network" "none" "ghcr.io/rust-lang/crates-build-env/linux@sha256:aa71247004a7fa38d13ec170f48f06cdedf5bc50b2a8645e56ed7e992e6fa513" "/opt/rustwide/cargo-home/bin/cargo" "+bd7d74411512a3dd3b35d2f699c51dd2557c7e7e" "build" "--frozen" "--message-format=json" "-Zbuild-dir-new-layout", kill_on_drop: false }`
[INFO] [stdout] 99b4a61e4889eb54548b2223d1ccc18465d4deddddc34f2a83d24ffafd032bae
[INFO] running `Command { std: "docker" "start" "-a" "99b4a61e4889eb54548b2223d1ccc18465d4deddddc34f2a83d24ffafd032bae", kill_on_drop: false }`
[INFO] [stderr]    Compiling getrandom v0.2.17
[INFO] [stderr]    Compiling parking_lot_core v0.9.12
[INFO] [stderr]    Compiling phf_shared v0.11.3
[INFO] [stderr]    Compiling yoke v0.8.1
[INFO] [stderr]    Compiling siphasher v0.3.11
[INFO] [stderr]    Compiling openssl-sys v0.9.111
[INFO] [stderr]    Compiling futures-macro v0.3.31
[INFO] [stderr]    Compiling new_debug_unreachable v1.0.6
[INFO] [stderr]    Compiling serde v1.0.228
[INFO] [stderr]    Compiling mac v0.1.1
[INFO] [stderr]    Compiling rand_core v0.6.4
[INFO] [stderr]    Compiling precomputed-hash v0.1.1
[INFO] [stderr]    Compiling parking_lot v0.12.5
[INFO] [stderr]    Compiling zerovec v0.11.5
[INFO] [stderr]    Compiling zerotrie v0.2.3
[INFO] [stderr]    Compiling phf_shared v0.10.0
[INFO] [stderr]    Compiling rand_chacha v0.3.1
[INFO] [stderr]    Compiling futf v0.1.5
[INFO] [stderr]    Compiling tokio v1.49.0
[INFO] [stderr]    Compiling rand v0.8.5
[INFO] [stderr]    Compiling openssl v0.10.75
[INFO] [stderr]    Compiling native-tls v0.2.14
[INFO] [stderr]    Compiling dtoa v1.0.11
[INFO] [stderr]    Compiling tendril v0.4.3
[INFO] [stderr]    Compiling string_cache v0.8.9
[INFO] [stderr]    Compiling futures-util v0.3.31
[INFO] [stderr]    Compiling tinystr v0.8.2
[INFO] [stderr]    Compiling potential_utf v0.1.4
[INFO] [stderr]    Compiling dtoa-short v0.3.5
[INFO] [stderr]    Compiling cssparser-macros v0.6.1
[INFO] [stderr]    Compiling icu_locale_core v2.1.1
[INFO] [stderr]    Compiling icu_collections v2.1.1
[INFO] [stderr]    Compiling html5ever v0.27.0
[INFO] [stderr]    Compiling phf v0.10.1
[INFO] [stderr]    Compiling phf_generator v0.11.3
[INFO] [stderr]    Compiling phf_generator v0.10.0
[INFO] [stderr]    Compiling phf_codegen v0.11.3
[INFO] [stderr]    Compiling phf_macros v0.11.3
[INFO] [stderr]    Compiling string_cache_codegen v0.5.4
[INFO] [stderr]    Compiling phf v0.11.3
[INFO] [stderr]    Compiling phf_codegen v0.10.0
[INFO] [stderr]    Compiling icu_provider v2.1.1
[INFO] [stderr]    Compiling selectors v0.25.0
[INFO] [stderr]    Compiling markup5ever v0.12.1
[INFO] [stderr]    Compiling cssparser v0.31.2
[INFO] [stderr]    Compiling icu_properties v2.1.2
[INFO] [stderr]    Compiling icu_normalizer v2.1.1
[INFO] [stderr]    Compiling derive_more v0.99.20
[INFO] [stderr]    Compiling iovec v0.1.4
[INFO] [stderr]    Compiling servo_arc v0.3.0
[INFO] [stderr]    Compiling futures-cpupool v0.1.8
[INFO] [stderr]    Compiling idna_adapter v1.2.1
[INFO] [stderr]    Compiling idna v1.1.0
[INFO] [stderr]    Compiling getopts v0.2.24
[INFO] [stderr]    Compiling serde_urlencoded v0.7.1
[INFO] [stderr]    Compiling futures-executor v0.3.31
[INFO] [stderr]    Compiling url v2.5.8
[INFO] [stderr]    Compiling bytes v0.4.12
[INFO] [stderr]    Compiling ego-tree v0.6.3
[INFO] [stderr]    Compiling scraper v0.20.0
[INFO] [stderr]    Compiling tempfile v3.24.0
[INFO] [stderr]    Compiling fs v0.0.5
[INFO] [stderr]    Compiling futures v0.3.31
[INFO] [stderr]    Compiling tokio-util v0.7.18
[INFO] [stderr]    Compiling tower v0.5.3
[INFO] [stderr]    Compiling tokio-native-tls v0.3.1
[INFO] [stderr]    Compiling tower-http v0.6.8
[INFO] [stderr]    Compiling h2 v0.4.13
[INFO] [stderr]    Compiling hyper v1.8.1
[INFO] [stderr]    Compiling hyper-util v0.1.19
[INFO] [stderr]    Compiling hyper-tls v0.6.0
[INFO] [stderr]    Compiling reqwest v0.12.28
[INFO] [stderr]    Compiling knee_scraper v0.1.8 (/opt/rustwide/workdir)
[INFO] [stdout] warning: unused import: `tempfile::TempDir`
[INFO] [stdout]  --> src/lib.rs:9:5
[INFO] [stdout]   |
[INFO] [stdout] 9 | use tempfile::TempDir;
[INFO] [stdout]   |     ^^^^^^^^^^^^^^^^^
[INFO] [stdout]   |
[INFO] [stdout]   = note: `#[warn(unused_imports)]` (part of `#[warn(unused)]`) on by default
[INFO] [stdout] 
[INFO] [stdout] 
[INFO] [stderr]     Finished `dev` profile [unoptimized + debuginfo] target(s) in 1m 02s
[INFO] running `Command { std: "docker" "inspect" "99b4a61e4889eb54548b2223d1ccc18465d4deddddc34f2a83d24ffafd032bae", kill_on_drop: false }`
[INFO] running `Command { std: "docker" "rm" "-f" "99b4a61e4889eb54548b2223d1ccc18465d4deddddc34f2a83d24ffafd032bae", kill_on_drop: false }`
[INFO] [stdout] 99b4a61e4889eb54548b2223d1ccc18465d4deddddc34f2a83d24ffafd032bae
[INFO] running `Command { std: "docker" "create" "-v" "/var/lib/crater-agent-workspace/builds/worker-0-tc2/target:/opt/rustwide/target:rw,Z" "-v" "/var/lib/crater-agent-workspace/builds/worker-0-tc2/source:/opt/rustwide/workdir:ro,Z" "-v" "/var/lib/crater-agent-workspace/cargo-home:/opt/rustwide/cargo-home:ro,Z" "-v" "/var/lib/crater-agent-workspace/rustup-home:/opt/rustwide/rustup-home:ro,Z" "-e" "SOURCE_DIR=/opt/rustwide/workdir" "-e" "CARGO_TARGET_DIR=/opt/rustwide/target" "-e" "CARGO_INCREMENTAL=0" "-e" "RUST_BACKTRACE=full" "-e" "RUSTFLAGS=--cap-lints=forbid" "-e" "RUSTDOCFLAGS=--cap-lints=forbid" "-e" "CARGO_HOME=/opt/rustwide/cargo-home" "-e" "RUSTUP_HOME=/opt/rustwide/rustup-home" "-w" "/opt/rustwide/workdir" "-m" "1610612736" "--user" "0:0" "--network" "none" "ghcr.io/rust-lang/crates-build-env/linux@sha256:aa71247004a7fa38d13ec170f48f06cdedf5bc50b2a8645e56ed7e992e6fa513" "/opt/rustwide/cargo-home/bin/cargo" "+bd7d74411512a3dd3b35d2f699c51dd2557c7e7e" "test" "--frozen" "--no-run" "--message-format=json" "-Zbuild-dir-new-layout", kill_on_drop: false }`
[INFO] [stdout] 8996993d0e5d8e5da1967ea096de5930ae73fffd9f5407fa425476467d66ced1
[INFO] running `Command { std: "docker" "start" "-a" "8996993d0e5d8e5da1967ea096de5930ae73fffd9f5407fa425476467d66ced1", kill_on_drop: false }`
[INFO] [stdout] warning: unused import: `tempfile::TempDir`
[INFO] [stdout]  --> src/lib.rs:9:5
[INFO] [stdout]   |
[INFO] [stdout] 9 | use tempfile::TempDir;
[INFO] [stdout]   |     ^^^^^^^^^^^^^^^^^
[INFO] [stdout]   |
[INFO] [stdout]   = note: `#[warn(unused_imports)]` (part of `#[warn(unused)]`) on by default
[INFO] [stdout] 
[INFO] [stdout] 
[INFO] [stderr]    Compiling knee_scraper v0.1.8 (/opt/rustwide/workdir)
[INFO] [stdout] warning: unused import: `tempfile::TempDir`
[INFO] [stdout]  --> src/lib.rs:9:5
[INFO] [stdout]   |
[INFO] [stdout] 9 | use tempfile::TempDir;
[INFO] [stdout]   |     ^^^^^^^^^^^^^^^^^
[INFO] [stdout]   |
[INFO] [stdout]   = note: `#[warn(unused_imports)]` (part of `#[warn(unused)]`) on by default
[INFO] [stdout] 
[INFO] [stdout] 
[INFO] [stderr]     Finished `test` profile [unoptimized + debuginfo] target(s) in 3.62s
[INFO] running `Command { std: "docker" "inspect" "8996993d0e5d8e5da1967ea096de5930ae73fffd9f5407fa425476467d66ced1", kill_on_drop: false }`
[INFO] running `Command { std: "docker" "rm" "-f" "8996993d0e5d8e5da1967ea096de5930ae73fffd9f5407fa425476467d66ced1", kill_on_drop: false }`
[INFO] [stdout] 8996993d0e5d8e5da1967ea096de5930ae73fffd9f5407fa425476467d66ced1
[INFO] running `Command { std: "docker" "create" "-v" "/var/lib/crater-agent-workspace/builds/worker-0-tc2/target:/opt/rustwide/target:rw,Z" "-v" "/var/lib/crater-agent-workspace/builds/worker-0-tc2/source:/opt/rustwide/workdir:ro,Z" "-v" "/var/lib/crater-agent-workspace/cargo-home:/opt/rustwide/cargo-home:ro,Z" "-v" "/var/lib/crater-agent-workspace/rustup-home:/opt/rustwide/rustup-home:ro,Z" "-e" "SOURCE_DIR=/opt/rustwide/workdir" "-e" "CARGO_TARGET_DIR=/opt/rustwide/target" "-e" "CARGO_INCREMENTAL=0" "-e" "RUST_BACKTRACE=full" "-e" "RUSTFLAGS=--cap-lints=forbid" "-e" "RUSTDOCFLAGS=--cap-lints=forbid" "-e" "CARGO_HOME=/opt/rustwide/cargo-home" "-e" "RUSTUP_HOME=/opt/rustwide/rustup-home" "-w" "/opt/rustwide/workdir" "-m" "1610612736" "--user" "0:0" "--network" "none" "ghcr.io/rust-lang/crates-build-env/linux@sha256:aa71247004a7fa38d13ec170f48f06cdedf5bc50b2a8645e56ed7e992e6fa513" "/opt/rustwide/cargo-home/bin/cargo" "+bd7d74411512a3dd3b35d2f699c51dd2557c7e7e" "test" "--frozen" "-Zbuild-dir-new-layout", kill_on_drop: false }`
[INFO] [stdout] 71d80e539c46da220e53b6e114a578f9ec27ffc7f041f6b73ae69a8ab9be4579
[INFO] running `Command { std: "docker" "start" "-a" "71d80e539c46da220e53b6e114a578f9ec27ffc7f041f6b73ae69a8ab9be4579", kill_on_drop: false }`
[INFO] [stderr] warning: unused import: `tempfile::TempDir`
[INFO] [stderr]  --> src/lib.rs:9:5
[INFO] [stderr]   |
[INFO] [stderr] 9 | use tempfile::TempDir;
[INFO] [stderr]   |     ^^^^^^^^^^^^^^^^^
[INFO] [stderr]   |
[INFO] [stderr]   = note: `#[warn(unused_imports)]` (part of `#[warn(unused)]`) on by default
[INFO] [stderr] 
[INFO] [stderr] warning: `knee_scraper` (lib) generated 1 warning (run `cargo fix --lib -p knee_scraper` to apply 1 suggestion)
[INFO] [stderr] warning: `knee_scraper` (lib test) generated 1 warning (1 duplicate)
[INFO] [stderr]     Finished `test` profile [unoptimized + debuginfo] target(s) in 0.28s
[INFO] [stderr]      Running unittests src/lib.rs (/opt/rustwide/target/debug/build/knee_scraper/f8f6f8b6ee03d6c8/deps/knee_scraper-f8f6f8b6ee03d6c8)
[INFO] [stdout] 
[INFO] [stdout] running 7 tests
[INFO] [stdout] test tests::test_cleanup ... ok
[INFO] [stdout] test tests::test_normalize_link ... ok
[INFO] [stdout] test tests::test_random_user_agent ... ok
[INFO] [stdout] test tests::test_extract_links ... ok
[INFO] [stdout] test tests::test_scrape_for_emails ... ok
[INFO] [stdout] test tests::test_recursive_scrape ... ok
[INFO] [stdout] test tests::test_download_media ... ok
[INFO] [stdout] 
[INFO] [stdout] test result: ok. 7 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.23s
[INFO] [stdout] 
[INFO] [stderr]    Doc-tests knee_scraper
[INFO] [stdout] 
[INFO] [stdout] running 16 tests
[INFO] [stdout] test src/lib.rs - normalize_link (line 153) ... FAILED
[INFO] [stdout] test src/lib.rs - fetch_robots_txt (line 503) ... FAILED
[INFO] [stdout] test src/lib.rs - extract_links (line 122) ... FAILED
[INFO] [stdout] test src/lib.rs - download_media (line 179) ... FAILED
[INFO] [stdout] test src/lib.rs - fetch_with_cookies (line 460) ... FAILED
[INFO] [stdout] test src/lib.rs - check_open_directories (line 479) ... FAILED
[INFO] [stdout] test src/lib.rs - extract_domain (line 364) ... FAILED
[INFO] [stdout] test src/lib.rs - ai_scrape (line 995) ... FAILED
[INFO] [stdout] test src/lib.rs - random_delay (line 599) ... FAILED
[INFO] [stdout] test src/lib.rs - scrape_for_emails (line 422) ... FAILED
[INFO] [stdout] test src/lib.rs - random_user_agent (line 33) ... FAILED
[INFO] [stdout] test src/lib.rs - scrape_for_errors (line 404) ... FAILED
[INFO] [stdout] test src/lib.rs - recursive_scrape (line 59) ... FAILED
[INFO] [stdout] test src/lib.rs - run (line 537) ... FAILED
[INFO] [stdout] test src/lib.rs - scrape_js (line 381) ... FAILED
[INFO] [stdout] test src/lib.rs - scrape_content (line 251) ... FAILED
[INFO] [stdout] 
[INFO] [stdout] failures:
[INFO] [stdout] 
[INFO] [stdout] ---- src/lib.rs - normalize_link (line 153) stdout ----
[INFO] [stdout] error[E0425]: cannot find function `normalize_link` in this scope
[INFO] [stdout]    --> src/lib.rs:154:21
[INFO] [stdout]     |
[INFO] [stdout] 154 | let absolute_link = normalize_link("/about", "https://example.com");
[INFO] [stdout]     |                     ^^^^^^^^^^^^^^ not found in this scope
[INFO] [stdout]     |
[INFO] [stdout] help: consider importing this function
[INFO] [stdout]     |
[INFO] [stdout] 153 + use knee_scraper::normalize_link;
[INFO] [stdout]     |
[INFO] [stdout] 
[INFO] [stdout] error: aborting due to 1 previous error
[INFO] [stdout] 
[INFO] [stdout] For more information about this error, try `rustc --explain E0425`.
[INFO] [stdout] Couldn't compile the test.
[INFO] [stdout] ---- src/lib.rs - fetch_robots_txt (line 503) stdout ----
[INFO] [stdout] error[E0425]: cannot find function `fetch_robots_txt` in this scope
[INFO] [stdout]    --> src/lib.rs:504:1
[INFO] [stdout]     |
[INFO] [stdout] 504 | fetch_robots_txt("https://example.com", &client).await;
[INFO] [stdout]     | ^^^^^^^^^^^^^^^^ not found in this scope
[INFO] [stdout]     |
[INFO] [stdout] help: consider importing this function
[INFO] [stdout]     |
[INFO] [stdout] 503 + use knee_scraper::fetch_robots_txt;
[INFO] [stdout]     |
[INFO] [stdout] 
[INFO] [stdout] error[E0425]: cannot find value `client` in this scope
[INFO] [stdout]    --> src/lib.rs:504:42
[INFO] [stdout]     |
[INFO] [stdout] 504 | fetch_robots_txt("https://example.com", &client).await;
[INFO] [stdout]     |                                          ^^^^^^ not found in this scope
[INFO] [stdout] 
[INFO] [stdout] error[E0728]: `await` is only allowed inside `async` functions and blocks
[INFO] [stdout]    --> src/lib.rs:504:50
[INFO] [stdout]     |
[INFO] [stdout] 503 | fn main() { #[allow(non_snake_case)] fn _doctest_main_src_lib_rs_503_0() {
[INFO] [stdout]     |                                      ----------------------------------- this is not `async`
[INFO] [stdout] 504 | fetch_robots_txt("https://example.com", &client).await;
[INFO] [stdout]     |                                                  ^^^^^ only allowed inside `async` functions and blocks
[INFO] [stdout] 
[INFO] [stdout] error: aborting due to 3 previous errors
[INFO] [stdout] 
[INFO] [stdout] Some errors have detailed explanations: E0425, E0728.
[INFO] [stdout] For more information about an error, try `rustc --explain E0425`.
[INFO] [stdout] Couldn't compile the test.
[INFO] [stdout] ---- src/lib.rs - extract_links (line 122) stdout ----
[INFO] [stdout] error[E0425]: cannot find function `extract_links` in this scope
[INFO] [stdout]    --> src/lib.rs:123:13
[INFO] [stdout]     |
[INFO] [stdout] 123 | let links = extract_links("<a href='/about'>About</a>", "https://example.com");
[INFO] [stdout]     |             ^^^^^^^^^^^^^ not found in this scope
[INFO] [stdout]     |
[INFO] [stdout] help: consider importing this function
[INFO] [stdout]     |
[INFO] [stdout] 122 + use knee_scraper::extract_links;
[INFO] [stdout]     |
[INFO] [stdout] 
[INFO] [stdout] error: aborting due to 1 previous error
[INFO] [stdout] 
[INFO] [stdout] For more information about this error, try `rustc --explain E0425`.
[INFO] [stdout] Couldn't compile the test.
[INFO] [stdout] ---- src/lib.rs - download_media (line 179) stdout ----
[INFO] [stdout] error[E0425]: cannot find function `download_media` in this scope
[INFO] [stdout]    --> src/lib.rs:180:1
[INFO] [stdout]     |
[INFO] [stdout] 180 | download_media(&client, "https://example.com/image.jpg", Path::new("./downloads/image.jpg")).await;
[INFO] [stdout]     | ^^^^^^^^^^^^^^ not found in this scope
[INFO] [stdout]     |
[INFO] [stdout] help: consider importing this function
[INFO] [stdout]     |
[INFO] [stdout] 179 + use knee_scraper::download_media;
[INFO] [stdout]     |
[INFO] [stdout] 
[INFO] [stdout] error[E0425]: cannot find value `client` in this scope
[INFO] [stdout]    --> src/lib.rs:180:17
[INFO] [stdout]     |
[INFO] [stdout] 180 | download_media(&client, "https://example.com/image.jpg", Path::new("./downloads/image.jpg")).await;
[INFO] [stdout]     |                 ^^^^^^ not found in this scope
[INFO] [stdout] 
[INFO] [stdout] error[E0433]: failed to resolve: use of undeclared type `Path`
[INFO] [stdout]    --> src/lib.rs:180:58
[INFO] [stdout]     |
[INFO] [stdout] 180 | download_media(&client, "https://example.com/image.jpg", Path::new("./downloads/image.jpg")).await;
[INFO] [stdout]     |                                                          ^^^^ use of undeclared type `Path`
[INFO] [stdout]     |
[INFO] [stdout] help: consider importing this struct
[INFO] [stdout]     |
[INFO] [stdout] 179 + use std::path::Path;
[INFO] [stdout]     |
[INFO] [stdout] 
[INFO] [stdout] error[E0728]: `await` is only allowed inside `async` functions and blocks
[INFO] [stdout]    --> src/lib.rs:180:94
[INFO] [stdout]     |
[INFO] [stdout] 179 | fn main() { #[allow(non_snake_case)] fn _doctest_main_src_lib_rs_179_0() {
[INFO] [stdout]     |                                      ----------------------------------- this is not `async`
[INFO] [stdout] 180 | download_media(&client, "https://example.com/image.jpg", Path::new("./downloads/image.jpg")).await;
[INFO] [stdout]     |                                                                                              ^^^^^ only allowed inside `async` functions and blocks
[INFO] [stdout] 
[INFO] [stdout] error: aborting due to 4 previous errors
[INFO] [stdout] 
[INFO] [stdout] Some errors have detailed explanations: E0425, E0433, E0728.
[INFO] [stdout] For more information about an error, try `rustc --explain E0425`.
[INFO] [stdout] Couldn't compile the test.
[INFO] [stdout] ---- src/lib.rs - fetch_with_cookies (line 460) stdout ----
[INFO] [stdout] error[E0425]: cannot find function `fetch_with_cookies` in this scope
[INFO] [stdout]    --> src/lib.rs:461:1
[INFO] [stdout]     |
[INFO] [stdout] 461 | fetch_with_cookies("https://example.com", &client).await;
[INFO] [stdout]     | ^^^^^^^^^^^^^^^^^^ not found in this scope
[INFO] [stdout]     |
[INFO] [stdout] help: consider importing this function
[INFO] [stdout]     |
[INFO] [stdout] 460 + use knee_scraper::fetch_with_cookies;
[INFO] [stdout]     |
[INFO] [stdout] 
[INFO] [stdout] error[E0425]: cannot find value `client` in this scope
[INFO] [stdout]    --> src/lib.rs:461:44
[INFO] [stdout]     |
[INFO] [stdout] 461 | fetch_with_cookies("https://example.com", &client).await;
[INFO] [stdout]     |                                            ^^^^^^ not found in this scope
[INFO] [stdout] 
[INFO] [stdout] error[E0728]: `await` is only allowed inside `async` functions and blocks
[INFO] [stdout]    --> src/lib.rs:461:52
[INFO] [stdout]     |
[INFO] [stdout] 460 | fn main() { #[allow(non_snake_case)] fn _doctest_main_src_lib_rs_460_0() {
[INFO] [stdout]     |                                      ----------------------------------- this is not `async`
[INFO] [stdout] 461 | fetch_with_cookies("https://example.com", &client).await;
[INFO] [stdout]     |                                                    ^^^^^ only allowed inside `async` functions and blocks
[INFO] [stdout] 
[INFO] [stdout] error: aborting due to 3 previous errors
[INFO] [stdout] 
[INFO] [stdout] Some errors have detailed explanations: E0425, E0728.
[INFO] [stdout] For more information about an error, try `rustc --explain E0425`.
[INFO] [stdout] Couldn't compile the test.
[INFO] [stdout] ---- src/lib.rs - check_open_directories (line 479) stdout ----
[INFO] [stdout] error[E0425]: cannot find function `check_open_directories` in this scope
[INFO] [stdout]    --> src/lib.rs:480:1
[INFO] [stdout]     |
[INFO] [stdout] 480 | check_open_directories("https://example.com", &client).await;
[INFO] [stdout]     | ^^^^^^^^^^^^^^^^^^^^^^ not found in this scope
[INFO] [stdout]     |
[INFO] [stdout] help: consider importing this function
[INFO] [stdout]     |
[INFO] [stdout] 479 + use knee_scraper::check_open_directories;
[INFO] [stdout]     |
[INFO] [stdout] 
[INFO] [stdout] error[E0425]: cannot find value `client` in this scope
[INFO] [stdout]    --> src/lib.rs:480:48
[INFO] [stdout]     |
[INFO] [stdout] 480 | check_open_directories("https://example.com", &client).await;
[INFO] [stdout]     |                                                ^^^^^^ not found in this scope
[INFO] [stdout] 
[INFO] [stdout] error[E0728]: `await` is only allowed inside `async` functions and blocks
[INFO] [stdout]    --> src/lib.rs:480:56
[INFO] [stdout]     |
[INFO] [stdout] 479 | fn main() { #[allow(non_snake_case)] fn _doctest_main_src_lib_rs_479_0() {
[INFO] [stdout]     |                                      ----------------------------------- this is not `async`
[INFO] [stdout] 480 | check_open_directories("https://example.com", &client).await;
[INFO] [stdout]     |                                                        ^^^^^ only allowed inside `async` functions and blocks
[INFO] [stdout] 
[INFO] [stdout] error: aborting due to 3 previous errors
[INFO] [stdout] 
[INFO] [stdout] Some errors have detailed explanations: E0425, E0728.
[INFO] [stdout] For more information about an error, try `rustc --explain E0425`.
[INFO] [stdout] Couldn't compile the test.
[INFO] [stdout] ---- src/lib.rs - extract_domain (line 364) stdout ----
[INFO] [stdout] error[E0425]: cannot find function `extract_domain` in this scope
[INFO] [stdout]    --> src/lib.rs:365:14
[INFO] [stdout]     |
[INFO] [stdout] 365 | let domain = extract_domain("https://example.com/path");
[INFO] [stdout]     |              ^^^^^^^^^^^^^^ not found in this scope
[INFO] [stdout]     |
[INFO] [stdout] help: consider importing this function
[INFO] [stdout]     |
[INFO] [stdout] 364 + use knee_scraper::extract_domain;
[INFO] [stdout]     |
[INFO] [stdout] 
[INFO] [stdout] error: aborting due to 1 previous error
[INFO] [stdout] 
[INFO] [stdout] For more information about this error, try `rustc --explain E0425`.
[INFO] [stdout] Couldn't compile the test.
[INFO] [stdout] ---- src/lib.rs - ai_scrape (line 995) stdout ----
[INFO] [stdout] error[E0425]: cannot find function `ai_scrape` in this scope
[INFO] [stdout]    --> src/lib.rs:996:1
[INFO] [stdout]     |
[INFO] [stdout] 996 | ai_scrape("https://example.com", &client, &mut visited).await;
[INFO] [stdout]     | ^^^^^^^^^ not found in this scope
[INFO] [stdout]     |
[INFO] [stdout] help: consider importing this function
[INFO] [stdout]     |
[INFO] [stdout] 995 + use knee_scraper::ai_scrape;
[INFO] [stdout]     |
[INFO] [stdout] 
[INFO] [stdout] error[E0425]: cannot find value `client` in this scope
[INFO] [stdout]    --> src/lib.rs:996:35
[INFO] [stdout]     |
[INFO] [stdout] 996 | ai_scrape("https://example.com", &client, &mut visited).await;
[INFO] [stdout]     |                                   ^^^^^^ not found in this scope
[INFO] [stdout] 
[INFO] [stdout] error[E0425]: cannot find value `visited` in this scope
[INFO] [stdout]    --> src/lib.rs:996:48
[INFO] [stdout]     |
[INFO] [stdout] 996 | ai_scrape("https://example.com", &client, &mut visited).await;
[INFO] [stdout]     |                                                ^^^^^^^ not found in this scope
[INFO] [stdout] 
[INFO] [stdout] error[E0728]: `await` is only allowed inside `async` functions and blocks
[INFO] [stdout]    --> src/lib.rs:996:57
[INFO] [stdout]     |
[INFO] [stdout] 995 | fn main() { #[allow(non_snake_case)] fn _doctest_main_src_lib_rs_995_0() {
[INFO] [stdout]     |                                      ----------------------------------- this is not `async`
[INFO] [stdout] 996 | ai_scrape("https://example.com", &client, &mut visited).await;
[INFO] [stdout]     |                                                         ^^^^^ only allowed inside `async` functions and blocks
[INFO] [stdout] 
[INFO] [stdout] error: aborting due to 4 previous errors
[INFO] [stdout] 
[INFO] [stdout] Some errors have detailed explanations: E0425, E0728.
[INFO] [stdout] For more information about an error, try `rustc --explain E0425`.
[INFO] [stdout] Couldn't compile the test.
[INFO] [stdout] ---- src/lib.rs - random_delay (line 599) stdout ----
[INFO] [stdout] error[E0425]: cannot find function `random_delay` in this scope
[INFO] [stdout]    --> src/lib.rs:600:1
[INFO] [stdout]     |
[INFO] [stdout] 600 | random_delay(1, 5).await;
[INFO] [stdout]     | ^^^^^^^^^^^^ not found in this scope
[INFO] [stdout]     |
[INFO] [stdout] help: consider importing this function
[INFO] [stdout]     |
[INFO] [stdout] 599 + use knee_scraper::random_delay;
[INFO] [stdout]     |
[INFO] [stdout] 
[INFO] [stdout] error[E0728]: `await` is only allowed inside `async` functions and blocks
[INFO] [stdout]    --> src/lib.rs:600:20
[INFO] [stdout]     |
[INFO] [stdout] 599 | fn main() { #[allow(non_snake_case)] fn _doctest_main_src_lib_rs_599_0() {
[INFO] [stdout]     |                                      ----------------------------------- this is not `async`
[INFO] [stdout] 600 | random_delay(1, 5).await;
[INFO] [stdout]     |                    ^^^^^ only allowed inside `async` functions and blocks
[INFO] [stdout] 
[INFO] [stdout] error: aborting due to 2 previous errors
[INFO] [stdout] 
[INFO] [stdout] Some errors have detailed explanations: E0425, E0728.
[INFO] [stdout] For more information about an error, try `rustc --explain E0425`.
[INFO] [stdout] Couldn't compile the test.
[INFO] [stdout] ---- src/lib.rs - scrape_for_emails (line 422) stdout ----
[INFO] [stdout] error[E0425]: cannot find function `scrape_for_emails` in this scope
[INFO] [stdout]    --> src/lib.rs:423:1
[INFO] [stdout]     |
[INFO] [stdout] 423 | scrape_for_emails("<p>Contact us at info@example.com</p>", "./scraped_data/example.com");
[INFO] [stdout]     | ^^^^^^^^^^^^^^^^^ not found in this scope
[INFO] [stdout]     |
[INFO] [stdout] help: consider importing this function
[INFO] [stdout]     |
[INFO] [stdout] 422 + use knee_scraper::scrape_for_emails;
[INFO] [stdout]     |
[INFO] [stdout] 
[INFO] [stdout] error: aborting due to 1 previous error
[INFO] [stdout] 
[INFO] [stdout] For more information about this error, try `rustc --explain E0425`.
[INFO] [stdout] Couldn't compile the test.
[INFO] [stdout] ---- src/lib.rs - random_user_agent (line 33) stdout ----
[INFO] [stdout] error[E0425]: cannot find function `random_user_agent` in this scope
[INFO] [stdout]   --> src/lib.rs:34:18
[INFO] [stdout]    |
[INFO] [stdout] 34 | let user_agent = random_user_agent();
[INFO] [stdout]    |                  ^^^^^^^^^^^^^^^^^ not found in this scope
[INFO] [stdout]    |
[INFO] [stdout] help: consider importing this function
[INFO] [stdout]    |
[INFO] [stdout] 33 + use knee_scraper::random_user_agent;
[INFO] [stdout]    |
[INFO] [stdout] 
[INFO] [stdout] error: aborting due to 1 previous error
[INFO] [stdout] 
[INFO] [stdout] For more information about this error, try `rustc --explain E0425`.
[INFO] [stdout] Couldn't compile the test.
[INFO] [stdout] ---- src/lib.rs - scrape_for_errors (line 404) stdout ----
[INFO] [stdout] error[E0425]: cannot find function `scrape_for_errors` in this scope
[INFO] [stdout]    --> src/lib.rs:405:1
[INFO] [stdout]     |
[INFO] [stdout] 405 | scrape_for_errors("<html><body>Error: Stack trace</body></html>");
[INFO] [stdout]     | ^^^^^^^^^^^^^^^^^ not found in this scope
[INFO] [stdout]     |
[INFO] [stdout] help: consider importing this function
[INFO] [stdout]     |
[INFO] [stdout] 404 + use knee_scraper::scrape_for_errors;
[INFO] [stdout]     |
[INFO] [stdout] 
[INFO] [stdout] error: aborting due to 1 previous error
[INFO] [stdout] 
[INFO] [stdout] For more information about this error, try `rustc --explain E0425`.
[INFO] [stdout] Couldn't compile the test.
[INFO] [stdout] ---- src/lib.rs - recursive_scrape (line 59) stdout ----
[INFO] [stdout] error[E0433]: failed to resolve: use of undeclared type `Client`
[INFO] [stdout]   --> src/lib.rs:60:14
[INFO] [stdout]    |
[INFO] [stdout] 60 | let client = Client::new();
[INFO] [stdout]    |              ^^^^^^ use of undeclared type `Client`
[INFO] [stdout]    |
[INFO] [stdout] help: consider importing this struct
[INFO] [stdout]    |
[INFO] [stdout] 59 + use reqwest::Client;
[INFO] [stdout]    |
[INFO] [stdout] 
[INFO] [stdout] error[E0433]: failed to resolve: use of undeclared type `HashSet`
[INFO] [stdout]   --> src/lib.rs:61:19
[INFO] [stdout]    |
[INFO] [stdout] 61 | let mut visited = HashSet::new();
[INFO] [stdout]    |                   ^^^^^^^ use of undeclared type `HashSet`
[INFO] [stdout]    |
[INFO] [stdout] help: consider importing this struct
[INFO] [stdout]    |
[INFO] [stdout] 59 + use std::collections::HashSet;
[INFO] [stdout]    |
[INFO] [stdout] 
[INFO] [stdout] error[E0425]: cannot find function `recursive_scrape` in this scope
[INFO] [stdout]   --> src/lib.rs:62:1
[INFO] [stdout]    |
[INFO] [stdout] 62 | recursive_scrape("https://example.com", &client, &mut visited).await;
[INFO] [stdout]    | ^^^^^^^^^^^^^^^^ not found in this scope
[INFO] [stdout]    |
[INFO] [stdout] help: consider importing this function
[INFO] [stdout]    |
[INFO] [stdout] 59 + use knee_scraper::recursive_scrape;
[INFO] [stdout]    |
[INFO] [stdout] 
[INFO] [stdout] error[E0728]: `await` is only allowed inside `async` functions and blocks
[INFO] [stdout]   --> src/lib.rs:62:64
[INFO] [stdout]    |
[INFO] [stdout] 59 | fn main() { #[allow(non_snake_case)] fn _doctest_main_src_lib_rs_59_0() {
[INFO] [stdout]    |                                      ---------------------------------- this is not `async`
[INFO] [stdout] ...
[INFO] [stdout] 62 | recursive_scrape("https://example.com", &client, &mut visited).await;
[INFO] [stdout]    |                                                                ^^^^^ only allowed inside `async` functions and blocks
[INFO] [stdout] 
[INFO] [stdout] error: aborting due to 4 previous errors
[INFO] [stdout] 
[INFO] [stdout] Some errors have detailed explanations: E0425, E0433, E0728.
[INFO] [stdout] For more information about an error, try `rustc --explain E0425`.
[INFO] [stdout] Couldn't compile the test.
[INFO] [stdout] ---- src/lib.rs - run (line 537) stdout ----
[INFO] [stdout] error[E0433]: failed to resolve: use of undeclared type `Client`
[INFO] [stdout]    --> src/lib.rs:538:14
[INFO] [stdout]     |
[INFO] [stdout] 538 | let client = Client::new();
[INFO] [stdout]     |              ^^^^^^ use of undeclared type `Client`
[INFO] [stdout]     |
[INFO] [stdout] help: consider importing this struct
[INFO] [stdout]     |
[INFO] [stdout] 537 + use reqwest::Client;
[INFO] [stdout]     |
[INFO] [stdout] 
[INFO] [stdout] error[E0425]: cannot find function `run` in this scope
[INFO] [stdout]    --> src/lib.rs:539:1
[INFO] [stdout]     |
[INFO] [stdout] 539 | run("https://example.com", &client).await;
[INFO] [stdout]     | ^^^ not found in this scope
[INFO] [stdout]     |
[INFO] [stdout] help: consider importing this function
[INFO] [stdout]     |
[INFO] [stdout] 537 + use knee_scraper::run;
[INFO] [stdout]     |
[INFO] [stdout] 
[INFO] [stdout] error[E0728]: `await` is only allowed inside `async` functions and blocks
[INFO] [stdout]    --> src/lib.rs:539:37
[INFO] [stdout]     |
[INFO] [stdout] 537 | fn main() { #[allow(non_snake_case)] fn _doctest_main_src_lib_rs_537_0() {
[INFO] [stdout]     |                                      ----------------------------------- this is not `async`
[INFO] [stdout] 538 | let client = Client::new();
[INFO] [stdout] 539 | run("https://example.com", &client).await;
[INFO] [stdout]     |                                     ^^^^^ only allowed inside `async` functions and blocks
[INFO] [stdout] 
[INFO] [stdout] error: aborting due to 3 previous errors
[INFO] [stdout] 
[INFO] [stdout] Some errors have detailed explanations: E0425, E0433, E0728.
[INFO] [stdout] For more information about an error, try `rustc --explain E0425`.
[INFO] [stdout] Couldn't compile the test.
[INFO] [stdout] ---- src/lib.rs - scrape_js (line 381) stdout ----
[INFO] [stdout] error[E0425]: cannot find function `scrape_js_content` in this scope
[INFO] [stdout]    --> src/lib.rs:382:1
[INFO] [stdout]     |
[INFO] [stdout] 382 | scrape_js_content("<script>var apiKey = '12345';</script>");
[INFO] [stdout]     | ^^^^^^^^^^^^^^^^^ not found in this scope
[INFO] [stdout]     |
[INFO] [stdout] help: consider importing this function
[INFO] [stdout]     |
[INFO] [stdout] 381 + use knee_scraper::scrape_js_content;
[INFO] [stdout]     |
[INFO] [stdout] 
[INFO] [stdout] error: aborting due to 1 previous error
[INFO] [stdout] 
[INFO] [stdout] For more information about this error, try `rustc --explain E0425`.
[INFO] [stdout] Couldn't compile the test.
[INFO] [stdout] ---- src/lib.rs - scrape_content (line 251) stdout ----
[INFO] [stdout] error[E0425]: cannot find function `scrape_content` in this scope
[INFO] [stdout]    --> src/lib.rs:252:1
[INFO] [stdout]     |
[INFO] [stdout] 252 | scrape_content("<html>...</html>", "https://example.com", &client).await;
[INFO] [stdout]     | ^^^^^^^^^^^^^^ not found in this scope
[INFO] [stdout]     |
[INFO] [stdout] help: consider importing this function
[INFO] [stdout]     |
[INFO] [stdout] 251 + use knee_scraper::scrape_content;
[INFO] [stdout]     |
[INFO] [stdout] 
[INFO] [stdout] error[E0425]: cannot find value `client` in this scope
[INFO] [stdout]    --> src/lib.rs:252:60
[INFO] [stdout]     |
[INFO] [stdout] 252 | scrape_content("<html>...</html>", "https://example.com", &client).await;
[INFO] [stdout]     |                                                            ^^^^^^ not found in this scope
[INFO] [stdout] 
[INFO] [stdout] error[E0728]: `await` is only allowed inside `async` functions and blocks
[INFO] [stdout]    --> src/lib.rs:252:68
[INFO] [stdout]     |
[INFO] [stdout] 251 | fn main() { #[allow(non_snake_case)] fn _doctest_main_src_lib_rs_251_0() {
[INFO] [stdout]     |                                      ----------------------------------- this is not `async`
[INFO] [stdout] 252 | scrape_content("<html>...</html>", "https://example.com", &client).await;
[INFO] [stdout]     |                                                                    ^^^^^ only allowed inside `async` functions and blocks
[INFO] [stdout] 
[INFO] [stdout] error: aborting due to 3 previous errors
[INFO] [stdout] 
[INFO] [stdout] Some errors have detailed explanations: E0425, E0728.
[INFO] [stdout] For more information about an error, try `rustc --explain E0425`.
[INFO] [stdout] Couldn't compile the test.
[INFO] [stdout] 
[INFO] [stdout] failures:
[INFO] [stdout]     src/lib.rs - ai_scrape (line 995)
[INFO] [stdout]     src/lib.rs - check_open_directories (line 479)
[INFO] [stdout]     src/lib.rs - download_media (line 179)
[INFO] [stdout]     src/lib.rs - extract_domain (line 364)
[INFO] [stdout]     src/lib.rs - extract_links (line 122)
[INFO] [stdout]     src/lib.rs - fetch_robots_txt (line 503)
[INFO] [stdout]     src/lib.rs - fetch_with_cookies (line 460)
[INFO] [stdout]     src/lib.rs - normalize_link (line 153)
[INFO] [stdout]     src/lib.rs - random_delay (line 599)
[INFO] [stdout]     src/lib.rs - random_user_agent (line 33)
[INFO] [stdout]     src/lib.rs - recursive_scrape (line 59)
[INFO] [stdout]     src/lib.rs - run (line 537)
[INFO] [stdout]     src/lib.rs - scrape_content (line 251)
[INFO] [stdout]     src/lib.rs - scrape_for_emails (line 422)
[INFO] [stdout]     src/lib.rs - scrape_for_errors (line 404)
[INFO] [stdout]     src/lib.rs - scrape_js (line 381)
[INFO] [stdout] 
[INFO] [stdout] test result: FAILED. 0 passed; 16 failed; 0 ignored; 0 measured; 0 filtered out; finished in 1.67s
[INFO] [stdout] 
[INFO] [stderr] error: doctest failed, to rerun pass `--doc`
[INFO] running `Command { std: "docker" "inspect" "71d80e539c46da220e53b6e114a578f9ec27ffc7f041f6b73ae69a8ab9be4579", kill_on_drop: false }`
[INFO] running `Command { std: "docker" "rm" "-f" "71d80e539c46da220e53b6e114a578f9ec27ffc7f041f6b73ae69a8ab9be4579", kill_on_drop: false }`
[INFO] [stdout] 71d80e539c46da220e53b6e114a578f9ec27ffc7f041f6b73ae69a8ab9be4579
