[INFO] cloning repository https://github.com/daniiialkhan/web-crawler
[INFO] running `Command { std: "git" "-c" "credential.helper=" "-c" "credential.helper=/workspace/cargo-home/bin/git-credential-null" "clone" "--bare" "https://github.com/daniiialkhan/web-crawler" "/workspace/cache/git-repos/https%3A%2F%2Fgithub.com%2Fdaniiialkhan%2Fweb-crawler", kill_on_drop: false }`
[INFO] [stderr] Cloning into bare repository '/workspace/cache/git-repos/https%3A%2F%2Fgithub.com%2Fdaniiialkhan%2Fweb-crawler'...
[INFO] running `Command { std: "git" "rev-parse" "HEAD", kill_on_drop: false }`
[INFO] [stdout] 1808fd05a109f0b5c787f8cb68df156625b29099
[INFO] checking daniiialkhan/web-crawler against 1.95.0 for pr-155915
[INFO] running `Command { std: "git" "clone" "/workspace/cache/git-repos/https%3A%2F%2Fgithub.com%2Fdaniiialkhan%2Fweb-crawler" "/workspace/builds/worker-5-tc1/source", kill_on_drop: false }`
[INFO] [stderr] Cloning into '/workspace/builds/worker-5-tc1/source'...
[INFO] [stderr] done.
[INFO] [stderr] Updating files:  51% (2185/4249)
Updating files:  52% (2210/4249)
Updating files:  53% (2252/4249)
Updating files:  54% (2295/4249)
Updating files:  55% (2337/4249)
Updating files:  56% (2380/4249)
Updating files:  57% (2422/4249)
Updating files:  58% (2465/4249)
Updating files:  59% (2507/4249)
Updating files:  60% (2550/4249)
Updating files:  61% (2592/4249)
Updating files:  61% (2593/4249)
Updating files:  62% (2635/4249)
Updating files:  62% (2639/4249)
Updating files:  63% (2677/4249)
Updating files:  64% (2720/4249)
Updating files:  65% (2762/4249)
Updating files:  66% (2805/4249)
Updating files:  66% (2831/4249)
Updating files:  66% (2845/4249)
Updating files:  67% (2847/4249)
Updating files:  67% (2885/4249)
Updating files:  68% (2890/4249)
Updating files:  69% (2932/4249)
Updating files:  69% (2967/4249)
Updating files:  70% (2975/4249)
Updating files:  70% (2994/4249)
Updating files:  71% (3017/4249)
Updating files:  72% (3060/4249)
Updating files:  72% (3081/4249)
Updating files:  73% (3102/4249)
Updating files:  74% (3145/4249)
Updating files:  75% (3187/4249)
Updating files:  76% (3230/4249)
Updating files:  77% (3272/4249)
Updating files:  78% (3315/4249)
Updating files:  79% (3357/4249)
Updating files:  80% (3400/4249)
Updating files:  80% (3424/4249)
Updating files:  81% (3442/4249)
Updating files:  82% (3485/4249)
Updating files:  83% (3527/4249)
Updating files:  84% (3570/4249)
Updating files:  85% (3612/4249)
Updating files:  86% (3655/4249)
Updating files:  87% (3697/4249)
Updating files:  88% (3740/4249)
Updating files:  89% (3782/4249)
Updating files:  90% (3825/4249)
Updating files:  91% (3867/4249)
Updating files:  92% (3910/4249)
Updating files:  93% (3952/4249)
Updating files:  94% (3995/4249)
Updating files:  95% (4037/4249)
Updating files:  96% (4080/4249)
Updating files:  96% (4115/4249)
Updating files:  97% (4122/4249)
Updating files:  98% (4165/4249)
Updating files:  99% (4207/4249)
Updating files: 100% (4249/4249)
Updating files: 100% (4249/4249), done.
[INFO] started tweaking git repo https://github.com/daniiialkhan/web-crawler
[INFO] finished tweaking git repo https://github.com/daniiialkhan/web-crawler
[INFO] tweaked toml for git repo https://github.com/daniiialkhan/web-crawler written to /workspace/builds/worker-5-tc1/source/Cargo.toml
[INFO] validating manifest of git repo https://github.com/daniiialkhan/web-crawler on toolchain 1.95.0
[INFO] running `Command { std: CARGO_HOME="/workspace/cargo-home" RUSTUP_HOME="/workspace/rustup-home" "/workspace/cargo-home/bin/cargo" "+1.95.0" "metadata" "--manifest-path" "Cargo.toml" "--no-deps", kill_on_drop: false }`
[INFO] crate git repo https://github.com/daniiialkhan/web-crawler already has a lockfile, it will not be regenerated
[INFO] running `Command { std: CARGO_HOME="/workspace/cargo-home" RUSTUP_HOME="/workspace/rustup-home" "/workspace/cargo-home/bin/cargo" "+1.95.0" "fetch" "--manifest-path" "Cargo.toml", kill_on_drop: false }`
[INFO] [stderr]     Updating crates.io index
[INFO] [stderr]  Downloading crates ...
[INFO] [stderr]   Downloaded robots_txt v0.7.0
[INFO] [stderr]   Downloaded scraper v0.17.1
[INFO] [stderr]   Downloaded tokio v1.33.0
[INFO] running `Command { std: "docker" "create" "-v" "/var/lib/crater-agent-workspace/builds/worker-5-tc1/target:/opt/rustwide/target:rw,Z" "-v" "/var/lib/crater-agent-workspace/builds/worker-5-tc1/source:/opt/rustwide/workdir:ro,Z" "-v" "/var/lib/crater-agent-workspace/cargo-home:/opt/rustwide/cargo-home:ro,Z" "-v" "/var/lib/crater-agent-workspace/rustup-home:/opt/rustwide/rustup-home:ro,Z" "-e" "SOURCE_DIR=/opt/rustwide/workdir" "-e" "CARGO_TARGET_DIR=/opt/rustwide/target" "-e" "CARGO_HOME=/opt/rustwide/cargo-home" "-e" "RUSTUP_HOME=/opt/rustwide/rustup-home" "-w" "/opt/rustwide/workdir" "-m" "1610612736" "--user" "0:0" "--network" "none" "ghcr.io/rust-lang/crates-build-env/linux@sha256:d429b63d4308055ea97f60fb1d3dfca48854a00942f1bd2ad806beaf015945ec" "/opt/rustwide/cargo-home/bin/cargo" "+1.95.0" "metadata" "--no-deps" "--format-version=1", kill_on_drop: false }`
[INFO] [stdout] b5e060973f73c3d2ba62b9383498bb6d7d6a0b7b59c1915809e8ca3209ac60d0
[INFO] running `Command { std: "docker" "start" "-a" "b5e060973f73c3d2ba62b9383498bb6d7d6a0b7b59c1915809e8ca3209ac60d0", kill_on_drop: false }`
[INFO] running `Command { std: "docker" "inspect" "b5e060973f73c3d2ba62b9383498bb6d7d6a0b7b59c1915809e8ca3209ac60d0", kill_on_drop: false }`
[INFO] running `Command { std: "docker" "rm" "-f" "b5e060973f73c3d2ba62b9383498bb6d7d6a0b7b59c1915809e8ca3209ac60d0", kill_on_drop: false }`
[INFO] [stdout] b5e060973f73c3d2ba62b9383498bb6d7d6a0b7b59c1915809e8ca3209ac60d0
[INFO] running `Command { std: "docker" "create" "-v" "/var/lib/crater-agent-workspace/builds/worker-5-tc1/target:/opt/rustwide/target:rw,Z" "-v" "/var/lib/crater-agent-workspace/builds/worker-5-tc1/source:/opt/rustwide/workdir:ro,Z" "-v" "/var/lib/crater-agent-workspace/cargo-home:/opt/rustwide/cargo-home:ro,Z" "-v" "/var/lib/crater-agent-workspace/rustup-home:/opt/rustwide/rustup-home:ro,Z" "-e" "SOURCE_DIR=/opt/rustwide/workdir" "-e" "CARGO_TARGET_DIR=/opt/rustwide/target" "-e" "CARGO_INCREMENTAL=0" "-e" "RUST_BACKTRACE=full" "-e" "RUSTFLAGS=--cap-lints=forbid" "-e" "RUSTDOCFLAGS=--cap-lints=forbid" "-e" "CARGO_HOME=/opt/rustwide/cargo-home" "-e" "RUSTUP_HOME=/opt/rustwide/rustup-home" "-w" "/opt/rustwide/workdir" "-m" "1610612736" "--user" "0:0" "--network" "none" "ghcr.io/rust-lang/crates-build-env/linux@sha256:d429b63d4308055ea97f60fb1d3dfca48854a00942f1bd2ad806beaf015945ec" "/opt/rustwide/cargo-home/bin/cargo" "+1.95.0" "check" "--frozen" "--all" "--all-targets" "--message-format=json", kill_on_drop: false }`
[INFO] [stdout] a049eac9e72c0deef2a3b8be34fd69a43d4e6e89d8d894b7be1776e34f7280ba
[INFO] running `Command { std: "docker" "start" "-a" "a049eac9e72c0deef2a3b8be34fd69a43d4e6e89d8d894b7be1776e34f7280ba", kill_on_drop: false }`
[INFO] [stderr]    Compiling libc v0.2.149
[INFO] [stderr]    Compiling proc-macro2 v1.0.69
[INFO] [stderr]    Compiling ppv-lite86 v0.2.17
[INFO] [stderr]     Checking once_cell v1.18.0
[INFO] [stderr]     Checking smallvec v1.11.1
[INFO] [stderr]    Compiling phf_shared v0.10.0
[INFO] [stderr]    Compiling serde v1.0.189
[INFO] [stderr]     Checking memchr v2.6.4
[INFO] [stderr]     Checking bitflags v2.4.1
[INFO] [stderr]     Checking new_debug_unreachable v1.0.4
[INFO] [stderr]    Compiling phf_shared v0.11.2
[INFO] [stderr]    Compiling openssl v0.10.57
[INFO] [stderr]     Checking http v0.2.9
[INFO] [stderr]     Checking mac v0.1.1
[INFO] [stderr]     Checking precomputed-hash v0.1.1
[INFO] [stderr]     Checking futf v0.1.5
[INFO] [stderr]     Checking tracing-core v0.1.32
[INFO] [stderr]     Checking dtoa v1.0.9
[INFO] [stderr]     Checking want v0.3.1
[INFO] [stderr]    Compiling ahash v0.8.3
[INFO] [stderr]    Compiling unicase v2.7.0
[INFO] [stderr]     Checking tendril v0.4.3
[INFO] [stderr]     Checking phf v0.10.1
[INFO] [stderr]     Checking dtoa-short v0.3.4
[INFO] [stderr]     Checking stable_deref_trait v1.2.0
[INFO] [stderr]     Checking url v2.4.1
[INFO] [stderr]     Checking getopts v0.2.21
[INFO] [stderr]     Checking encoding_rs v0.8.33
[INFO] [stderr]     Checking servo_arc v0.3.0
[INFO] [stderr]     Checking ego-tree v0.6.2
[INFO] [stderr]     Checking ipnet v2.8.0
[INFO] [stderr]     Checking futures-util v0.3.28
[INFO] [stderr]     Checking csv-core v0.1.11
[INFO] [stderr]     Checking tracing v0.1.40
[INFO] [stderr]     Checking http-body v0.4.5
[INFO] [stderr]    Compiling quote v1.0.33
[INFO] [stderr]     Checking robots_txt v0.7.0
[INFO] [stderr]     Checking parking_lot_core v0.9.9
[INFO] [stderr]     Checking num_cpus v1.16.0
[INFO] [stderr]     Checking socket2 v0.5.4
[INFO] [stderr]     Checking mio v0.8.8
[INFO] [stderr]     Checking signal-hook-registry v1.4.1
[INFO] [stderr]     Checking socket2 v0.4.9
[INFO] [stderr]     Checking getrandom v0.2.10
[INFO] [stderr]    Compiling syn v2.0.38
[INFO] [stderr]    Compiling syn v1.0.109
[INFO] [stderr]    Compiling cc v1.0.83
[INFO] [stderr]     Checking parking_lot v0.12.1
[INFO] [stderr]    Compiling rand_core v0.6.4
[INFO] [stderr]    Compiling rand_chacha v0.3.1
[INFO] [stderr]    Compiling openssl-sys v0.9.93
[INFO] [stderr]    Compiling rand v0.8.5
[INFO] [stderr]    Compiling native-tls v0.2.11
[INFO] [stderr]    Compiling phf_generator v0.10.0
[INFO] [stderr]    Compiling phf_generator v0.11.2
[INFO] [stderr]    Compiling phf_codegen v0.10.0
[INFO] [stderr]    Compiling string_cache_codegen v0.5.2
[INFO] [stderr]    Compiling selectors v0.25.0
[INFO] [stderr]    Compiling markup5ever v0.11.0
[INFO] [stderr]     Checking string_cache v0.8.7
[INFO] [stderr]     Checking serde_urlencoded v0.7.1
[INFO] [stderr]     Checking csv v1.3.0
[INFO] [stderr]    Compiling tokio-macros v2.1.0
[INFO] [stderr]    Compiling openssl-macros v0.1.1
[INFO] [stderr]    Compiling phf_macros v0.11.2
[INFO] [stderr]    Compiling cssparser-macros v0.6.1
[INFO] [stderr]     Checking phf v0.11.2
[INFO] [stderr]     Checking tokio v1.33.0
[INFO] [stderr]     Checking cssparser v0.31.2
[INFO] [stderr]    Compiling html5ever v0.26.0
[INFO] [stderr]    Compiling derive_more v0.99.17
[INFO] [stderr]     Checking scraper v0.17.1
[INFO] [stderr]     Checking tokio-util v0.7.9
[INFO] [stderr]     Checking tokio-native-tls v0.3.1
[INFO] [stderr]     Checking h2 v0.3.21
[INFO] [stderr]     Checking hyper v0.14.27
[INFO] [stderr]     Checking hyper-tls v0.5.0
[INFO] [stderr]     Checking reqwest v0.11.22
[INFO] [stderr]     Checking web-crawler v0.1.0 (/opt/rustwide/workdir)
[INFO] [stdout] warning: unused import: `parse`
[INFO] [stdout]   --> src/crawler.rs:11:51
[INFO] [stdout]    |
[INFO] [stdout] 11 |     use robots_txt::{Robots, matcher::SimpleMatcher, parse};
[INFO] [stdout]    |                                                      ^^^^^
[INFO] [stdout]    |
[INFO] [stdout]    = note: `#[warn(unused_imports)]` (part of `#[warn(unused)]`) on by default
[INFO] [stdout] 
[INFO] [stdout] 
[INFO] [stdout] warning: unused import: `header::HeaderValue`
[INFO] [stdout]   --> src/crawler.rs:12:22
[INFO] [stdout]    |
[INFO] [stdout] 12 |     use reqwest::{self, header::HeaderValue, Client};
[INFO] [stdout]    |                         ^^^^^^^^^^^^^^^^^^^
[INFO] [stdout] 
[INFO] [stdout] 
[INFO] [stdout] warning: unused import: `Response`
[INFO] [stdout]   --> src/crawler.rs:15:16
[INFO] [stdout]    |
[INFO] [stdout] 15 |     use reqwest::{Response, Request};
[INFO] [stdout]    |                   ^^^^^^^^
[INFO] [stdout] 
[INFO] [stdout] 
[INFO] [stdout] warning: unused import: `parse`
[INFO] [stdout]   --> src/crawler.rs:11:51
[INFO] [stdout]    |
[INFO] [stdout] 11 |     use robots_txt::{Robots, matcher::SimpleMatcher, parse};
[INFO] [stdout]    |                                                      ^^^^^
[INFO] [stdout]    |
[INFO] [stdout]    = note: `#[warn(unused_imports)]` (part of `#[warn(unused)]`) on by default
[INFO] [stdout] 
[INFO] [stdout] 
[INFO] [stdout] warning: unused import: `header::HeaderValue`
[INFO] [stdout]   --> src/crawler.rs:12:22
[INFO] [stdout]    |
[INFO] [stdout] 12 |     use reqwest::{self, header::HeaderValue, Client};
[INFO] [stdout]    |                         ^^^^^^^^^^^^^^^^^^^
[INFO] [stdout] 
[INFO] [stdout] 
[INFO] [stdout] warning: unused import: `Response`
[INFO] [stdout]   --> src/crawler.rs:15:16
[INFO] [stdout]    |
[INFO] [stdout] 15 |     use reqwest::{Response, Request};
[INFO] [stdout]    |                   ^^^^^^^^
[INFO] [stdout] 
[INFO] [stdout] 
[INFO] [stdout] warning: variable `k` is assigned to, but never used
[INFO] [stdout]    --> src/crawler.rs:226:9
[INFO] [stdout]     |
[INFO] [stdout] 226 |                 let mut k = 1;
[INFO] [stdout]     |                     ^^^^^
[INFO] [stdout]     |
[INFO] [stdout]     = note: consider using `_k` instead
[INFO] [stdout]     = note: `#[warn(unused_variables)]` (part of `#[warn(unused)]`) on by default
[INFO] [stdout] 
[INFO] [stdout] 
[INFO] [stdout] warning: value assigned to `k` is never read
[INFO] [stdout]    --> src/crawler.rs:237:6
[INFO] [stdout]     |
[INFO] [stdout] 237 |                     k += 1;
[INFO] [stdout]     |                     ^^^^^^
[INFO] [stdout]     |
[INFO] [stdout]     = help: maybe it is overwritten before being read?
[INFO] [stdout]     = note: `#[warn(unused_assignments)]` (part of `#[warn(unused)]`) on by default
[INFO] [stdout] 
[INFO] [stdout] 
[INFO] [stdout] warning: value assigned to `filepath` is never read
[INFO] [stdout]   --> src/links_and_files.rs:48:24
[INFO] [stdout]    |
[INFO] [stdout] 48 |                 let mut filepath = "".to_string();
[INFO] [stdout]    |                                    ^^^^^^^^^^^^^^
[INFO] [stdout]    |
[INFO] [stdout]    = help: maybe it is overwritten before being read?
[INFO] [stdout] 
[INFO] [stdout] 
[INFO] [stdout] warning: field `domain` is never read
[INFO] [stdout]   --> src/crawler.rs:25:7
[INFO] [stdout]    |
[INFO] [stdout] 21 |     pub struct Crawler<'z> {
[INFO] [stdout]    |                ------- field in this struct
[INFO] [stdout] ...
[INFO] [stdout] 25 |         pub domain: &'z str,                // to store domain name of the website (stays till lifetime exists= meaning till the str...
[INFO] [stdout]    |             ^^^^^^
[INFO] [stdout]    |
[INFO] [stdout]    = note: `#[warn(dead_code)]` (part of `#[warn(unused)]`) on by default
[INFO] [stdout] 
[INFO] [stdout] 
[INFO] [stdout] warning: method `disallowed_links` is never used
[INFO] [stdout]   --> src/crawler.rs:41:10
[INFO] [stdout]    |
[INFO] [stdout] 27 |     impl<'z> Crawler<'z> {
[INFO] [stdout]    |     -------------------- method in this implementation
[INFO] [stdout] ...
[INFO] [stdout] 41 |         pub fn disallowed_links(&self) -> &Vec<String> {
[INFO] [stdout]    |                ^^^^^^^^^^^^^^^^
[INFO] [stdout] 
[INFO] [stdout] 
[INFO] [stdout] warning: variable `k` is assigned to, but never used
[INFO] [stdout]    --> src/crawler.rs:226:9
[INFO] [stdout]     |
[INFO] [stdout] 226 |                 let mut k = 1;
[INFO] [stdout]     |                     ^^^^^
[INFO] [stdout]     |
[INFO] [stdout]     = note: consider using `_k` instead
[INFO] [stdout]     = note: `#[warn(unused_variables)]` (part of `#[warn(unused)]`) on by default
[INFO] [stdout] 
[INFO] [stdout] 
[INFO] [stdout] warning: value assigned to `k` is never read
[INFO] [stdout]    --> src/crawler.rs:237:6
[INFO] [stdout]     |
[INFO] [stdout] 237 |                     k += 1;
[INFO] [stdout]     |                     ^^^^^^
[INFO] [stdout]     |
[INFO] [stdout]     = help: maybe it is overwritten before being read?
[INFO] [stdout]     = note: `#[warn(unused_assignments)]` (part of `#[warn(unused)]`) on by default
[INFO] [stdout] 
[INFO] [stdout] 
[INFO] [stdout] warning: value assigned to `filepath` is never read
[INFO] [stdout]   --> src/links_and_files.rs:48:24
[INFO] [stdout]    |
[INFO] [stdout] 48 |                 let mut filepath = "".to_string();
[INFO] [stdout]    |                                    ^^^^^^^^^^^^^^
[INFO] [stdout]    |
[INFO] [stdout]    = help: maybe it is overwritten before being read?
[INFO] [stdout] 
[INFO] [stdout] 
[INFO] [stdout] warning: field `domain` is never read
[INFO] [stdout]   --> src/crawler.rs:25:7
[INFO] [stdout]    |
[INFO] [stdout] 21 |     pub struct Crawler<'z> {
[INFO] [stdout]    |                ------- field in this struct
[INFO] [stdout] ...
[INFO] [stdout] 25 |         pub domain: &'z str,                // to store domain name of the website (stays till lifetime exists= meaning till the str...
[INFO] [stdout]    |             ^^^^^^
[INFO] [stdout]    |
[INFO] [stdout]    = note: `#[warn(dead_code)]` (part of `#[warn(unused)]`) on by default
[INFO] [stdout] 
[INFO] [stdout] 
[INFO] [stdout] warning: method `disallowed_links` is never used
[INFO] [stdout]   --> src/crawler.rs:41:10
[INFO] [stdout]    |
[INFO] [stdout] 27 |     impl<'z> Crawler<'z> {
[INFO] [stdout]    |     -------------------- method in this implementation
[INFO] [stdout] ...
[INFO] [stdout] 41 |         pub fn disallowed_links(&self) -> &Vec<String> {
[INFO] [stdout]    |                ^^^^^^^^^^^^^^^^
[INFO] [stdout] 
[INFO] [stdout] 
[INFO] [stderr]     Finished `dev` profile [unoptimized + debuginfo] target(s) in 33.51s
[INFO] running `Command { std: "docker" "inspect" "a049eac9e72c0deef2a3b8be34fd69a43d4e6e89d8d894b7be1776e34f7280ba", kill_on_drop: false }`
[INFO] running `Command { std: "docker" "rm" "-f" "a049eac9e72c0deef2a3b8be34fd69a43d4e6e89d8d894b7be1776e34f7280ba", kill_on_drop: false }`
[INFO] [stdout] a049eac9e72c0deef2a3b8be34fd69a43d4e6e89d8d894b7be1776e34f7280ba
