[INFO] cloning repository https://github.com/klion26/build-a-large-language-model [INFO] running `Command { std: "git" "-c" "credential.helper=" "-c" "credential.helper=/workspace/cargo-home/bin/git-credential-null" "clone" "--bare" "https://github.com/klion26/build-a-large-language-model" "/workspace/cache/git-repos/https%3A%2F%2Fgithub.com%2Fklion26%2Fbuild-a-large-language-model", kill_on_drop: false }` [INFO] [stderr] Cloning into bare repository '/workspace/cache/git-repos/https%3A%2F%2Fgithub.com%2Fklion26%2Fbuild-a-large-language-model'... [INFO] [stderr] /workspace/cache/git-repos/https%3A%2F%2Fgithub.com%2Fklion26%2Fbuild-a-large-language-model/branches/: No space left on device [WARN] Retrying crate fetch in 3 seconds (attempt 1) [INFO] cloning repository https://github.com/klion26/build-a-large-language-model [INFO] running `Command { std: "git" "-c" "credential.helper=" "-c" "credential.helper=/workspace/cargo-home/bin/git-credential-null" "clone" "--bare" "https://github.com/klion26/build-a-large-language-model" "/workspace/cache/git-repos/https%3A%2F%2Fgithub.com%2Fklion26%2Fbuild-a-large-language-model", kill_on_drop: false }` [INFO] [stderr] Cloning into bare repository '/workspace/cache/git-repos/https%3A%2F%2Fgithub.com%2Fklion26%2Fbuild-a-large-language-model'... [INFO] [stderr] /workspace/cache/git-repos/https%3A%2F%2Fgithub.com%2Fklion26%2Fbuild-a-large-language-model: No space left on device [WARN] Retrying crate fetch in 3 seconds (attempt 2) [INFO] cloning repository https://github.com/klion26/build-a-large-language-model [INFO] running `Command { std: "git" "-c" "credential.helper=" "-c" "credential.helper=/workspace/cargo-home/bin/git-credential-null" "clone" "--bare" "https://github.com/klion26/build-a-large-language-model" "/workspace/cache/git-repos/https%3A%2F%2Fgithub.com%2Fklion26%2Fbuild-a-large-language-model", kill_on_drop: false }` [INFO] [stderr] Cloning into bare repository '/workspace/cache/git-repos/https%3A%2F%2Fgithub.com%2Fklion26%2Fbuild-a-large-language-model'... [INFO] [stderr] /workspace/cache/git-repos/https%3A%2F%2Fgithub.com%2Fklion26%2Fbuild-a-large-language-model: No space left on device [WARN] Retrying crate fetch in 3 seconds (attempt 3) [INFO] cloning repository https://github.com/klion26/build-a-large-language-model [INFO] running `Command { std: "git" "-c" "credential.helper=" "-c" "credential.helper=/workspace/cargo-home/bin/git-credential-null" "clone" "--bare" "https://github.com/klion26/build-a-large-language-model" "/workspace/cache/git-repos/https%3A%2F%2Fgithub.com%2Fklion26%2Fbuild-a-large-language-model", kill_on_drop: false }` [INFO] [stderr] Cloning into bare repository '/workspace/cache/git-repos/https%3A%2F%2Fgithub.com%2Fklion26%2Fbuild-a-large-language-model'... [INFO] [stderr] /workspace/cache/git-repos/https%3A%2F%2Fgithub.com%2Fklion26%2Fbuild-a-large-language-model: No space left on device [WARN] Retrying crate fetch in 3 seconds (attempt 4) [INFO] cloning repository https://github.com/klion26/build-a-large-language-model [INFO] running `Command { std: "git" "-c" "credential.helper=" "-c" "credential.helper=/workspace/cargo-home/bin/git-credential-null" "clone" "--bare" "https://github.com/klion26/build-a-large-language-model" "/workspace/cache/git-repos/https%3A%2F%2Fgithub.com%2Fklion26%2Fbuild-a-large-language-model", kill_on_drop: false }` [INFO] [stderr] Cloning into bare repository '/workspace/cache/git-repos/https%3A%2F%2Fgithub.com%2Fklion26%2Fbuild-a-large-language-model'... [INFO] [stderr] /workspace/cache/git-repos/https%3A%2F%2Fgithub.com%2Fklion26%2Fbuild-a-large-language-model: No space left on device [WARN] Retrying crate fetch in 3 seconds (attempt 5) [INFO] cloning repository https://github.com/klion26/build-a-large-language-model [INFO] running `Command { std: "git" "-c" "credential.helper=" "-c" "credential.helper=/workspace/cargo-home/bin/git-credential-null" "clone" "--bare" "https://github.com/klion26/build-a-large-language-model" "/workspace/cache/git-repos/https%3A%2F%2Fgithub.com%2Fklion26%2Fbuild-a-large-language-model", kill_on_drop: false }` [INFO] [stderr] Cloning into bare repository '/workspace/cache/git-repos/https%3A%2F%2Fgithub.com%2Fklion26%2Fbuild-a-large-language-model'... [INFO] [stderr] /workspace/cache/git-repos/https%3A%2F%2Fgithub.com%2Fklion26%2Fbuild-a-large-language-model: No space left on device [WARN] Retrying crate fetch in 3 seconds (attempt 6) [INFO] cloning repository https://github.com/klion26/build-a-large-language-model [INFO] running `Command { std: "git" "-c" "credential.helper=" "-c" "credential.helper=/workspace/cargo-home/bin/git-credential-null" "clone" "--bare" "https://github.com/klion26/build-a-large-language-model" "/workspace/cache/git-repos/https%3A%2F%2Fgithub.com%2Fklion26%2Fbuild-a-large-language-model", kill_on_drop: false }` [INFO] [stderr] Cloning into bare repository '/workspace/cache/git-repos/https%3A%2F%2Fgithub.com%2Fklion26%2Fbuild-a-large-language-model'... [INFO] [stderr] /workspace/cache/git-repos/https%3A%2F%2Fgithub.com%2Fklion26%2Fbuild-a-large-language-model: No space left on device [WARN] Retrying crate fetch in 3 seconds (attempt 7) [INFO] cloning repository https://github.com/klion26/build-a-large-language-model [INFO] running `Command { std: "git" "-c" "credential.helper=" "-c" "credential.helper=/workspace/cargo-home/bin/git-credential-null" "clone" "--bare" "https://github.com/klion26/build-a-large-language-model" "/workspace/cache/git-repos/https%3A%2F%2Fgithub.com%2Fklion26%2Fbuild-a-large-language-model", kill_on_drop: false }` [INFO] [stderr] Cloning into bare repository '/workspace/cache/git-repos/https%3A%2F%2Fgithub.com%2Fklion26%2Fbuild-a-large-language-model'... [INFO] [stderr] /workspace/cache/git-repos/https%3A%2F%2Fgithub.com%2Fklion26%2Fbuild-a-large-language-model: No space left on device [WARN] Retrying crate fetch in 3 seconds (attempt 8) [INFO] cloning repository https://github.com/klion26/build-a-large-language-model [INFO] running `Command { std: "git" "-c" "credential.helper=" "-c" "credential.helper=/workspace/cargo-home/bin/git-credential-null" "clone" "--bare" "https://github.com/klion26/build-a-large-language-model" "/workspace/cache/git-repos/https%3A%2F%2Fgithub.com%2Fklion26%2Fbuild-a-large-language-model", kill_on_drop: false }` [INFO] [stderr] Cloning into bare repository '/workspace/cache/git-repos/https%3A%2F%2Fgithub.com%2Fklion26%2Fbuild-a-large-language-model'... [INFO] [stderr] /workspace/cache/git-repos/https%3A%2F%2Fgithub.com%2Fklion26%2Fbuild-a-large-language-model: No space left on device [WARN] Retrying crate fetch in 3 seconds (attempt 9) [INFO] cloning repository https://github.com/klion26/build-a-large-language-model [INFO] running `Command { std: "git" "-c" "credential.helper=" "-c" "credential.helper=/workspace/cargo-home/bin/git-credential-null" "clone" "--bare" "https://github.com/klion26/build-a-large-language-model" "/workspace/cache/git-repos/https%3A%2F%2Fgithub.com%2Fklion26%2Fbuild-a-large-language-model", kill_on_drop: false }` [INFO] [stderr] Cloning into bare repository '/workspace/cache/git-repos/https%3A%2F%2Fgithub.com%2Fklion26%2Fbuild-a-large-language-model'... [INFO] [stderr] /workspace/cache/git-repos/https%3A%2F%2Fgithub.com%2Fklion26%2Fbuild-a-large-language-model: No space left on device [WARN] Retrying crate fetch in 3 seconds (attempt 10) [INFO] cloning repository https://github.com/klion26/build-a-large-language-model [INFO] running `Command { std: "git" "-c" "credential.helper=" "-c" "credential.helper=/workspace/cargo-home/bin/git-credential-null" "clone" "--bare" "https://github.com/klion26/build-a-large-language-model" "/workspace/cache/git-repos/https%3A%2F%2Fgithub.com%2Fklion26%2Fbuild-a-large-language-model", kill_on_drop: false }` [INFO] [stderr] Cloning into bare repository '/workspace/cache/git-repos/https%3A%2F%2Fgithub.com%2Fklion26%2Fbuild-a-large-language-model'... [INFO] [stderr] /workspace/cache/git-repos/https%3A%2F%2Fgithub.com%2Fklion26%2Fbuild-a-large-language-model: No space left on device [WARN] Retrying crate fetch in 3 seconds (attempt 11) [INFO] cloning repository https://github.com/klion26/build-a-large-language-model [INFO] running `Command { std: "git" "-c" "credential.helper=" "-c" "credential.helper=/workspace/cargo-home/bin/git-credential-null" "clone" "--bare" "https://github.com/klion26/build-a-large-language-model" "/workspace/cache/git-repos/https%3A%2F%2Fgithub.com%2Fklion26%2Fbuild-a-large-language-model", kill_on_drop: false }` [INFO] [stderr] Cloning into bare repository '/workspace/cache/git-repos/https%3A%2F%2Fgithub.com%2Fklion26%2Fbuild-a-large-language-model'... [INFO] [stderr] /workspace/cache/git-repos/https%3A%2F%2Fgithub.com%2Fklion26%2Fbuild-a-large-language-model: No space left on device [WARN] Retrying crate fetch in 3 seconds (attempt 12) [INFO] cloning repository https://github.com/klion26/build-a-large-language-model [INFO] running `Command { std: "git" "-c" "credential.helper=" "-c" "credential.helper=/workspace/cargo-home/bin/git-credential-null" "clone" "--bare" "https://github.com/klion26/build-a-large-language-model" "/workspace/cache/git-repos/https%3A%2F%2Fgithub.com%2Fklion26%2Fbuild-a-large-language-model", kill_on_drop: false }` [INFO] [stderr] Cloning into bare repository '/workspace/cache/git-repos/https%3A%2F%2Fgithub.com%2Fklion26%2Fbuild-a-large-language-model'... [INFO] running `Command { std: "git" "rev-parse" "HEAD", kill_on_drop: false }` [INFO] [stdout] 38afb6e9aba81bad2b8aa6afa1f0344b355adc56 [INFO] checking klion26/build-a-large-language-model against try#15e8bc6907ab2b3623cb52acc726f4787458df64 for pr-141352 [INFO] running `Command { std: "git" "clone" "/workspace/cache/git-repos/https%3A%2F%2Fgithub.com%2Fklion26%2Fbuild-a-large-language-model" "/workspace/builds/worker-5-tc2/source", kill_on_drop: false }` [INFO] [stderr] Cloning into '/workspace/builds/worker-5-tc2/source'... [INFO] [stderr] done. [INFO] validating manifest of git repo https://github.com/klion26/build-a-large-language-model on toolchain 15e8bc6907ab2b3623cb52acc726f4787458df64 [INFO] running `Command { std: CARGO_HOME="/workspace/cargo-home" RUSTUP_HOME="/workspace/rustup-home" "/workspace/cargo-home/bin/cargo" "+15e8bc6907ab2b3623cb52acc726f4787458df64" "metadata" "--manifest-path" "Cargo.toml" "--no-deps", kill_on_drop: false }` [INFO] started tweaking git repo https://github.com/klion26/build-a-large-language-model [INFO] finished tweaking git repo https://github.com/klion26/build-a-large-language-model [INFO] tweaked toml for git repo https://github.com/klion26/build-a-large-language-model written to /workspace/builds/worker-5-tc2/source/Cargo.toml [INFO] crate git repo https://github.com/klion26/build-a-large-language-model already has a lockfile, it will not be regenerated [INFO] running `Command { std: CARGO_HOME="/workspace/cargo-home" RUSTUP_HOME="/workspace/rustup-home" "/workspace/cargo-home/bin/cargo" "+15e8bc6907ab2b3623cb52acc726f4787458df64" "fetch" "--manifest-path" "Cargo.toml", kill_on_drop: false }` [INFO] [stderr] Updating git repository `https://github.com/huggingface/candle.git` [INFO] [stderr] warning: spurious network error (3 tries remaining): cannot extend packfile '/workspace/cargo-home/git/db/candle-5b4d092929d18d36/objects/pack/pack_git2_99520a1efb9a68d1': No space left on device; class=Os (2) [INFO] [stderr] warning: spurious network error (2 tries remaining): cannot extend packfile '/workspace/cargo-home/git/db/candle-5b4d092929d18d36/objects/pack/pack_git2_0a77e750fd14206b': No space left on device; class=Os (2) [INFO] [stderr] warning: spurious network error (1 try remaining): cannot extend packfile '/workspace/cargo-home/git/db/candle-5b4d092929d18d36/objects/pack/pack_git2_a5bd85c94121af7d': No space left on device; class=Os (2) [INFO] [stderr] Updating git submodule `https://github.com/NVIDIA/cutlass.git` [INFO] [stderr] error: failed to get `candle-core` as a dependency of package `build-a-large-language-model v0.1.0 (/workspace/builds/worker-5-tc2/source)` [INFO] [stderr] [INFO] [stderr] Caused by: [INFO] [stderr] failed to load source for dependency `candle-core` [INFO] [stderr] [INFO] [stderr] Caused by: [INFO] [stderr] Unable to update https://github.com/huggingface/candle.git#99549813 [INFO] [stderr] [INFO] [stderr] Caused by: [INFO] [stderr] failed to update submodule `candle-examples/examples/flash-attn/cutlass` [INFO] [stderr] [INFO] [stderr] Caused by: [INFO] [stderr] could not write to '/workspace/cargo-home/git/checkouts/candle-5b4d092929d18d36/9954981/candle-flash-attn/cutlass/include/cutlass/gemm/warp/mma.h': No space left on device; class=Os (2) [INFO] checking klion26/build-a-large-language-model against try#15e8bc6907ab2b3623cb52acc726f4787458df64 for pr-141352 [INFO] running `Command { std: "git" "clone" "/workspace/cache/git-repos/https%3A%2F%2Fgithub.com%2Fklion26%2Fbuild-a-large-language-model" "/workspace/builds/worker-5-tc2/source", kill_on_drop: false }` [INFO] [stderr] Cloning into '/workspace/builds/worker-5-tc2/source'... [INFO] [stderr] done. [INFO] [stderr] fatal: sha1 file '/workspace/builds/worker-5-tc2/source/.git/index.lock' write error. Out of diskspace [INFO] [stderr] warning: Clone succeeded, but checkout failed. [INFO] [stderr] You can inspect what was checked out with 'git status' [INFO] [stderr] and retry with 'git restore --source=HEAD :/' [INFO] [stderr] [INFO] checking klion26/build-a-large-language-model against try#15e8bc6907ab2b3623cb52acc726f4787458df64 for pr-141352 [INFO] running `Command { std: "git" "clone" "/workspace/cache/git-repos/https%3A%2F%2Fgithub.com%2Fklion26%2Fbuild-a-large-language-model" "/workspace/builds/worker-5-tc2/source", kill_on_drop: false }` [INFO] [stderr] Cloning into '/workspace/builds/worker-5-tc2/source'... [INFO] [stderr] done. [INFO] [stderr] fatal: sha1 file '/workspace/builds/worker-5-tc2/source/.git/index.lock' write error. Out of diskspace [INFO] [stderr] warning: Clone succeeded, but checkout failed. [INFO] [stderr] You can inspect what was checked out with 'git status' [INFO] [stderr] and retry with 'git restore --source=HEAD :/' [INFO] [stderr] [INFO] checking klion26/build-a-large-language-model against try#15e8bc6907ab2b3623cb52acc726f4787458df64 for pr-141352 [INFO] running `Command { std: "git" "clone" "/workspace/cache/git-repos/https%3A%2F%2Fgithub.com%2Fklion26%2Fbuild-a-large-language-model" "/workspace/builds/worker-5-tc2/source", kill_on_drop: false }` [INFO] [stderr] Cloning into '/workspace/builds/worker-5-tc2/source'... [INFO] [stderr] done. [INFO] [stderr] error: unable to write file Cargo.lock [INFO] [stderr] error: unable to write file Cargo.toml [INFO] [stderr] error: unable to write file README.md [INFO] [stderr] fatal: cannot create directory at 'data': No space left on device [INFO] [stderr] warning: Clone succeeded, but checkout failed. [INFO] [stderr] You can inspect what was checked out with 'git status' [INFO] [stderr] and retry with 'git restore --source=HEAD :/' [INFO] [stderr] [INFO] checking klion26/build-a-large-language-model against try#15e8bc6907ab2b3623cb52acc726f4787458df64 for pr-141352 [INFO] running `Command { std: "git" "clone" "/workspace/cache/git-repos/https%3A%2F%2Fgithub.com%2Fklion26%2Fbuild-a-large-language-model" "/workspace/builds/worker-5-tc2/source", kill_on_drop: false }` [INFO] [stderr] Cloning into '/workspace/builds/worker-5-tc2/source'... [INFO] [stderr] done. [INFO] [stderr] error: unable to write file Cargo.lock [INFO] [stderr] error: unable to write file Cargo.toml [INFO] [stderr] error: unable to write file README.md [INFO] [stderr] fatal: cannot create directory at 'data': No space left on device [INFO] [stderr] warning: Clone succeeded, but checkout failed. [INFO] [stderr] You can inspect what was checked out with 'git status' [INFO] [stderr] and retry with 'git restore --source=HEAD :/' [INFO] [stderr] failed to checkout https://github.com/klion26/build-a-large-language-model Caused by: command failed: exit status: 128